Commit Graph

46 Commits

Author SHA1 Message Date
Anders Kaseorg 684dad8145 tools: Use root-based absolute import for tools.lib, etc.
Mypy can’t follow absolute imports based on directories other than the
root.  This was hiding some type errors due to ignore_missing_imports.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 12:21:52 -07:00
PIG208 540072e860 tools: Support sending mock messages for non-webhook integrations.
Non-webhook integrations should have fixtures containing mock messages
in json format with fields "subject" and "body" indicating the topic
and content respectively.
2021-05-10 17:50:32 -07:00
Tim Abbott 2d9631191e tools: Fix mypy errors with generate-integration-docs-screenshot.
3f4d0f72fd adjusted the types in
preparation for extending the functionality, but only in later commits
that are not merged yet did it make these updates to the types.
2021-04-28 08:47:38 -07:00
PIG208 3f4d0f72fd tools: Create get_fixture_info to support non-webhook integrations. 2021-04-28 08:16:51 -07:00
Adam Birds 3b974d9ef7 tools: Add support for generating integration screenshots remotely.
I have added support for generating integration screenshots remotely by
adding a `realm_uri` parameter to `tools/message-screenshot.js` which we
then pass `realm.uri` to from within
`tools/generate-integration-docs-screenshot`.
2021-04-19 10:44:43 -07:00
shanukun 0bf067b681 refactor: Make acting_user a mandatory kwarg for bulk_add_subscriptions. 2021-04-02 14:44:41 -07:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Steve Howell a9356508ca events: Stop sending occupy/vacate events.
We used to send occupy/vacate events when
either the first person entered a stream
or the last person exited.

It appears that our two main apps have never
looked at these events.  Instead, it's
generally the case that clients handle
events related to stream creation/deactivation
and subscribe/unsubscribe.

Note that we removed the apply_events code
related to these events.  This doesn't affect
the webapp, because the webapp doesn't care
about the "streams" field in do_events_register.

There is a theoretical situation where a
third party client could be the victim of
a race where the "streams" data includes
a stream where the last subscriber has left.
I suspect in most of those situations it
will be harmless, or possibly even helpful
to the extent that they'll learn about
streams that are in a "quasi" state where
they're activated but not occupied.

We could try to patch apply_event to
detect when subscriptions get added
or removed. Or we could just make the
"streams" piece of do_events_register
not care about occupy/vacate semantics.
I favor the latter, since it might
actually be what users what, and it will
also simplify the code and improve
performance.
2020-10-14 10:53:10 -07:00
Steve Howell 9df9934ed6 refactor: Pass realm to bulk_add_subscriptions.
I think it's important that the callers understand
that bulk_add_subscriptions assumes all streams
are being created within a single realm, so I make
it an explicit parameter.

This may be overkill--I would also be happy if we
just included the assertions from this commit.
2020-10-13 11:28:17 -07:00
Anders Kaseorg a50fae89e2 python: Elide type=str from argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg b4597a8ca8 python: Elide default for store_{true,false} argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg d0f4af5f8c python: Catch JSONDecodeError instead of ValueError when decoding JSON.
These weren’t wrong since orjson.JSONDecodeError subclasses
json.JSONDecodeError which subclasses ValueError, but the more
specific ones express the intention more clearly.

(ujson raised ValueError directly, as did json in Python 2.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 11:59:59 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Steve Howell c44500175d database: Remove short_name from UserProfile.
A few major themes here:

    - We remove short_name from UserProfile
      and add the appropriate migration.

    - We remove short_name from various
      cache-related lists of fields.

    - We allow import tools to continue to
      write short_name to their export files,
      and then we simply ignore the field
      at import time.

    - We change functions like do_create_user,
      create_user_profile, etc.

    - We keep short_name in the /json/bots
      API.  (It actually gets turned into
      an email.)

    - We don't modify our LDAP code much
      here.
2020-07-17 11:15:15 -07:00
arpit551 01f12b9fc2 audit_log: Log acting_user in user creation and user activation. 2020-07-06 17:32:09 -07:00
arpit551 653928bdfe audit_log: Log acting_user in do_change_avatar_fields. 2020-07-06 17:24:18 -07:00
Anders Kaseorg 91a86c24f5 python: Replace None defaults with empty collections where appropriate.
Use read-only types (List ↦ Sequence, Dict ↦ Mapping, Set ↦
AbstractSet) to guard against accidental mutation of the default
value.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Puneeth Chaganti d2dadde381 tools: Use the correct path for bot avatar to generate screenshots. 2020-06-01 18:08:23 -07:00
Puneeth Chaganti 29f981c836 tools: Allow choosing one integration from the screenshot config. 2020-06-01 18:08:22 -07:00
Puneeth Chaganti 752fe58f4f tools: Allow specifying & capturing screenshots for all integrations. 2020-06-01 18:08:22 -07:00
Puneeth Chaganti 04b2653364 tools: Allow generating screenshot without fixture file.
All the data is passed as URL parameters, and no fixture file is used.
2020-06-01 18:03:27 -07:00
Puneeth Chaganti ce51c6ade9 tools: Generate screenshots for integrations with txt fixtures. 2020-06-01 18:03:27 -07:00
Puneeth Chaganti b533420d3b tools: Allow specifying extra query params in screenshot config. 2020-06-01 18:03:27 -07:00
Puneeth Chaganti 56c31a8647 tools: Allow specifying the bot name for notifications. 2020-06-01 18:03:27 -07:00
Puneeth Chaganti 55444c33a0 tools: Allow passing payload as a query param. 2020-06-01 18:03:27 -07:00
Puneeth Chaganti d4c6195bca tools: Allow specifying use of basic auth in ScreenshotConfig. 2020-06-01 18:03:27 -07:00
Puneeth Chaganti 5ffc51382d tools: Allow specifying custom headers in ScreenshotConfig. 2020-06-01 18:03:27 -07:00
Puneeth Chaganti 80e432b466 tools: Move screenshot captured message to Python. 2020-04-28 13:00:09 -07:00
Puneeth Chaganti 88c297dffe tools: Allow specifying more options when capturing screenshot.
The name and directory into which the screenshot should be captured can be
specified now, apart from specifying the fixture file to use.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti 4e5c30e7fd tools: Pass image path as an argument to the screenshot tool.
Instead of figuring out the image path from the integration name in the
puppeteer script, we do it in the `generate-integration-docs-screenshot`
script and pass it as an argument to `message-screenshot.js`.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti 39bea656f7 generate-integration-docs-screenshot: Remove call to sys.exit.
Preparatory refactor to allow calling the function in a loop, to generate
screenshots for all integrations.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti ac7d8ce4b3 generate-integration-docs-screenshot: Notify on webhook's stream.
Use the stream specified by `stream_name` on the WebhookIntegration, instead
of using a common stream to send messages.
2020-04-28 13:00:09 -07:00
Puneeth Chaganti 18ae06404b tools: Use pre-generated avatar images for documentation bots.
Avatar images for bots used by the tool to generate integration
documentation screenshots are pre-generated and committed to the repository.
The `generate-integration-docs-screenshot` tool now uses these images,
instead of trying to create these avatar images on the fly.

Also, deleted the unused `create_png_from_svg` function.
2020-04-22 17:45:30 -07:00
Puneeth Chaganti e97c39c587 tools: Extract png generation code to separate module. 2020-04-22 17:45:30 -07:00
Puneeth Chaganti 6b3358b14f generate-integration-docs-screenshot: Allow sending custom headers.
This commit makes it convenient to specify additional headers to send in the
integration webhook request from the command line.
2020-04-17 09:41:55 -07:00
Puneeth Chaganti 84e411c543 generate-integration-docs-screenshot: Fix crash on ignored fixtures.
When using a fixture that is correctly handled by our webhooks, but don't
cause a notification message to be sent, the tool shouldn't crash.
2020-04-17 09:41:55 -07:00
Puneeth Chaganti cee240fb74 generate-integration-docs-screenshot: Raise ArgumentTypeError.
When an invalid fixture is passed as an argument to the tool, raise an
`ArgumentTypeError`, instead of a `ValueError`.
2020-04-17 09:41:55 -07:00
Puneeth Chaganti ae754887eb generate-integration-docs-screenshot: Message to run dev server.
If the dev server is not running, when this tool is run, we print a clear
error message to start the dev server first, and then run this tool.
2020-04-17 09:41:55 -07:00
Puneeth Chaganti 874e473fc4 generate-integration-docs-screenshot: Refactor code into functions. 2020-04-17 09:41:55 -07:00
Puneeth Chaganti 4719226d0f generate-integration-docs-screenshot: Ensure puppeteer is installed. 2020-04-17 09:41:55 -07:00
Puneeth Chaganti 464cbcd9c1 tools: Capture incoming bot message screenshot using puppeteer. 2020-04-16 19:25:13 -07:00
Puneeth Chaganti 4d2ce607c9 tools: Add script to trigger webhook notification using fixtures.
When creating a webhook integration or creating a new one, it is a pain to
create or update the screenshots in the documentation. This commit adds a
tool that can trigger a sample notification for the webhook using a fixture,
that is likely already written for the tests.

Currently, the developer needs to take a screenshot manually, but this could
be automated using puppeteer or something like that.

Also, the tool does not support webhooks with basic auth, and only supports
webhooks that use json fixtures. These can be fixed in subsequent commits.
2020-04-16 19:25:13 -07:00