Commit Graph

754 Commits

Author SHA1 Message Date
Anders Kaseorg d0f4af5f8c python: Catch JSONDecodeError instead of ValueError when decoding JSON.
These weren’t wrong since orjson.JSONDecodeError subclasses
json.JSONDecodeError which subclasses ValueError, but the more
specific ones express the intention more clearly.

(ujson raised ValueError directly, as did json in Python 2.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 11:59:59 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Anders Kaseorg 123790a72d webhooks: Fix invalid JSON in Hello World test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:47:13 -07:00
Anders Kaseorg 5043f6039b webhooks: Fix invalid JSON in Freshdesk test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:47:13 -07:00
Anders Kaseorg ec6856bc3b webhooks: Fix invalid JSON in Dialogflow test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:47:13 -07:00
Anders Kaseorg 768f9f93cd docs: Capitalize Markdown consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Anders Kaseorg 60a25b2721 docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Alex Vandiver e3a093b73f sentry: Add two more languages seen. 2020-08-11 10:15:02 -07:00
Alex Vandiver 34250d76a2 sentry: Don't drop (and server error) on languages we don't know.
It is better to show the stacktrace without highlighting than to drop
it entirely.
2020-08-11 10:15:02 -07:00
Alex Vandiver 569de55b82 sentry: Rename mapping variable to be more explicitly-named. 2020-08-11 10:15:02 -07:00
Alex Vandiver 712affd4df sentry: Document default topic. 2020-08-11 10:15:02 -07:00
Anders Kaseorg f166f4c28c webhooks: Fix invalid JSON in Papertrail test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:07:27 -07:00
Anders Kaseorg ae3af9cd56 webhooks: Fix invalid JSON in Jira test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:06:05 -07:00
Hemanth V. Alluri 4e1024da5c webhooks/gitlab: Add an option to exclude MR title from topics.
Since the title of a merge request can often change, it shouldn't be a
part of the topic that we send the message to. Otherwise things would
get messy and confusing.

But at the same time we don't want to make this mandatory. So we add
a new boolean GET parameter that can toggle whether or not the topic
should include the MR title (`use_merge_request_title`).

Fixes #15951.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-07-31 10:33:52 -07:00
davidscherer 0e90442d8b
integrations: Don't call prettify_date on Trello card descriptions.
Card descriptions aren't dates, and calling prettify_date on them results in removing upper case T characters, replacing uppercase Z characters with " UTC", etc. in descriptions when they appear in Zulip.

This was pretty clearly just a copy/paste mistake (these functions are very closely parallel to the *_due_date_* functions above, which do work on dates and call prettify_date).
2020-07-28 17:58:19 -07:00
Hemanth V. Alluri eb2809effe webhooks/freshdesk: Remove key checking step.
The idea behind doing this is that we would rather let the code error
out rather than add to the logs. It's webhook code usually never uses
the logging module so this section of legacy code needed to be changed
or removed.

Assists PR #15942.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-07-28 10:55:11 -07:00
David Wood 366cb30db1 integrations: Re-add Canarytoken integration.
This commit re-adds the integration for canarytokens.org, now separate
from the primary Thinkst integration.

Signed-off-by: David Wood <david@davidtw.co>
2020-07-20 15:34:40 -07:00
David Wood f1c9d9c2fe integrations: Fix Thinkst Canary integration.
This commit fixes the Thinkst Canary integration which - based on the
schema in upstream documentation - incorrectly assumed that some fields
would always be sent, which meant that the integration would fail. In
addition, this commit adjusts support for canarytokens to only support
the canarytoken schema with Thinkst Canaries (not Thinkst's
canarytokens.org).

Signed-off-by: David Wood <david@davidtw.co>
2020-07-20 15:34:40 -07:00
Steve Howell 0a88202763 webhooks: Ignore short_name to guess user.
The short_name field isn't super reliable, and
we are about to eliminate it.
2020-07-17 11:04:54 -07:00
Anders Kaseorg 654a03c897 webhooks: Fix strict_optional errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 11:25:48 -07:00
Anders Kaseorg 768e8ccc55 tests: Make all tests inherit ZulipTestCase.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-01 10:48:12 -07:00
Anders Kaseorg f2e7076e2a decorator: Replace type: ignore with cast, avoid Any.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-30 18:58:23 -07:00
Till Schneidereit 6d6d43188d webhooks/github: Include full repository name in notification messages.
This changes the notification messages for events that currently just
include the string `"the repository"` to also include the full (`org/repo`)
name of the affected repository. Messages for the following events are
changed:
- `public`
- `star`
- `watch`
- `repository`
- `team_add`

Background: we're using the GitHub integration for org-wide notifications
for the [Bytecode Alliance Zulip](bytecodealliance.zulipchat.com/), and
having all messages just say "the repository" isn't ideal. Even now one
can hover over the link to see the repo's url, but it'd be much nicer if
the message just contained the full name.

I also changed the message for `star` to include a link to the repository,
same as the `watch` notification.
2020-06-29 13:12:16 -07:00
Anders Kaseorg 596bfb47c4 papertrail: Replace custom validator with check_dict.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-20 22:29:15 -07:00
Anders Kaseorg 114f859d3a api_travis_webhook: Fix message type annotation.
Not all values of this dict are strings, just the ones we use.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-20 22:29:15 -07:00
Anders Kaseorg 87f7874a79 CVE-2020-12759: Fix reflected XSS vulnerability in Dropbox webhook.
Also check the challenge argument’s presence before using it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-16 22:46:16 -07:00
Tim Abbott f4ac4be851 sentry webhook: Check for platform support early.
Otherwise, we don't know whether the event format might not have the
`filename` or other parameters.
2020-06-16 16:49:31 -07:00
Tim Abbott 54604257e0 sentry: Provide more clarity around unsupported platforms. 2020-06-16 14:04:16 -07:00
Anders Kaseorg f364d06fb5 python: Convert percent formatting to .format for translated strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-15 16:24:46 -07:00
Tim Abbott 05455f432e sentry: Enable the javascript name for that platform.
Hopefully this will mean our Sentry integration properly handles
javascript events now.
2020-06-15 13:01:39 -07:00
Anders Kaseorg 5dc9b55c43 python: Manually convert more percent-formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Hemanth V. Alluri 545f40b2c2 webhooks/circleci: Add support for super minimal payloads.
After merging PR #15355 we found that CircleCI may send
super minimal payloads. This does not seem to depend on
.circleci/config.yml since we received two different
types of payloads off of the same configuration.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-06-14 22:35:41 -07:00
Hemanth V. Alluri 5bbfaa225b webhooks/circleci: Overhaul the CircleCI integration.
This change should add a lot more information to
our CircleCI integration's output.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-06-14 13:14:55 -07:00
Anders Kaseorg 57a80856a5 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:39:00 -07:00
Anders Kaseorg 0d6c771baf python: Guard against default value mutation with read-only types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 91a86c24f5 python: Replace None defaults with empty collections where appropriate.
Use read-only types (List ↦ Sequence, Dict ↦ Mapping, Set ↦
AbstractSet) to guard against accidental mutation of the default
value.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
Anders Kaseorg 67e7a3631d python: Convert percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 15:02:09 -07:00
Anders Kaseorg 6480deaf27 python: Convert more "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, with more
restrictions patched out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-10 14:48:09 -07:00
David Wood f0f42f7a94 integrations: Add Thinkst Canary integration.
This commit adds an integration for Thinkst Canaries - physical, VM and
cloud-based canaries for detecting attackers to a network. Thinkst
Canaries can send webhook alerts when canaries have been tripped, and
this integration will post Zulip messages when these webhooks are
received.

Signed-off-by: David Wood <david@davidtw.co>
2020-06-09 16:28:28 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Anders Kaseorg 8c15081069 python: Further pyupgrade changes.
Generated by pyupgrade --py3-plus --keep-percent-format.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:23:10 -07:00
Johannes Hochreiter d78d9099d1 gitlab: added pipeline number to event notification
- link to pipeline now also features the pipeline number
- updated tests accordingly
2020-06-07 14:01:04 -07:00
Johannes Hochreiter 60077f3097 gitlab: Added artifact link to pipeline event body
- if artifact was built file name, download link and browse link will be added to pipeline event body
- added test for pipeline with built artifacts
2020-06-07 14:01:04 -07:00
Mateusz Mandera 5b746b4128 docs: Add missing step to dropbox webhook instructions. 2020-06-05 10:21:25 -07:00
Anders Kaseorg 1f565a9f41 timezone: Use standard library datetime.timezone.utc consistently.
datetime.timezone is available in Python ≥ 3.2.  This also lets us
remove a pytz dependency from the PostgreSQL scripts.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-05 09:34:17 -07:00
Anders Kaseorg 95f29b5409 tests: Avoid deprecated TestCase method aliases.
Fixes these warnings with python -Wd:

/home/circleci/zulip/zerver/tests/test_middleware.py:43: DeprecationWarning: Please use assertRegex instead.
  self.assertRegexpMatches(
/home/circleci/zulip/zerver/webhooks/travis/tests.py:40: DeprecationWarning: Please use assertNotEqual instead.
  self.assertNotEquals(msg.topic_name(), self.TOPIC)

https://docs.python.org/3/library/unittest.html#deprecated-aliases

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-04 14:15:52 -07:00
Hemanth V. Alluri 9a9c1e0794 webhooks/stripe: Fix the invoice_created event.
The previous code for this event was using a key that's not actually
a part of the payload. So here we simple remove the usage of that key
and add a (previously missing) test for this event.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-06-03 17:18:43 -07:00