Commit Graph

38642 Commits

Author SHA1 Message Date
Alex Vandiver 569de55b82 sentry: Rename mapping variable to be more explicitly-named. 2020-08-11 10:15:02 -07:00
Alex Vandiver 712affd4df sentry: Document default topic. 2020-08-11 10:15:02 -07:00
Mateusz Mandera a7039c815e queue_processors: Fix UnboundLocalError in QueueProcessingWorker.
consume_time_seconds wasn't properly defined at the beginning, so when
a BaseException that isn't a subclass of Exception is thrown, the
finally: block could be entered with it still undefined.
2020-08-11 10:09:42 -07:00
Vishnu KS 4dc83a139c counts: Create 7day_actives::day counstat. 2020-08-10 17:22:19 -07:00
Anders Kaseorg 6698fedd1e docs: Change Zoom app type to user-managed.
Based on the Zoom documentation, a user-managed app seems more
appropriate for our use.

https://marketplace.zoom.us/docs/guides/build#account-level-user-managed-apps

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 17:00:41 -07:00
Alex Vandiver 8fc6d740cd tornado: Set a half-second default timeout on requests from django.
By defaults, `requests` has no timeout on requests, which can lead to
waiting indefinitely.  Add a half-second timeout on these; this is
applied _inside_ each retry, not overall -- that is, with retries any
of these functions may take a total of 1.5s.
2020-08-10 16:55:56 -07:00
Alex Vandiver 89cd31e07c tornado: Switch trust_env = False out for no_proxy.
Use the `no_proxy` proxy, which explicitly disables proxy usage for
particular hosts.  This is a slightly cleaner solution than ignoring
all of the environment, as removing proxies is specifically what we
are attempting to accomplish.
2020-08-10 16:55:56 -07:00
Alex Vandiver 5bea344fde tornado: Factor out common error-handling pattern into an HTTP adapter.
The change in #2764 provided a better error message on one of the
three calls into Tornado, but left the other two with the old error
message.  `raise_for_status` was used on two out of three.

Use a custom HTTPAdapter to apply this pattern to all requests from
Django to Tornado.
2020-08-10 16:55:56 -07:00
Alex Vandiver bb754e0902 tornado: Add a retry with backoff to django-to-tornado requests.
This better hides errors from users during the moments when Tornado is
being restarted.
2020-08-10 16:55:56 -07:00
Alex Vandiver e87aecf61c tornado: Make requests_client a singleton, not a global. 2020-08-10 16:55:56 -07:00
Alex Vandiver 6918556648 tornado: Swap to early-abort, for fewer long hanging blocks. 2020-08-10 16:55:56 -07:00
Alex Vandiver 39368cad3a tornado: Extract functions called from django into one module.
This makes clearer the separation of concerns.
2020-08-10 16:55:56 -07:00
Sahil Batra 33e19fa7d1
puppeteer: Replace 'visible: false' with 'hidden: true'.
We need to replace 'visible: false' with 'hidden: true' to wait
for elements to get hidden. Using 'visible: false' just checks
whether the selector exists or not and does not check whether
the element is hidden or not.
2020-08-10 16:48:53 -07:00
Tim Abbott dedb28055a stats: Fix a few strings not tagged for translation. 2020-08-10 16:26:37 -07:00
Anders Kaseorg f869701d15 Revert "vagrant: Add installation support for MacOS 10.15.6."
This reverts commit 6e0d4b9502.  It was
merged with no review and has various issues (see #16086).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 12:10:15 -07:00
Anders Kaseorg 18dcb86de5 Revert "provision: Fix missing <sasl/sasl.h> headers during provision."
This reverts commit cc9dc07b07.  It was
merged with no review and has various issues (see #16086).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 12:09:37 -07:00
Aman cc9dc07b07 provision: Fix missing <sasl/sasl.h> headers during provision.
This fixes the error of missing sasl.h when compiling python-ldap
from source during provision.
2020-08-09 23:21:13 -07:00
Aman 6e0d4b9502 vagrant: Add installation support for MacOS 10.15.6.
There are file sharing issues with the mac version 10.15.6 and
vagrant. Using parallels as a provider for vagrant fixes the issue.
2020-08-09 23:21:13 -07:00
Tim Abbott d63cf04a20 docs: Clarify how manage.py backup --output works. 2020-08-09 17:44:32 -07:00
Tim Abbott e12225ace7 realm_audit_log: Fix malformed RealmAuditLog entries.
In f8bcf39014, we fixed buggy
marshalling of Streams and similar data structures where we were
including the Stream object rather than its ID in dictionaries passed
to ujson, and ujson happily wrote that large object dump into the
RealmAuditLog.extra_data field.

This commit includes a migration to fix those corrupted RealmAuditLog
entries, and because the migration loop is the same, also fixes the
format of similar RealmAuditLog entries to be in a more natural format
that doesn't weirdly nest and duplicate the "property" field.

Fixes #16066.
2020-08-09 16:51:29 -07:00
Alex Vandiver 83645a3115 sentry: Ignore SystemExit and similar exceptions.
There are three exceptions in Python3 which are descended from
BaseException, but not Exception: GeneratorExit, KeyboardInterrupt,
and SystemExit.  None of these are suitable to be sent to Sentry.
For example, SystemExit is raised by `sys.exit`; in that sense, it is
never "uncaught" because we chose to cause it explicitly.

Use the suggested form[1] for ignoring specific classes of exceptions.

[1] https://github.com/getsentry/sentry-python/issues/149#issuecomment-434448781
2020-08-09 16:49:12 -07:00
Shanu b5b7bc9a7c
loading: Add aria-hidden="true" attribute to loading spinners.
Fixes #16082.
2020-08-09 16:47:56 -07:00
Steve Howell 8ba55b090a node fixtures: Add id field to realm_filters tuple. 2020-08-08 11:30:46 -04:00
Steve Howell 6f97e9dfa9 mypy: Use object, not Any, in event_schema.
This requires a few redundant runtime isinstance
checks, but the extra assertions arguably make
the code more readable, and isinstance checks
are extremely negligible.
2020-08-08 11:30:46 -04:00
Anders Kaseorg 3582183fba setup_venv: Install libyaml-dev.
This will let PyYAML link against LibYAML when PyYAML is next
installed.  Due to virtualenv-clone, that won’t happen until the next
Python package removal anyway, so we don’t bother bumping
PROVISION_VERSION.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 20:58:07 -07:00
Tim Abbott ba34e2a61c help: Mark Zoom integration as stable.
We've had plenty of happy production usage at this point.
2020-08-07 16:45:30 -07:00
Anders Kaseorg e789a8bb20 stylelint: Enable no-vendor-prefix rules.
Our vendor prefixes are generated by autoprefixer.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:17:20 -07:00
Anders Kaseorg 56931b68e0 styles: Unprefix appearance property.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:17:20 -07:00
Anders Kaseorg f166f4c28c webhooks: Fix invalid JSON in Papertrail test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:07:27 -07:00
Anders Kaseorg ae3af9cd56 webhooks: Fix invalid JSON in Jira test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:06:05 -07:00
Anders Kaseorg 02fcb75ded analytics: Convert datetime to UNIX timestamp before JSON serialization.
datetime objects are not ordinarily JSON serializable.  While both
ujson and orjson have special cases to serialize datetime objects,
they do it in different ways.  So we want to do this explicitly.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:05:01 -07:00
Anders Kaseorg 219fc36051 stripe_fixtures: Fix incorrectly normalized JSON.
JSON does not allow leading zeros.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:05:01 -07:00
Anders Kaseorg f63d132276 js: Use unminified KaTeX (and minifiy it with everything else).
Since our Webpack config passes pre-minified JS files to
script-loader, they can’t be used as modules.  Use the normal
unminified version, letting Webpack minify it and give us source maps.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:27:44 -07:00
Anders Kaseorg c523657d48 test_subs: Remove incorrect encoding before JSON serialization.
bytes is not JSON serializable, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:14:37 -07:00
Anders Kaseorg 0b12d38e4d fixtures: Fix invalid JSON in narrow.json.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 23b815bb50 queue: Fix types to reflect that Pika channels receive bytes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg a40c67712c locale: Fix invalid JSON in locale/tr/translations.json.
I also fixed this in Transifex.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 0d1cc8c171 test_subs: Remove absurd bot_owner parameter from request.
Seriously now.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg c9d1f547bd test_auth_backends: Remove absurd user_profile parameter from requests.
What even did you think this was supposed to do.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 36bae2e86c message_fetch: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 3e7503a607 html_diff: Really return str, not bytes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 80c0e60edc integrations: Decode webhook responses to str.
bytes is not JSON serializable, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 25c20a21bd event_queue: Convert users argument of send_event to list.
set is not JSON serializable, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg fe5c524890 user_status: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg a329b538d2 test_helpers: Fix instrument_url decorator type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg 28143c0ee2 drafts: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg b35e4428a9 test_custom_profile_data: Convert order QuerySet to list.
A QuerySet object is not JSON serializable, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg 44af001045 custom_profile_fields: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.  Mypy didn’t
catch the mismatched type of profiles_by_user_id because it doesn’t
understand CustomProfileFieldValue.field_id.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg b250e42f61 realm_user_count_by_role: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg f8bcf39014 actions: Do not attempt JSON serialization of a Stream object.
It doesn’t end well.  Or sometimes it doesn’t end (OverflowError:
Maximum recursion level reached).

Introduced by commits ccdf52fef6 and
94d2de8b4a (#15601).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00