Commit Graph

38647 Commits

Author SHA1 Message Date
Anders Kaseorg 768f9f93cd docs: Capitalize Markdown consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Anders Kaseorg 60a25b2721 docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Alex Vandiver 2928bbc8bd logging: Report stack_info on logging.exception calls.
The exception trace only goes from where the exception was thrown up
to where the `logging.exception` call is; any context as to where
_that_ was called from is lost, unless `stack_info` is passed as well.
Having the stack is particularly useful for Sentry exceptions, which
gain the full stack trace.

Add `stack_info=True` on all `logging.exception` calls with a
non-trivial stack; we omit `wsgi.py`.  Adjusts tests to match.
2020-08-11 10:16:54 -07:00
Alex Vandiver e3a093b73f sentry: Add two more languages seen. 2020-08-11 10:15:02 -07:00
Alex Vandiver 34250d76a2 sentry: Don't drop (and server error) on languages we don't know.
It is better to show the stacktrace without highlighting than to drop
it entirely.
2020-08-11 10:15:02 -07:00
Alex Vandiver 569de55b82 sentry: Rename mapping variable to be more explicitly-named. 2020-08-11 10:15:02 -07:00
Alex Vandiver 712affd4df sentry: Document default topic. 2020-08-11 10:15:02 -07:00
Mateusz Mandera a7039c815e queue_processors: Fix UnboundLocalError in QueueProcessingWorker.
consume_time_seconds wasn't properly defined at the beginning, so when
a BaseException that isn't a subclass of Exception is thrown, the
finally: block could be entered with it still undefined.
2020-08-11 10:09:42 -07:00
Vishnu KS 4dc83a139c counts: Create 7day_actives::day counstat. 2020-08-10 17:22:19 -07:00
Anders Kaseorg 6698fedd1e docs: Change Zoom app type to user-managed.
Based on the Zoom documentation, a user-managed app seems more
appropriate for our use.

https://marketplace.zoom.us/docs/guides/build#account-level-user-managed-apps

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 17:00:41 -07:00
Alex Vandiver 8fc6d740cd tornado: Set a half-second default timeout on requests from django.
By defaults, `requests` has no timeout on requests, which can lead to
waiting indefinitely.  Add a half-second timeout on these; this is
applied _inside_ each retry, not overall -- that is, with retries any
of these functions may take a total of 1.5s.
2020-08-10 16:55:56 -07:00
Alex Vandiver 89cd31e07c tornado: Switch trust_env = False out for no_proxy.
Use the `no_proxy` proxy, which explicitly disables proxy usage for
particular hosts.  This is a slightly cleaner solution than ignoring
all of the environment, as removing proxies is specifically what we
are attempting to accomplish.
2020-08-10 16:55:56 -07:00
Alex Vandiver 5bea344fde tornado: Factor out common error-handling pattern into an HTTP adapter.
The change in #2764 provided a better error message on one of the
three calls into Tornado, but left the other two with the old error
message.  `raise_for_status` was used on two out of three.

Use a custom HTTPAdapter to apply this pattern to all requests from
Django to Tornado.
2020-08-10 16:55:56 -07:00
Alex Vandiver bb754e0902 tornado: Add a retry with backoff to django-to-tornado requests.
This better hides errors from users during the moments when Tornado is
being restarted.
2020-08-10 16:55:56 -07:00
Alex Vandiver e87aecf61c tornado: Make requests_client a singleton, not a global. 2020-08-10 16:55:56 -07:00
Alex Vandiver 6918556648 tornado: Swap to early-abort, for fewer long hanging blocks. 2020-08-10 16:55:56 -07:00
Alex Vandiver 39368cad3a tornado: Extract functions called from django into one module.
This makes clearer the separation of concerns.
2020-08-10 16:55:56 -07:00
Sahil Batra 33e19fa7d1
puppeteer: Replace 'visible: false' with 'hidden: true'.
We need to replace 'visible: false' with 'hidden: true' to wait
for elements to get hidden. Using 'visible: false' just checks
whether the selector exists or not and does not check whether
the element is hidden or not.
2020-08-10 16:48:53 -07:00
Tim Abbott dedb28055a stats: Fix a few strings not tagged for translation. 2020-08-10 16:26:37 -07:00
Anders Kaseorg f869701d15 Revert "vagrant: Add installation support for MacOS 10.15.6."
This reverts commit 6e0d4b9502.  It was
merged with no review and has various issues (see #16086).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 12:10:15 -07:00
Anders Kaseorg 18dcb86de5 Revert "provision: Fix missing <sasl/sasl.h> headers during provision."
This reverts commit cc9dc07b07.  It was
merged with no review and has various issues (see #16086).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-10 12:09:37 -07:00
Aman cc9dc07b07 provision: Fix missing <sasl/sasl.h> headers during provision.
This fixes the error of missing sasl.h when compiling python-ldap
from source during provision.
2020-08-09 23:21:13 -07:00
Aman 6e0d4b9502 vagrant: Add installation support for MacOS 10.15.6.
There are file sharing issues with the mac version 10.15.6 and
vagrant. Using parallels as a provider for vagrant fixes the issue.
2020-08-09 23:21:13 -07:00
Tim Abbott d63cf04a20 docs: Clarify how manage.py backup --output works. 2020-08-09 17:44:32 -07:00
Tim Abbott e12225ace7 realm_audit_log: Fix malformed RealmAuditLog entries.
In f8bcf39014, we fixed buggy
marshalling of Streams and similar data structures where we were
including the Stream object rather than its ID in dictionaries passed
to ujson, and ujson happily wrote that large object dump into the
RealmAuditLog.extra_data field.

This commit includes a migration to fix those corrupted RealmAuditLog
entries, and because the migration loop is the same, also fixes the
format of similar RealmAuditLog entries to be in a more natural format
that doesn't weirdly nest and duplicate the "property" field.

Fixes #16066.
2020-08-09 16:51:29 -07:00
Alex Vandiver 83645a3115 sentry: Ignore SystemExit and similar exceptions.
There are three exceptions in Python3 which are descended from
BaseException, but not Exception: GeneratorExit, KeyboardInterrupt,
and SystemExit.  None of these are suitable to be sent to Sentry.
For example, SystemExit is raised by `sys.exit`; in that sense, it is
never "uncaught" because we chose to cause it explicitly.

Use the suggested form[1] for ignoring specific classes of exceptions.

[1] https://github.com/getsentry/sentry-python/issues/149#issuecomment-434448781
2020-08-09 16:49:12 -07:00
Shanu b5b7bc9a7c
loading: Add aria-hidden="true" attribute to loading spinners.
Fixes #16082.
2020-08-09 16:47:56 -07:00
Steve Howell 8ba55b090a node fixtures: Add id field to realm_filters tuple. 2020-08-08 11:30:46 -04:00
Steve Howell 6f97e9dfa9 mypy: Use object, not Any, in event_schema.
This requires a few redundant runtime isinstance
checks, but the extra assertions arguably make
the code more readable, and isinstance checks
are extremely negligible.
2020-08-08 11:30:46 -04:00
Anders Kaseorg 3582183fba setup_venv: Install libyaml-dev.
This will let PyYAML link against LibYAML when PyYAML is next
installed.  Due to virtualenv-clone, that won’t happen until the next
Python package removal anyway, so we don’t bother bumping
PROVISION_VERSION.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 20:58:07 -07:00
Tim Abbott ba34e2a61c help: Mark Zoom integration as stable.
We've had plenty of happy production usage at this point.
2020-08-07 16:45:30 -07:00
Anders Kaseorg e789a8bb20 stylelint: Enable no-vendor-prefix rules.
Our vendor prefixes are generated by autoprefixer.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:17:20 -07:00
Anders Kaseorg 56931b68e0 styles: Unprefix appearance property.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:17:20 -07:00
Anders Kaseorg f166f4c28c webhooks: Fix invalid JSON in Papertrail test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:07:27 -07:00
Anders Kaseorg ae3af9cd56 webhooks: Fix invalid JSON in Jira test fixtures.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:06:05 -07:00
Anders Kaseorg 02fcb75ded analytics: Convert datetime to UNIX timestamp before JSON serialization.
datetime objects are not ordinarily JSON serializable.  While both
ujson and orjson have special cases to serialize datetime objects,
they do it in different ways.  So we want to do this explicitly.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:05:01 -07:00
Anders Kaseorg 219fc36051 stripe_fixtures: Fix incorrectly normalized JSON.
JSON does not allow leading zeros.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:05:01 -07:00
Anders Kaseorg f63d132276 js: Use unminified KaTeX (and minifiy it with everything else).
Since our Webpack config passes pre-minified JS files to
script-loader, they can’t be used as modules.  Use the normal
unminified version, letting Webpack minify it and give us source maps.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:27:44 -07:00
Anders Kaseorg c523657d48 test_subs: Remove incorrect encoding before JSON serialization.
bytes is not JSON serializable, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:14:37 -07:00
Anders Kaseorg 0b12d38e4d fixtures: Fix invalid JSON in narrow.json.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 23b815bb50 queue: Fix types to reflect that Pika channels receive bytes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg a40c67712c locale: Fix invalid JSON in locale/tr/translations.json.
I also fixed this in Transifex.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 0d1cc8c171 test_subs: Remove absurd bot_owner parameter from request.
Seriously now.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg c9d1f547bd test_auth_backends: Remove absurd user_profile parameter from requests.
What even did you think this was supposed to do.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 36bae2e86c message_fetch: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 3e7503a607 html_diff: Really return str, not bytes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 80c0e60edc integrations: Decode webhook responses to str.
bytes is not JSON serializable, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg 25c20a21bd event_queue: Convert users argument of send_event to list.
set is not JSON serializable, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 11:12:32 -07:00
Anders Kaseorg fe5c524890 user_status: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg a329b538d2 test_helpers: Fix instrument_url decorator type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00