Commit Graph

39074 Commits

Author SHA1 Message Date
Alex Vandiver cba1722129 webhooks: Do log non-"unsupported" errors to main logfiles.
This undoes a small part of b8a2e6b5f8; namely, logs to
`zulip.zerver.webhooks`, which are all exceptions from webhooks except
UnsupportedWebhookEventType, should still be logged to the main error
loggers.  This maintains the property that exceptions generating 500's
are all present in `errors.log`.
2020-09-14 12:58:16 -07:00
Ryan Rehman 20775e5429 minor: Rename `previously_selected` of message selected event.
This is done since we have logic in place which compares `event.id`
and `event.previously_selected`, both of which point to message ids.
2020-09-14 10:56:37 -07:00
Priyansh Garg 4d38cab9b9 frontend_tests: Fix typo in node_tests/general.js 2020-09-13 21:01:59 -07:00
Anders Kaseorg fd59db292f requirements: Upgrade Python requirements.
Fixes #16335.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 20:59:09 -07:00
Anders Kaseorg ddf8ec33df upload: Strip leading slash from deleted S3 export paths.
Previously, S3UploadBackend.delete_export_tarball failed to strip the
leading ‘/’ from the export path.  This mistake is now caught by Moto
1.3.15.  I expect it caused deletion failures in the real S3, although
I haven’t verified this.

We store export_path in the audit log with a leading ‘/’, but the
actual S3 keys do not have a leading ‘/’.  Changing either system
would require a migration.  So the new convention is that the
variables named ‘export_path’ have a leading ‘/’, while variables
named ‘path_id’ or ‘key’ do not.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 20:59:09 -07:00
Anders Kaseorg 42d038f09b migrations: Fix 0301 to replace a Python loop with SQL.
The previous code is correctly flagged by semgrep 0.23 as a violation
of our sql-format rule.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 20:59:09 -07:00
Hemanth V. Alluri f57196b1d0 webhooks/gitlab: Add fixture for MR opened with multiple assignees.
Fixes #16173.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-09-13 20:54:14 -07:00
Hemanth V. Alluri 54aba8d402 webhooks/gitlab: Use information about all assignees.
Previously, the GitLab webhook code, namely the `get_objects_assignee`
method first tried to get a single assignee and if that failed then it
looks for multiple assignees and then it would return the first
assignee that it found (there's actually a code smell here - a loop
which would always return on the first iteration).

Instead, this commit will change that behavior to first check for
multiple assignees first then for a single assignee if we can't find
multiple assignees. Ultimately it will return a list of all of the
assignees (however many that might be [0, n]). This method has then
aptly been renamed to `get_assignees`.

Finally, we tweked the code using this method to always use it's
output as an "assignees" parameter to templates (there's also an
assignee parameter which we want to avoid here for consistency).

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-09-13 20:54:14 -07:00
Hemanth V. Alluri bbe7a54171 webhooks/gitlab: Make permissions consistent across all fixtures.
For some reasons, some of the fixtures had the +x bit set, while
some didn't. What this commit does is make sure that no fixture
is marked as "executable" (for anyone).

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-09-13 20:54:14 -07:00
Anders Kaseorg dfab09b17d markdown: Replace hyperlink requirement with urllib.parse.
The previous code only worked by accident and hyperlink 20.0.0 breaks
it.

>>> hyperlink.parse("example.com").replace(scheme="https")
DecodedURL(url=URL.from_text('https:example.com'))

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 15:37:28 -07:00
Anders Kaseorg f39d4cf1f0 sentry: Pass format arguments to logging.
semgrep 0.23 correctly flags this as a violation of our logging-format
rule.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-13 15:37:28 -07:00
Mateusz Mandera 70bcc7a8e9 migrations: Add case-insensitive unique index on realm and stream name.
This is closely analogical to b9b146c809.
2020-09-13 15:36:43 -07:00
Mateusz Mandera cf5d544c82 ldap: Fix inaccuracy in django_to_ldap_username comment. 2020-09-13 18:39:00 +02:00
Anders Kaseorg 463929f349 urls: Migrate re_path routes to path.
Django treats path("<name>") like re_path(r"(?P<name>[^/]+)") and
path("<path:name>") like re_path(r"(?P<name>.+)").

This is more readable and consistent than the mix of slightly
different regexes we had before, and fixes various bugs:

• The r'apps/(.*)$' regex was missing a start anchor ^, so it
  incorrectly matched all URLs that included apps/ as a substring
  anywhere.
• The r'accounts/login/(google)/$' regex was missing a start anchor ^,
  so it incorrectly matched all URLs that ended with
  accounts/login/google/.
• The type annotation of zerver.views.realm_export.delete_realm_export
  takes export_id as an int, but it was previously passed as a string.
• The type annotation of zerver.views.users.avatar takes medium as a
  bool, but it was previously passed as a string.
• The [0-9A-Za-z]+ pattern for uidb64 was missing the - and _
  characters that can validly be part of a base64url encoded
  string (although I think the id is actually a decimal integer here,
  in which case only 012345ADEIMNOQTUYcgjkwxyz are present in its
  base64url encoding).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 14:29:26 -07:00
Aryan Shridhar 4e8067aadc
message_edit: Add hide event for clipboard tooltip after copy.
The clipboard tooltip for the "copy and close" button was incorrectly staying visible after the
copy event.  Fix this by explicitly hiding tooltips in the click handler.

Fixes #16328.
2020-09-12 12:04:24 -07:00
Anders Kaseorg c21818e093 check-openapi: Disallow siblings of $ref.
The specification says “any sibling elements of a $ref are ignored”,
so their presence, although not invalid, indicates a mistake.  yamole
incorrectly merges these siblings into the referenced object, but we
should not rely on this nonstandard behavior.

https://swagger.io/docs/specification/using-ref/#sibling

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:57:13 -07:00
Anders Kaseorg 7c17bdb9c5 openapi: Remove ‘example’ $ref siblings.
$ref siblings are ignored according to the OpenAPI specification, and
the referenced definitions already have examples.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:57:13 -07:00
Anders Kaseorg caa08d76b5 openapi: Inline parameter references to avoid ‘required’ $ref siblings.
$ref siblings are ignored according to the OpenAPI specification.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:57:13 -07:00
Thomas Güttler c567661015
docs: Added PyCharm to the list of editors.
PyCharm is great for writing Python.
2020-09-12 11:42:22 -07:00
Thomas Güttler 740a4a1eff
docs: Spell "front end" as "frontend" consistently.
I think it makes sense avoid diversity here.
2020-09-12 11:41:35 -07:00
Tim Abbott 45cd979d46 tools: Fix webpack wrapper logic for finding config file to watch. 2020-09-12 11:31:44 -07:00
Thomas Güttler c5a7103b4d docs: Use zulip.com rather than zulip.org (which just redirects).
I guess it make sense to use zulip.com in the first place.
2020-09-12 11:22:08 -07:00
Anders Kaseorg b41165fd01 dev_urls: Give the user_avatars URLPattern a name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:21:40 -07:00
Anders Kaseorg 1c80188648 urls: Elide str converter, which is the default.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:21:40 -07:00
palash 2632317b06 test_message_send: Refactor mock.patch to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file zerver/tests/test_message_send.py
2020-09-12 11:04:51 -07:00
palash f2f8034b76 test_markdown: Refactor mock.patch to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file zerver/tests/test_markdown.py
2020-09-12 11:04:51 -07:00
palash 588fff236d test_service_bot_system: Refactor mock.patch with assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file zerver/tests/test_service_bot_system.py
2020-09-12 10:59:36 -07:00
palash 4e2769c31c test_decorators: Refactor mock.patch to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file zerver/tests/test_decorators.py
2020-09-12 10:59:36 -07:00
palash c3fee5059f test_email_mirror: Refactor mock.path to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file zerver/tests/test_email_mirror.py
2020-09-12 10:59:36 -07:00
palash 34003fc7f6 email_mirror: Change root logger to zerver.lib.email_mirror. 2020-09-12 10:53:56 -07:00
palash 60f9f87c5d test_external: Refactor mock.patch to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file zerver/tests/test_external.py
2020-09-12 10:53:56 -07:00
palash 485d740fe8 apple auth: Change logger to zulip.auth.apple from root logger.
Replace default root logger with zulip.auth.apple for apple auth
in file zproject/backends.py and update the test cases
accordingly in file zerver/tests/test_auth_backends.py
2020-09-12 10:53:56 -07:00
palash 04f08f45e7 test_auth_backends: Refactor mock.patch to assertLogs.
Replaced mock.patch with assertLogs for testing log outputs
in file test_auth_backends.py.

This change requires adjusting
test_log_into_subdomain_when_email_is_none to use an explicit token
since that appears in the log output.
2020-09-12 10:53:37 -07:00
Alex Vandiver 1c17583ad5 puppet: Restrict postfix incoming addresses to postmaster and zulip.
This removes the possibility of local user enumeration via RCPT TO.
2020-09-11 18:49:22 -07:00
Aman Agrawal ed8796f23f report: Allow anonymous queries to report (un)narrow times. 2020-09-11 16:57:27 -07:00
Alex Vandiver 3f6e4ff303 webhooks: Move the extra logging information into a formatter.
This clears it out of the data sent to Sentry, where it is duplicative
with the indexed metadata -- and potentially exposes PHI if Sentry's
"make this issue public" feature is used.
2020-09-11 16:43:29 -07:00
Alex Vandiver b8a2e6b5f8 webhooks: Configure webhook loggers in zproject/computed_settings.py.
This limits the webhook errors to only go to their respective log
files, and not to the general server logs.
2020-09-11 16:43:29 -07:00
Alex Vandiver 006875e7d0 sentry: Add additional context (client, fallback realm) to Sentry tags. 2020-09-11 16:43:29 -07:00
Alex Vandiver 6323218a0e request: Maintain a thread-local of the current request.
This allows logging (to Sentry, or disk) to be annotated with richer
data about the request.
2020-09-11 16:43:29 -07:00
Mateusz Mandera bb4567f57e queue: Extract get_remaining_queue_size method. 2020-09-11 15:51:07 -07:00
Mateusz Mandera aae84197e8 check-rabbitmq-queue: Use list_queues output for current backlog size.
The value in the stats file can get outdated if the queue hasn't done
enough iterations to update the stats file for a while. The queue size
output by rabbitmqctl list_queues is more up to date, and empirically
tends to agree with the value in the stats file (when the stats file is
fresh).
2020-09-11 15:51:07 -07:00
Mateusz Mandera 1d466a4fc5 queue: Make embed_link updates stats on every iteration. 2020-09-11 15:51:07 -07:00
Tim Abbott f0c2c640b6 api docs: Add changes for community_topic_editing_limit_seconds.
The previous commit fixed this in the changelog, but it should have
also been documented directly as well.
2020-09-11 15:50:06 -07:00
Kaustubh Nair acf31f8934 docs: Add community_topic_editing_limit_seconds value to docs.
This was added to the response in e242ddc. Specifying the value
makes sure that the user does not have to dig in the commit history.
2020-09-11 15:48:08 -07:00
Thomas Güttler 57ad171112
docs: Remove typo (two periods at end of sentence). 2020-09-11 15:47:09 -07:00
Alex Vandiver 4917391133 webhooks: Derive payload from request itself. 2020-09-10 17:47:22 -07:00
Alex Vandiver a1f5f6502c webhooks: In logger, pull user from request, rather than parameter.
request.user is set by validate_api_key, which is called by
webhook_view and authenticated_rest_api_view.
2020-09-10 17:47:22 -07:00
Alex Vandiver d04db7c5fe webhooks: Remove repetitive argument to UnsupportedWebhookEventType.
The name of the webhook can be added by the webhook decorator.
2020-09-10 17:47:21 -07:00
Alex Vandiver e2ab7b9e17 webhooks: Update API_KEY_ONLY_WEBHOOK_LOG_PATH to WEBHOOK_LOG_PATH.
The existence of "API_KEY" in this configuration variable is
confusing.  It is fundamentally about webhooks.
2020-09-10 17:47:21 -07:00
Alex Vandiver cf6ebb9c8d webhooks: Rename api_key_only_webhook_view to webhook_view.
There are no other types of webhook views; this is more concise.
2020-09-10 17:47:21 -07:00