Commit Graph

44196 Commits

Author SHA1 Message Date
Anders Kaseorg 404ef284bb validator: Remove unused check_tuple function.
Tuples cannot be deserialized from JSON.

While we do use these validators for other things, like event
dictionaries, we have migrated the API away from using those.  The
last use was removed in 4f3d5f2d87

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:41 -07:00
Anders Kaseorg f3e5ed57ae utils: Remove unused split_by function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 58b7a4eb44 utils: Remove unused query_chunker function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 271333301d user_groups: Remove unused check_remove_user_from_user_group function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 4e366bfff5 user_groups: Remove unused check_add_user_to_user_group function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 57acac12fa user_groups: Remove unused user_groups_in_realm function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg fdbde9f9c2 push_notifications: Remove unused num_push_devices_for_user function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg b2239e45ac statistics: Remove unused seconds_usage_between function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 27325eb2ae exceptions: Remove unused to_json method of JsonableError.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 79f7f9b8d9 actions: Remove unused get_emails_from_user_ids function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 6ae5be6e75 exceptions: Remove AbstractEnum in favor of enum.auto.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 0d061f44c1 actions: Remove acting_client parameter from bulk_remove_subscriptions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 0b795e492f thumbnail: Remove unused is_camo_url parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 4206e5f00b python: Remove locally dead code.
These changes are all independent of each other; I just didn’t feel
like making dozens of commits for them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 9afe6d0829 templates: Fix footer links on /plans.
The footer was being covered by a bunch of invisible divs, rendering
most of the links non-functional.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:41:29 -07:00
Anders Kaseorg 672358c5da docs: Migrate from recommonmark to MyST-Parser.
Recommonmark is no longer maintained, and MyST-Parser is much more
complete.

https://myst-parser.readthedocs.io/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 8f2827a65d docs: Remove trailing newlines from code blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg cb61ea69c6 docs: Fix misaligned Markdown source indentation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 25c6d5c99c docs: Fix code span syntax in embedded reST block.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg c3646ec67f docs: Move authentication-methods#ldap anchor to appropriate heading.
Commit 30eaed0378 (#15001) incorrectly
inserted a different section between the anchor and the heading.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Mateusz Mandera fdbde59b07 rate_limit: Add management command to reset auth rate limit.
The auth attempt rate limit is quite low (on purpose), so this can be a
common scenario where a user asks their admin to reset the limit instead
of waiting. We should provide a tool for administrators to handle such
requests without fiddling around with code in manage.py shell.
2021-08-19 00:45:17 -07:00
Alex Vandiver 56344e4765 scheduled_email: Only call .save() if necessary.
Calling `email.save()` is only needed if we altered `email.address`;
it is unnecessary if we called `email.users.add(...)` which will have
done its own INSERT.
2021-08-19 00:44:33 -07:00
Alex Vandiver d4f6b93859 scheduled_email: Create ScheduledEmail objects in a transaction.
This fixes two bugs: the most obvious is that there is a race where a
ScheduledEmail object could be observed in the window between creation
and when users are added; this is a momentary instance when the object
has no users, but one that will resolve itself.

The more subtle is that .save() will, if no records were found to be
updated, _re-create_ the object as it exists in memory, using an
INSERT[1].  Thus, there is a race with `deliver_scheduled_emails`
between when the users are added, and when `email.save()` runs:

 1. Web request creates ScheduledEmail object
 2. Web request creates ScheduledEmailUsers object
 3. deliver_scheduled_emails locks the former, preventing updates.
 4. deliver_scheduled_emails deletes both objects, commits, releasing lock
 5. Web request calls `email.save()`; UPDATE finds no rows, so it
    re-creates the ScheduledEmail object.
 6. Future deliver_scheduled_emails runs find a ScheduledEmail with no
    attending ScheduledEmailUsers objects

Wrapping the logical creation of both of these in a single transaction
avoids both of these races.

[1] https://docs.djangoproject.com/en/3.2/ref/models/instances/#how-django-knows-to-update-vs-insert
2021-08-19 00:44:33 -07:00
Alex Vandiver 4c518c2bba scheduled_email: Consistently lock users table.
Only clear_scheduled_emails previously took a lock on the users before
removing them; make deliver_scheduled_emails do so as well, by using
prefetch_related to ensure that the table appears in the SELECT.  This
is not necessary for correctness, since all accesses of
ScheduledEmailUser first access the ScheduledEmail and lock it; it is
merely for consistency.

Since SELECT ... FOR UPDATE takes an UPDATE lock on all tables
mentioned in the SELECT, merely doing the prefetch is sufficient to
lock both tables; no `on=(...)` is needed to `select_for_update`.

This also does not address the pre-existing potential deadlock from
these two use cases, where both try to lock the same ScheduledEmail
rows in opposite orders.
2021-08-19 00:44:33 -07:00
Alex Vandiver ebaafb32f3 send_email: Change clear_scheduled_emails to only take one user.
No codepath except tests passes in more than one user_profile -- and
doing so is what makes the deduplication necessary.

Simplify the API by making it only take one user_profile id.
2021-08-19 00:44:33 -07:00
Suyash Vardhan Mathur 6fbe7ad61e apidocs: Remove automatically added : at end of response. 2021-08-17 06:25:02 -07:00
Eeshan Garg 190ce61eed landing-page: Fix mobile scrolling bug when sidebar is toggled.
On mobile, when the sidebar is toggled, the following three issues
are encountered:
- When none of the sidebar menus are expanded, the sidebar has no
  scrollbar, which is expected. But if you scroll, the background
  content scrolls, which is a bug.
- When some of the sidebar menus are expanded such that the content
  overflows and is "scrollable", once you get to the end of the
  sidebar content, the background content keeps scrolling in a weird
  way.
- If the mobile screen is wide enough, if you scroll the sidebar
  content, it scrolls as expected. But if you move the pointer to
  the side of the background content that is still visible, you
  can scroll the background content even though it should be fixed.

This commit fixes all of the above issues.
2021-08-17 06:23:46 -07:00
Abhijeet Prasad Bodas ac70a2d2e1 notifications: Fix unnecessary wildcard mention notifications.
This fixes a bug where email notifications were sent for wildcard
mentions even if the `enable_offline_email_notifications` setting was
turned off.
This was because the `notification_data` class incorrectly considered
`wildcard_mentions_notify` as an indeoendent setting, instead of a wrapper
around `enable_offline_email_notifications` and `enable_offline_push_notifications`.

Also add a test for this case.
2021-08-13 09:48:18 -07:00
Alex Vandiver ec354ee013 changelog: Add 2.1.8 release notes. 2021-08-12 01:06:15 +00:00
Alex Vandiver fc9e5f5187 github: Use logging.exception so we know which exception happened. 2021-08-10 22:38:18 -07:00
Anders Kaseorg 2bfe8d5c90 node_tests: Fix incorrect usage of assert.throws.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-10 13:18:53 -07:00
PIG208 c99c423c81 support: Remove `is True` in comparisons.
This is a followup for #19432 that fixes a remaining issue of the
original PR.
2021-08-10 12:08:13 -07:00
Anders Kaseorg 1bdb7b1141 mypy: Add boto3-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg bfdb2f4628 export: Fix error message generation in _check_key_metadata.
There is no key.name.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 05124737f7 mypy: Add backoff-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 09564e95ac mypy: Add types-psycopg2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 5c90522e69 mypy: Add types-Pillow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg b798b34b23 generate_integration_bots_avatars: Fix transparency initialization.
None results in an uninitialized image (that happens to be transparent
most of the time); we want to explicitly initialize the image to
transparent.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 14f0594795 upload: Replace exif_rotate with Pillow exif_transpose.
Fixes #18599.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 6289803368 mypy: Add types-oauthlib.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 4c73926ad9 mypy: Add types-jsonschema.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg 58cfc152a0 mypy: Add types-commonmark.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Priyansh Garg 54452fef6c data_import: Fix channel mentions in Rocket.Chat import.
While the STREAM_LINK_REGEX and STREAM_TOPIC_LINK_REGEX
identifies the stream and topic mentions in the content
correctly (tested by printing out the matches), the
stream/topic mentions are still not linked to the
corresponding streams/topics for imported messages, as
a `zulip_message` instance is required for linking these
mentions to actual streams/topics (see `StreamPattern`
class in `markdown/__init__.py`) which is not provided
while processing the markdown for imported messages.
2021-08-09 06:38:26 -07:00
Priyansh Garg aed4e48da7 data_import: Import attachments from Rocket.Chat. 2021-08-09 06:38:26 -07:00
Priyansh Garg 65e28907cb data_import: Import custom emoji from Rocket.Chat. 2021-08-09 06:38:26 -07:00
Tim Abbott a7890f046b api: Fix tests for avatar_url. 2021-08-09 02:15:55 -07:00
PIG208 a583e9ffad webhooks: Refactor alertmanager to use REQ. 2021-08-08 17:11:18 -07:00
PIG208 4905d73fe4 auth: Refactor oauth_redirect_to_root to use REQ. 2021-08-08 17:11:18 -07:00
PIG208 bf86649f4d billing: Refactor initial_upgrade to use REQ. 2021-08-08 17:11:18 -07:00
PIG208 6a61dc2de2 billing: Refactor billing_home to use REQ. 2021-08-08 17:11:18 -07:00