Commit Graph

44412 Commits

Author SHA1 Message Date
PIG208 460119986b management: Fix typing for management scripts.
There are some remaining errors related to the django `Manager[T]` and
the `List[T]` type that we use to annotate the `Manage[T]` objects.
2021-08-20 05:54:18 -07:00
PIG208 50ce906f31 tornado: Update the `addrport` argument.
The ability to use multiple ports has been removed a long time ago.
And the "optional" note in the help message is in fact incorrect
since `addrport` being `None` is not supported.
2021-08-20 05:49:35 -07:00
Eeshan Garg dea4187aff devlogin: Add support for creating development demo realms.
This is a part of #19523.
2021-08-19 03:09:36 -07:00
Eeshan Garg 082dd99ed6 realm: Add datetime field for scheduled deletion of demo orgs.
This commit is a part of #19523.
2021-08-19 03:09:36 -07:00
Eeshan Garg a808f02a22 billing/sponsorship: Allow blank organization URLs.
We ran into a bug in production caused by two issues:
- Some users came from orgs that didn't have a website and since
  the URL field was required, they submitted invalid URLs.
- We didn't properly respond to invalid form submissions, which
  led to UnboundLocalError exceptions in another part of the
  code.

This commit solves this by doing the following:
- We now allow blank URLs and have a convenient placeholder text
  label that tells users that they may leave the URL field blank.
- This commit refactors the code such that invalid form submissions
  result in an informative error message about what exactly went
  wrong.
2021-08-19 03:08:17 -07:00
Sahil Batra 6117c3824a user groups: Hide system user groups temporarily from UI.
We hide the system groups from the typeaheads and the list
of user groups in settings page temporarily.
2021-08-19 02:56:45 -07:00
Sahil Batra 5f5c88a4e2 user_groups: Add 'is_system_group' field to objects passed to clients. 2021-08-19 02:56:43 -07:00
Sahil Batra 4c290a49d3 user_groups: Do not allow editing system user groups from API.
We do not allow any user to edit the system user groups (including
renaming, deleting, adding or removing members, etc.) from the
API. These user groups will change only by the code when a new
user is added or role of a user is changed.

This is implemented by rejecting access_user_group_by_id always
except the case when it is use to get the user group for sending
email and push notifications, as we would need to send notifications
to the mentioned user group.
2021-08-19 02:54:29 -07:00
Sahil Batra 4bd1dc0a56 models: Add is_system_group field to UserGroup model. 2021-08-19 02:54:29 -07:00
Sahil Batra 0e96a92a6d user_groups: Make description as keyword-only arg in create_user_group.
We make the description parameter in create_user_group as keyword-only
to improve readability. We would also keep the is_system_group
parameter which will be added in future keyword-only.
2021-08-19 02:54:29 -07:00
Sahil Batra 7ea3a53d49 settings: Remove unnecessary code.
We were using incorrect element as we changed the input from
select to radio buttons in 439bc2920a.
This code was originally added in 6cb03ea78e (though in a
different file) and was removed from sometime and then added
again in e7311cdf5d. But in the meantime the input was
changed to radio, which resulted in incorrect code.
2021-08-19 02:50:50 -07:00
Sahil Batra 7b74bd1428 settings: Fix incorrect elements used for emojiset loading spinner.
We were using emojiset_spinner element in destroy_indicator, but
there is no such element and emoji-settings-status element is used
in make_indicator, so destroy_indicator should also use the same
element.

This was added originally in 6cb03ea78e (though in a different
file) and was removed for sometime and then readded in e7311cdf5d.
But between these two commits, the setting was changed to be a
radio element and thus the spinner elements were also changed.
2021-08-19 02:50:50 -07:00
Anders Kaseorg 4b83003b3e html_diff: Remove unused highlight_with_class function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:53:47 -07:00
Anders Kaseorg 89277e84df openapi: Remove unused to_python_type function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:53:47 -07:00
Anders Kaseorg aa0768a1a4 validator: Remove unused check_or function.
check_union is more general.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:52:24 -07:00
Anders Kaseorg 4fe030e6ea validator: Remove unused to_positive_or_allowed_int function.
The last use was removed in 1562ec758e.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:52:24 -07:00
Anders Kaseorg 404ef284bb validator: Remove unused check_tuple function.
Tuples cannot be deserialized from JSON.

While we do use these validators for other things, like event
dictionaries, we have migrated the API away from using those.  The
last use was removed in 4f3d5f2d87

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:41 -07:00
Anders Kaseorg f3e5ed57ae utils: Remove unused split_by function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 58b7a4eb44 utils: Remove unused query_chunker function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 271333301d user_groups: Remove unused check_remove_user_from_user_group function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 4e366bfff5 user_groups: Remove unused check_add_user_to_user_group function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 57acac12fa user_groups: Remove unused user_groups_in_realm function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg fdbde9f9c2 push_notifications: Remove unused num_push_devices_for_user function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg b2239e45ac statistics: Remove unused seconds_usage_between function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 27325eb2ae exceptions: Remove unused to_json method of JsonableError.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 79f7f9b8d9 actions: Remove unused get_emails_from_user_ids function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 6ae5be6e75 exceptions: Remove AbstractEnum in favor of enum.auto.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 0d061f44c1 actions: Remove acting_client parameter from bulk_remove_subscriptions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 0b795e492f thumbnail: Remove unused is_camo_url parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 4206e5f00b python: Remove locally dead code.
These changes are all independent of each other; I just didn’t feel
like making dozens of commits for them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:51:37 -07:00
Anders Kaseorg 9afe6d0829 templates: Fix footer links on /plans.
The footer was being covered by a bunch of invisible divs, rendering
most of the links non-functional.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 01:41:29 -07:00
Anders Kaseorg 672358c5da docs: Migrate from recommonmark to MyST-Parser.
Recommonmark is no longer maintained, and MyST-Parser is much more
complete.

https://myst-parser.readthedocs.io/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 8f2827a65d docs: Remove trailing newlines from code blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg cb61ea69c6 docs: Fix misaligned Markdown source indentation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 25c6d5c99c docs: Fix code span syntax in embedded reST block.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg c3646ec67f docs: Move authentication-methods#ldap anchor to appropriate heading.
Commit 30eaed0378 (#15001) incorrectly
inserted a different section between the anchor and the heading.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Mateusz Mandera fdbde59b07 rate_limit: Add management command to reset auth rate limit.
The auth attempt rate limit is quite low (on purpose), so this can be a
common scenario where a user asks their admin to reset the limit instead
of waiting. We should provide a tool for administrators to handle such
requests without fiddling around with code in manage.py shell.
2021-08-19 00:45:17 -07:00
Alex Vandiver 56344e4765 scheduled_email: Only call .save() if necessary.
Calling `email.save()` is only needed if we altered `email.address`;
it is unnecessary if we called `email.users.add(...)` which will have
done its own INSERT.
2021-08-19 00:44:33 -07:00
Alex Vandiver d4f6b93859 scheduled_email: Create ScheduledEmail objects in a transaction.
This fixes two bugs: the most obvious is that there is a race where a
ScheduledEmail object could be observed in the window between creation
and when users are added; this is a momentary instance when the object
has no users, but one that will resolve itself.

The more subtle is that .save() will, if no records were found to be
updated, _re-create_ the object as it exists in memory, using an
INSERT[1].  Thus, there is a race with `deliver_scheduled_emails`
between when the users are added, and when `email.save()` runs:

 1. Web request creates ScheduledEmail object
 2. Web request creates ScheduledEmailUsers object
 3. deliver_scheduled_emails locks the former, preventing updates.
 4. deliver_scheduled_emails deletes both objects, commits, releasing lock
 5. Web request calls `email.save()`; UPDATE finds no rows, so it
    re-creates the ScheduledEmail object.
 6. Future deliver_scheduled_emails runs find a ScheduledEmail with no
    attending ScheduledEmailUsers objects

Wrapping the logical creation of both of these in a single transaction
avoids both of these races.

[1] https://docs.djangoproject.com/en/3.2/ref/models/instances/#how-django-knows-to-update-vs-insert
2021-08-19 00:44:33 -07:00
Alex Vandiver 4c518c2bba scheduled_email: Consistently lock users table.
Only clear_scheduled_emails previously took a lock on the users before
removing them; make deliver_scheduled_emails do so as well, by using
prefetch_related to ensure that the table appears in the SELECT.  This
is not necessary for correctness, since all accesses of
ScheduledEmailUser first access the ScheduledEmail and lock it; it is
merely for consistency.

Since SELECT ... FOR UPDATE takes an UPDATE lock on all tables
mentioned in the SELECT, merely doing the prefetch is sufficient to
lock both tables; no `on=(...)` is needed to `select_for_update`.

This also does not address the pre-existing potential deadlock from
these two use cases, where both try to lock the same ScheduledEmail
rows in opposite orders.
2021-08-19 00:44:33 -07:00
Alex Vandiver ebaafb32f3 send_email: Change clear_scheduled_emails to only take one user.
No codepath except tests passes in more than one user_profile -- and
doing so is what makes the deduplication necessary.

Simplify the API by making it only take one user_profile id.
2021-08-19 00:44:33 -07:00
Suyash Vardhan Mathur 6fbe7ad61e apidocs: Remove automatically added : at end of response. 2021-08-17 06:25:02 -07:00
Eeshan Garg 190ce61eed landing-page: Fix mobile scrolling bug when sidebar is toggled.
On mobile, when the sidebar is toggled, the following three issues
are encountered:
- When none of the sidebar menus are expanded, the sidebar has no
  scrollbar, which is expected. But if you scroll, the background
  content scrolls, which is a bug.
- When some of the sidebar menus are expanded such that the content
  overflows and is "scrollable", once you get to the end of the
  sidebar content, the background content keeps scrolling in a weird
  way.
- If the mobile screen is wide enough, if you scroll the sidebar
  content, it scrolls as expected. But if you move the pointer to
  the side of the background content that is still visible, you
  can scroll the background content even though it should be fixed.

This commit fixes all of the above issues.
2021-08-17 06:23:46 -07:00
Abhijeet Prasad Bodas ac70a2d2e1 notifications: Fix unnecessary wildcard mention notifications.
This fixes a bug where email notifications were sent for wildcard
mentions even if the `enable_offline_email_notifications` setting was
turned off.
This was because the `notification_data` class incorrectly considered
`wildcard_mentions_notify` as an indeoendent setting, instead of a wrapper
around `enable_offline_email_notifications` and `enable_offline_push_notifications`.

Also add a test for this case.
2021-08-13 09:48:18 -07:00
Alex Vandiver ec354ee013 changelog: Add 2.1.8 release notes. 2021-08-12 01:06:15 +00:00
Alex Vandiver fc9e5f5187 github: Use logging.exception so we know which exception happened. 2021-08-10 22:38:18 -07:00
Anders Kaseorg 2bfe8d5c90 node_tests: Fix incorrect usage of assert.throws.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-10 13:18:53 -07:00
PIG208 c99c423c81 support: Remove `is True` in comparisons.
This is a followup for #19432 that fixes a remaining issue of the
original PR.
2021-08-10 12:08:13 -07:00
Anders Kaseorg 1bdb7b1141 mypy: Add boto3-stubs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00
Anders Kaseorg bfdb2f4628 export: Fix error message generation in _check_key_metadata.
There is no key.name.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-09 20:32:19 -07:00