Commit Graph

2777 Commits

Author SHA1 Message Date
Gaurav Pandey f82aba5a3d compatibility: Shift functions to new module.
Shift functions used for compatibility from
zerver.lib.home (is_outdated_server) and
zerver.view.compatibility (pop_numerals,
version_lt, find_mobile_os,
is_outdated_desktop_app, is_unsupported_browser)
to zerver.lib.compatibility module.
2021-06-05 07:33:25 -07:00
Abhijeet Prasad Bodas 90b6fa7881 reactions: Select Message FOR UPDATE when adding/removing.
This locks the message row while a reaction is being added/removed,
which will handle race conditions caused by deleting the message
at the same time.

We make sure that events work happens outside the transaction,
so that in case there's some problem with the queue processor, the
locks aren't held for too long.

As a nice side-effect, we also handle race conditions from double
adding reactions, because once the message is locked, a duplicate
request will wait till the earlier transaction commits, and hence
will not throw `IntegrityErrors`s (rather, will be handled in our
safety check in the /views code itself), which earlier had to be
handled explicitly.
2021-06-04 08:18:17 -07:00
Abhijeet Prasad Bodas efc2f49e3c submessage: Select Message FOR UPDATE when creating.
This locks the message while creating a submessage, which
will handle race conditions caused by deleting the message
simultaneously.

We make sure that events work happens outside the transaction,
so that in case there's a problem with the queue processor,
the locks aren't held for too long.
2021-06-04 08:18:17 -07:00
Abhijeet Prasad Bodas 5f4113cf60 message delete: Select Message FOR UPDATE when archiving.
Further commits will start locking the message rows while
adding related fields like reactions or submessages,
to handle races caused by deleting the message itself at the
same time.

The message locking implemented then will create a possibility
of deadlocks, where the related field transaction holds a lock
on the message row, and the message-delete transaction holds a
lock on the database row of the related field (which will also
need to be deleted when the message is deleted), and both
transactions wait for each other.

To prevent such a deadlock, we lock the message itself while
it is being deleted, so that the message-delete transaction
will have to wait till the other transaction (which is about
to delete the related field, and also holds a lock on the
message row) commits.

https://chat.zulip.org/#narrow/near/1185943 has more details.
2021-06-04 08:18:17 -07:00
Suyash Vardhan Mathur d1ccf15cd3 apidocs: Fix invalid API page bug.
The current logic to get API pages' title using
OperationID should be used when the first line
of the file explicitly mentions so.

In cases where the files didn't begin with `#` but also
didn't need to get title from OpenAPI summary,
this logic fails and causes Server error.
This particularly happens when the article is invalid,
and the `missing.md` file doesn't need title to be
generated, but doesn't start with `#` either.

This commit fixes the logic of using the generated title and covers the bug.
2021-06-03 10:47:47 -07:00
Gilbert Bishop-White e4d9e58c7d emails: Make it obvious when registering creates new realm.
Checked the email looked OK in `/emails` for both creating realm and
registering within an existing one.

Not sure zerver/tests/test_i18n.py test has been suppressed correctly.

Fixes #17786.
2021-06-02 15:18:03 -07:00
Gaurav Pandey f67fc37f7d settings: Rename MAX_EMOJI_FILE_SIZE.
Rename MAX_EMOJI_FILE_SIZE to
MAX_EMOJI_FILE_SIZE_MIB reflecting
the size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey 73059dd0c7 settings: Rename MAX_LOGO_FILE_SIZE.
Rename MAX_LOGO_FILE_SIZE to
MAX_LOGO_FILE_SIZE_MIB reflecting
the size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey 2ddc62e9b3 settings: Rename MAX_ICON_FILE_SIZE.
Rename MAX_ICON_FILE_SIZE to
MAX_ICON_FILE_SIZE_MIB reflecting
size in mebibytes.
2021-06-02 13:55:15 -07:00
Gaurav Pandey 0f6bb210a6 settings: Rename MAX_AVATAR_FILE_SIZE.
Rename MAX_AVATAR_FILE_SIZE to
MAX_AVATAR_FILE_SIZE_MIB reflecting
size in mebibytes.
2021-06-02 13:55:15 -07:00
Mateusz Mandera e17758f8ad auth: Implement a generic OpenID Connect backend.
Fixes #11939.
2021-05-28 09:43:06 -07:00
Suyash Vardhan Mathur 52b9c96c5d apidocs: Add support for using OpenAPI title for /api pages.
The current logic of geneerating HTML titles requires the title to be
present as a heading in the first line of .md file. However, this will
shortly be no longer true for /api pages where these are
auto-generated from OpenAPI data.  Modified the code to fetch the
title from OpenAPI data in case of such pages.
2021-05-26 17:38:01 -07:00
Mateusz Mandera d17a4f64d6 registration: Return status code 404 if invalid confirmation key passed.
We were returning 200 before - 404 is more appropriate.
2021-05-20 17:29:09 -07:00
Mateusz Mandera 40169353ad auth: Validate multiuse invite link confirmation key in social auth. 2021-05-20 17:29:09 -07:00
Tim Abbott e1c885aeb9 home: Remove unused has_mobile_devices parameter.
This parameter has never been used, and causes an unnecessary database
query.

We keep the num_push_devices_for_user function, since we may have uses
for it down the line.

Fixes part of #14166.
2021-05-18 17:36:55 -07:00
Anders Kaseorg e015f3ed7d docs: Correct “webapp” to “web app”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-14 13:15:38 -07:00
Anders Kaseorg 668b5137b0 version: Display Zulip version in About Zulip dialog.
We record Git details about the merge-base with upstream branches in
the zulip-git-version file, if the upstream repository is available.

Note that the first Git upgrade after merging the parent commit will
not include the merge-base details, since the upstream repository will
not have been available.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-13 11:36:12 -07:00
Vishnu KS 051dab58ea navbar: Add gear menu advertisement for sponsoring zulip. 2021-05-12 10:21:48 -07:00
Gaurav Pandey 96e035a2f0 api: Fix encoding of strings in streams endpoint.
* Remove unnecessary json_validator for string parameters.
* Update frontend to pass right parameter.

Bump api feature level and highlight the fix for `emojiset`
parameter of `settings/display` endpoint in zulip.yaml file.

Fixes part of #18035.
2021-05-10 10:29:22 -07:00
Gaurav Pandey 9abe1cafbe api: Fix encoding of strings in display settings endpoint.
* Remove unnecessary json validator for string validator.
* Update frontend to pass right validator.
* Update zulip.yaml to pass right parameter for curl request
  in openapi.
* Update python_examples to pass right paramater.

Fixes part of #18035.
2021-05-10 10:03:32 -07:00
Anders Kaseorg 544bbd5398 docs: Fix capitalization mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-10 09:57:26 -07:00
PIG208 7150fe5dc5 backend: Extract check_update_message from update_message_backend. 2021-05-09 20:44:04 -07:00
Gaurav Pandey 00647b6fa7 api: Fix encoding of strings in video calls endpoint.
This removes unnecessary json_validator for string parameters in the
BigBlueButton video calls endpoints.  Note that this breaks links to
video meetings sent before the upgrade; there's not much we can do
about that.

Since this is the last commit in this series, we update the
ZULIP_FEATURE_LEVEL for this batch of changes.

Fixes part of #18035.
2021-05-07 14:11:48 -07:00
Gaurav Pandey 36ad9b7d0e api: Fix encoding of strings in user settings endpoints.
* Remove unnecessary json_validator for string parameters.
* Remove unnecessary JSON encoding in frontend calls.  Structurally,
  JavaScript does correct encoding without explicit JSON encoding.

Fixes part of #18035.
2021-05-07 14:10:45 -07:00
Gaurav Pandey 232c7107eb api: Fix encoding of strings in tutorial endpoint.
* Remove unnecessary json validator for string parameter.
* Update frontend to pass right parameter.

Fixes part of #18035.
2021-05-07 14:06:54 -07:00
Gaurav Pandey d4219da254 api: Fix encoding of strings in realm domain endpoint.
* Remove unnecessary json_validator for string parameters.
* Update frontend to pass right parameters.

Fixes part of #18035.
2021-05-07 14:06:54 -07:00
Gaurav Pandey b2111f5753 api: Fix encoding of strings in default stream groups.
Remove unnecessary json_validator for string parameters.  This change
does not modify JavaScript because we don't have a frontend for these
API endpoints yet.

Fixes part of #18035.
2021-05-07 14:06:37 -07:00
Gaurav Pandey d6cf1ef27b api: Fix encoding of strings in hotspot endpoint.
* Remove unnecessary json_validator for string parameter.
* Update frontend to pass right parameters.

Fixes part of #18035.
2021-05-07 11:45:25 -07:00
Mateusz Mandera 4006bb5153 auth: Improve display of errors when user needs to reset password.
Raising jsonableError in the authentication form was non-ideal because
it took the user to an ugly page with the returned json.

We also add logging of this rare occurence of the scenario being
handled here.
2021-05-07 09:10:45 -07:00
Anders Kaseorg 405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00
Cyril Pletinckx e4ff372fc3 emails: Transform SMTPException into EmailNotDeliveredException.
Django's default SMTP implementation can raise various exceptions
when trying to send an email. In order to allow Zulip calling code
to catch fewer exceptions to handle any cause of "email not
sent", we translate most of them into EmailNotDeliveredException.
The non-translated exceptions concern the connection with the
SMTP server. They were not merged with the rest to keep some
details about the nature of these.

Tests are implemented in the test_send_email.py module.
2021-05-05 20:16:11 -07:00
Tim Abbott 97b08b77df registration: Improve error handling for source_realm_id.
This restores the previous behavior that we store None rather than
throwing an exception in the presence of invalid input.
2021-05-02 11:12:49 -07:00
Siddharth Asthana 55da3329ef registration: Rename source_realm field to source_realm_id.
Now that we are passing source realm's id instead of string_id in
source realm selector, it makes sense to rename the "source_realm" field
to "source_realm_id".
2021-05-02 11:12:49 -07:00
Siddharth Asthana 4262c04db1 registration: Encode source realm as an integer.
In the source realm selector, when we select a realm from which we want
to import the data, we pass the source realm's string_id. The problem
with this approach is that the string_id can be an empty string. This
commit makes the source_realm pass the realm's id instead of string_id.
Now, the source_realm's value will either be an integer or "" (empty
string) when we don't want to import settings from any realm.
2021-05-02 11:12:49 -07:00
sahil839 e669818544 invites: Allow user to be invited as a moderator.
This commit adds both frontend and backend code to invite a user as
moderator. We allow only existing owners and admins to invite a user
as a moderator.
2021-04-30 15:57:09 -07:00
sahil839 a4c3224328 users: Remove redundant get_role_for_new_user in lib/create_user.py.
The function get_role_for_new_user was added to get role from the
invited_as value, as invited_as values were one of (1,2,3,4)
previously, but it was then changed to be the actual role value,
i.e. one of (100, 200, 400, 600), in 1f8f227444.

So, we can safely remove this function now and use invited_as value
directly and handle realm_creation case by using an if condition.
2021-04-30 15:57:09 -07:00
Ganesh Pawar ddf2127035 widgets: Prevent edits to widgets.
As of now, editing a widget doesn't update the rendered content.
It's important to ensure that existing votes or options added later on
don't get deleted when rendered.
This seems more complex than it's worth.

For now, we just prevent edits to widgets.
This commit makes the UI clearer that editing widgets isn't allowed.

See also:
https://github.com/zulip/zulip/issues/14229
https://github.com/zulip/zulip/issues/14799

Fixes #17156
2021-04-30 09:55:25 -07:00
Anders Kaseorg 871e73ab8f mypy: Don’t use Iterable for values iterated multiple times.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-29 16:06:17 -07:00
Tim Abbott a965c77af6 unsubscribe: Configure as csrf_exempt.
The reason we didn't have this before is that a GET to one of these
URLs would suffice to unsubscribe the user; but with the
List-Unsubscribe system, we need to allow POST from a third-party page
as well.
2021-04-28 15:33:32 -07:00
PIG208 9c0ce19104 reactions: Extract check_add_reaction from add_reaction. 2021-04-28 09:11:08 -07:00
PIG208 35fae79422 backend: Use JsonableError instead for update_message_backend. 2021-04-28 08:32:21 -07:00
Tim Abbott 50f00d3e97 emails: Add unsubscribe link infrastructure for marketing emails.
This is intended to be used in any marketing emails that we send with
the send_custom_email infrastructure.
2021-04-27 21:36:09 -07:00
Dinesh 27e4f5da92 typing: Support sending stream/topic typing status.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
    - recipients for a PM
    - stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).

On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
2021-04-27 20:52:21 -07:00
Dinesh 734d935d4a client_capabilities: Add stream_typing_notifications. 2021-04-27 20:52:21 -07:00
akshatdalton b573964bfa notifications: Add support for `None` option in `Notification sound`.
This commit adds support for a `None` option in the dropdown menu
of `Notification sound`. When this option is selected, no audible
notification is sent to the user.

`None` will appear as the first option in the dropdown menu, since
this is not categorized as a playable audio.

This new option is added so that folks can disable audio notifications
without losing their other notification configuration (like for PMs, mentions).

Necessary test case is added for this new option.

Fixes #16090.
2021-04-27 16:42:16 -07:00
sahil839 a84725de9b dev_login: Show 'shiva' in a separate Moderators section. 2021-04-26 17:14:06 -07:00
Anders Kaseorg 6060d0d364 docs: Add missing space to compound verbs “log in”, “set up”, etc.
Noun: backup, checkout, cleanup, login, logout, setup, shutdown, signup,
timeout.

Verb: back up, check out, clean up, log in, log out, set up, shut
down, sign up, time out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg e3f2ffa681 docs: Capitalize “Markdown” consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Riken Shah 0e5f6c21ad refactor: Split dev related code from `auth.py` into `dev_login.py`.
This refactor will help us make our development-related authentication
code to avoid production security impact.
2021-04-23 15:11:44 -07:00
Mateusz Mandera 1a8ad796f8 models: Replace __id syntax with _id where possible.
model__id syntax implies needing a JOIN on the model table to fetch the
id. That's usually redundant, because the first table in the query
simply has a 'model_id' column, so the id can be fetched directly.
Django is actually smart enough to not do those redundant joins, but we
should still avoid this misguided syntax.

The exceptions are ManytoMany fields and queries doing a backward
relationship lookup. If "streams" is a many-to-many relationship, then
streams_id is invalid - streams__id syntax is needed. If "y" is a
foreign fields from X to Y:
class X:
  y = models.ForeignKey(Y)

then object x of class X has the field x.y_id, but y of class Y doesn't
have y.x_id. Thus Y queries need to be done like
Y.objects.filter(x__id__in=some_list)
2021-04-22 14:53:00 -07:00