Commit Graph

13298 Commits

Author SHA1 Message Date
Aman Agrawal f47e93481e giphy: Set rating according to realm_giphy_rating.
We set rating of GIFs retrieved from GIPHY according to
realm_giphy_rating setting. Also, we allow user to set
a rating in organization settings.
2021-04-28 07:07:36 -07:00
Tim Abbott a0e18cb06a send_custom_email: Improve printing of outgoing emails.
The fake email addresses are not useful for debugging/confirmation,
and additionally it's nice to have the realms listed as well.
2021-04-27 21:36:17 -07:00
Tim Abbott 123a46e41d send_custom_email: Add option for emailing sponsored org admins.
Long-term, we probably want to make the filtering options more
generic, but there's little harm in adding an option for a specific
group we're likely to email multiple times.
2021-04-27 21:36:17 -07:00
Tim Abbott be2d93036d emails: Include marketing unsubscribe link in send_custom_email context. 2021-04-27 21:36:09 -07:00
Tim Abbott 50f00d3e97 emails: Add unsubscribe link infrastructure for marketing emails.
This is intended to be used in any marketing emails that we send with
the send_custom_email infrastructure.
2021-04-27 21:36:09 -07:00
Tim Abbott 1afc0496ca event_schema: Fix translation of typing events message_type.
This was incorrectly/lazily marked as a string, not as an enum of two
string values, in the previous commit.
2021-04-27 21:35:32 -07:00
Dinesh 27e4f5da92 typing: Support sending stream/topic typing status.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
    - recipients for a PM
    - stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).

On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
2021-04-27 20:52:21 -07:00
Dinesh 734d935d4a client_capabilities: Add stream_typing_notifications. 2021-04-27 20:52:21 -07:00
Tim Abbott 2bdbbbd234 api docs: Add design details for muted users.
And also link to those details from the documentation.

We may later (partially?) move this content to Help Center
documentation on the feature.
2021-04-27 18:10:39 -07:00
Tim Abbott 05f2ebb2b8 settings: Include units in SERVER_UPGRADE_NAG_DEADLINE_DAYS name.
When naming things with units of time involved, it's always good to
include units in the name, so that the meaning is clear.
2021-04-27 17:15:16 -07:00
Tim Abbott a3640223b8 security: Check when server directory was unpacked for upgrade nag.
The comment explains in more detail, but this should help avoid cases
where a Zulip server accidentally avoids the nag by having upgraded to
a 2-year old Zulip version from a 3-year-old version 2 months ago.
2021-04-27 17:09:27 -07:00
Tim Abbott 5607365b88 emails: Fix incorrect onboarding email going to realm owners.
This was apparently missed when implementing realm owners last year.
2021-04-27 17:09:27 -07:00
akshatdalton b573964bfa notifications: Add support for `None` option in `Notification sound`.
This commit adds support for a `None` option in the dropdown menu
of `Notification sound`. When this option is selected, no audible
notification is sent to the user.

`None` will appear as the first option in the dropdown menu, since
this is not categorized as a playable audio.

This new option is added so that folks can disable audio notifications
without losing their other notification configuration (like for PMs, mentions).

Necessary test case is added for this new option.

Fixes #16090.
2021-04-27 16:42:16 -07:00
Nyior 8e2042d378 security: Add tooling to nag users if a Zulip server is very old.
This will help ensure that users upgrade their Zulip server.

Essentially rewritten by tabbott.

Fixes part of #17826.
2021-04-27 12:10:48 -07:00
Siddharth Asthana a81c4b5e4c send_custom_email: Add dry-run flag to verify recipients and email.
Add a `--dry-run` flag to send_custom_email management command
in order to provide a mechanism to verify the emails of the recipients
and the text of the email being sent before actually sending them.

Add tests to:
- Check that no emails are actually sent when we are in the dry-run mode.
- Check if the emails are printed correctly when we are in the dry-run mode.

Fixes #17767
2021-04-26 17:31:34 -07:00
Cyril Pletinckx 9afde790c6 email: Open a single SMTP connection to send email batches.
Previously the outgoing emails were sent over several SMTP
connections through the EmailSendingWorker; establishing a new
connection each time adds notable overhead.

Redefine EmailSendingWorker worker to be a LoopQueueProcessingWorker,
which allows it to handle batches of events. At the same time, persist
the connection across email sending, if possible.

The connection is initialized in the constructor of the worker
in order to keep the same connection throughout the whole process.
The concrete implementation of the consume_batch function is simply
processing each email one at a time until they have all been sent.

In order to reuse the previously implemented decorator to retry
sending failures a new method that meets the decorator's required
arguments is declared inside the EmailSendingWorker class. This
allows to retry the sending process of a particular email inside
the batch if the caught exception leaves this process retriable.

A second retry mechanism is used inside the initialize_connection
function to redo the opening of the connection until it works or
until three attempts failed. For this purpose the backoff module
has been added to the dependencies and a test has been added to
ensure that this retry mechanism works well.

The connection is closed when the stop method is called.

Fixes: #17672.
2021-04-26 17:27:22 -07:00
Alex Vandiver 0ad17925eb send_email: Remove unnecessary send_email_from_dict.
This was introduced in 8321bd3f92 to serve as a sort of drop-in
replacement for zerver.lib.queue.queue_json_publish, but its use has
been subsequently cut out (e.g. `9fcdb6c83ac5`).

Remote its last callsite.
2021-04-26 17:27:22 -07:00
sahil839 4f716d4ad8 management: Include moderator option in change_user_role command.
We now allow to change a user role to moderator and to change
role from moderator to any other role using change_user_role
command.
2021-04-26 17:14:06 -07:00
sahil839 a84725de9b dev_login: Show 'shiva' in a separate Moderators section. 2021-04-26 17:14:06 -07:00
Mateusz Mandera b7b1ec0aeb outgoing_webhook: Improve invalid json handling when parsing response.
It's better to just raise JsonableError here, as that makes this error
processed in the central place for this kind of thing in do_rest_call:
---------
except JsonableError as e:
    response_message = e.msg
    logging.info("Outhook trigger failed:", stack_info=True)
    fail_with_message(event, response_message)
    response_message = f"The outgoing webhook server attempted to send a message in Zulip, but that request resulted in the following error:\n> {e}"
    notify_bot_owner(event, failure_message=response_message)
    return None
----------

which does all the things that are supposed to happen -
fail_with_message, appropriate logging and notifying the bot owner.
2021-04-26 09:32:35 -07:00
Mateusz Mandera f1a3f6056a outgoing_webhook: Fix invalid ResponseMock(200) mocks in tests.
These aren't good mocks of a good reponse - a good response is supposed
to contain valid json that doesn't trigger error-handling in the
codepath. Without this change, all these actually trip up on
json.loads(response.text) in process_success_response.
2021-04-26 09:32:35 -07:00
Mateusz Mandera b998138d3a outgoing_webhook: Handle valid, but unexpected json in response.
Responses such as "null" or "true" are valid json, but json.loads
returns different objects than dicts that the codepath expects.

Fixes #18223.
2021-04-26 09:32:35 -07:00
Anders Kaseorg 6060d0d364 docs: Add missing space to compound verbs “log in”, “set up”, etc.
Noun: backup, checkout, cleanup, login, logout, setup, shutdown, signup,
timeout.

Verb: back up, check out, clean up, log in, log out, set up, shut
down, sign up, time out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg e3f2ffa681 docs: Capitalize “Markdown” consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg 178736c8eb docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Aman Agrawal ebe822341d message_edit: Don't add content edit to all messages.
Remove content edit keys if present in edit_history_event
when passing to update_messages_for_topic_edit.

Since content edit is only applied to the edited_message,
this shouldn't be part of the rest of the messages for which
topic was edited. This was a bug identified by
editing topic and content of a message at the same time
when more than 1 message is affected.
2021-04-24 13:51:49 -07:00
Anders Kaseorg cef2ecf004 i18n: Pass language name with every response.
With a comment added by tabbott on live-update considerations.

See https://chat.zulip.org/#narrow/stream/9-issues/topic/FormatJS.20error.20in.20standalone.20pages/near/1168413 for context.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-24 13:00:27 -07:00
Aman Agrawal 79d748ba7d message_edit: Use update_edit_history to update message history. 2021-04-23 15:12:09 -07:00
Aman Agrawal de50f4ae25 message_edit: Extract update_edit_history. 2021-04-23 15:12:09 -07:00
Aman Agrawal 736fdcda49 update_messages_for_topic_edit: Remame `message` variable. 2021-04-23 15:12:09 -07:00
Riken Shah 0e5f6c21ad refactor: Split dev related code from `auth.py` into `dev_login.py`.
This refactor will help us make our development-related authentication
code to avoid production security impact.
2021-04-23 15:11:44 -07:00
Mateusz Mandera 1a8ad796f8 models: Replace __id syntax with _id where possible.
model__id syntax implies needing a JOIN on the model table to fetch the
id. That's usually redundant, because the first table in the query
simply has a 'model_id' column, so the id can be fetched directly.
Django is actually smart enough to not do those redundant joins, but we
should still avoid this misguided syntax.

The exceptions are ManytoMany fields and queries doing a backward
relationship lookup. If "streams" is a many-to-many relationship, then
streams_id is invalid - streams__id syntax is needed. If "y" is a
foreign fields from X to Y:
class X:
  y = models.ForeignKey(Y)

then object x of class X has the field x.y_id, but y of class Y doesn't
have y.x_id. Thus Y queries need to be done like
Y.objects.filter(x__id__in=some_list)
2021-04-22 14:53:00 -07:00
Alex Vandiver 11177a40da soft_deactivate: Log and continue on failure to catch up a user.
There exists a logic bug (see #18236) which causes duplicate
usermessage rows to be inserted.  Currently, this stops catch-up for
all users.

Catch and record the exception for each affected user, so we at least
make catch-up progress on other users.
2021-04-22 14:38:03 -07:00
Mateusz Mandera 8d4ab69a46 docs: Move the /configure-missed-message-emails help page.
configure-message-notification-emails is the correct name now.
2021-04-21 10:10:54 -07:00
Mateusz Mandera 977a2f7fa0 emails: Rename "missed message email" to "message notification email". 2021-04-21 10:10:54 -07:00
Mateusz Mandera cc11575b21 emails: Rename "New missed messages" email subjects to "New messages". 2021-04-21 10:10:54 -07:00
Mateusz Mandera 716449030d emails: Rename missed message email sender to "Zulip notifcations".
It was decided that this is more appropriate naming. "Missed message"
gives it a bit of a sound like something went wrong.
2021-04-21 10:10:54 -07:00
Anders Kaseorg dd3fa4ac52 templates: Mark all void tags as self-closing.
This reverses the policy that was set, but incompletely enforced, by
commit 951514dd7d.  The self-closing tag
syntax is clearer, more consistent, simpler to parse, compatible with
XML, preferred by Prettier, and (most importantly now) required by
FormatJS.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 09:49:34 -07:00
Tim Abbott 6346b9d3eb models: Replace user_profile__is_active queries with is_user_active.
This saves a couple database queries by using the recently added
denormalization for Subscription objects.
2021-04-19 18:30:31 -07:00
Tim Abbott a1cfe25f8d streams: Move can_access_stream_user_ids into streams.py.
This belongs either here or in stream_subscription.py, which arguably
should just be merged into streams.py anyway.
2021-04-19 18:30:31 -07:00
Tim Abbott e4932bd952 actions: Fixed deactivated user IDs being included for stream events.
This was a mostly harmless bug, since those users cannot have active
clients, but fixing it will improve performance in any Zulip
organization where the vast majority of users are deactivated.
2021-04-19 18:30:31 -07:00
akshatdalton 6509c4f8f4 linkifiers: Add an API to support the editing of linkifier.
This commit adds an API to `zproject/urls.py` to edit/update
the realm linkifier. Its helper function to update the
database is added in `zerver/lib/actions.py`.

`zulip.yaml` is documented accordingly as well, clearly
stating that this API updates one linkifier at a time.

The tests are added for the API and helper function which
updates the realm linkifier.

Fixes #10830.
2021-04-19 18:01:45 -07:00
akshatdalton c180cd5fa1 Refactor: Use backend-endpoint function instead of helper function.
Use backend-endpoint function instead of helper function in
`test_realm_linkifiers.py` so that tests are more end-to-end.

The removed helper function: `do_add_linkifier` is tested in
`zerver/tests/test_events.py`.
2021-04-19 18:01:45 -07:00
akshatdalton b29bd71a9c Refactor: Use `id` instead of `pk` as key.
Use `id` instead of `pk` as key to get RealmFilter
object in `do_remove_linkifier` function in `actions.py`.
2021-04-19 18:01:45 -07:00
akshatdalton 5bbfa24d74 Refactor: Rename `Filter` to `Linkifier`.
Linkifier error message: `Filter not found` is
updated to `Linkifier not found.`.

Similarly, `filter_id` description is updated to:
`The ID of the linkifier that you want to remove.`,
renamed the term `filter` with `linkifier`, in `zulip.yaml`.
2021-04-19 18:01:45 -07:00
Aman Agrawal 70d444a8eb compose: Move to `hbs` file and append it to DOM on init.
We move compose.html to compose.hbs file while keeping
`#compose` still in `home.html` as a hanger
where append rest of the elements.

This will provide us with two benefits:
* We could share common elements between message_edit_form and
compose.
* We can insert compose directly in any element. We may decide to
do it for recent topics.
2021-04-19 16:53:30 -07:00
Tim Abbott 3810eaaa25 api docs: Document that servers ignore unexpected event types.
Clients like the mobile and terminal apps will want to rely on this
detail of our API, so we should write it down.
2021-04-19 14:14:14 -07:00
Sumanth V Rao f9b79999ed playgrounds: Avoid json encoding each field in POST /realm/playgrounds.
This prevents us from having to json encode every field in the POST
request to /realm/playgrounds, and keeps the client logic simpler
when adding a playground.
2021-04-19 11:23:09 -07:00
Riken Shah 1d9417f9a1 refactor: Rename `dev_panel.html` for more clarity.
This commit renames `dev_panel.html` to `integrations_dev_panel.html`.
2021-04-19 10:25:05 -07:00
Riken Shah bd9e6ae97e refactor: Move dev-only templates to `templates/zerver/development`.
This a prep commit for adding a check to lint to ensure
only development-related templates entrypoint name
starts with `dev`.
2021-04-19 10:25:05 -07:00