Commit Graph

5487 Commits

Author SHA1 Message Date
sahil839 6e672ebbee settings: Add moderators option for wildcard_mention_policy. 2021-05-03 12:12:01 -07:00
sahil839 141232749b test_message_send: Modify test_wildcard_mention_restrictions.
This commit modifies the test_wildcard_mention_restrictions test
for checking that moderators are not allowed to send messages
with wildcard mention if wildcard_mention_policy is set to
WILDCARD_MENTION_POLICY_ADMINS. Previously, we were checking
for members, but it is better to check for moderators.
2021-05-03 12:12:01 -07:00
Tim Abbott 1489c24860 test_decorators: Don't try to mutate is_staff.
Apparently, after upgrading to Django 3.2, mutating is_staff and then
saving can result in a user's session being destroyed.

In any case, this test is probably better written using two different
users with the different roles, which we have in our initial database
anyway.
2021-05-03 09:26:00 -07:00
Tim Abbott 935a7b18cb test_signup: For test overly sensitive to whitespace.
This test started failing after a rebase from a branch 2 weeks ago
where it passed.
2021-05-02 11:35:32 -07:00
Tim Abbott 97b08b77df registration: Improve error handling for source_realm_id.
This restores the previous behavior that we store None rather than
throwing an exception in the presence of invalid input.
2021-05-02 11:12:49 -07:00
Siddharth Asthana 55da3329ef registration: Rename source_realm field to source_realm_id.
Now that we are passing source realm's id instead of string_id in
source realm selector, it makes sense to rename the "source_realm" field
to "source_realm_id".
2021-05-02 11:12:49 -07:00
Siddharth Asthana 4262c04db1 registration: Encode source realm as an integer.
In the source realm selector, when we select a realm from which we want
to import the data, we pass the source realm's string_id. The problem
with this approach is that the string_id can be an empty string. This
commit makes the source_realm pass the realm's id instead of string_id.
Now, the source_realm's value will either be an integer or "" (empty
string) when we don't want to import settings from any realm.
2021-05-02 11:12:49 -07:00
Anders Kaseorg 995389b4c1 markdown: Don’t apply further Markdown processing to KaTeX output.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-01 15:43:35 -07:00
Anders Kaseorg 802c8de0e8 fenced_code: Wrap math blocks in the frontend like the backend.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-01 15:36:15 -07:00
pilgrim2308 a74b52db22 UI: Add Zulip version in gear menu.
Currently only enabled in development, since the exact details don't
seem right..

Co-Author-By: Signior-X <b19188@students.iitmandi.ac.in>
Co-Author-By: Aman Agrawal <amanagr@zulip.com>

Implements UI for #8005.
2021-04-30 17:58:08 -07:00
sahil839 e669818544 invites: Allow user to be invited as a moderator.
This commit adds both frontend and backend code to invite a user as
moderator. We allow only existing owners and admins to invite a user
as a moderator.
2021-04-30 15:57:09 -07:00
Alex Vandiver fd1774dcba push_notifications: Give full stack information on an exception.
This error has been seen in production instances, but we need more
context to be able to determine what might be causing it.
2021-04-30 14:03:52 -07:00
Wesley Aptekar-Cassels 6b7a3fb74a markdown: Rewrite all external images to use Camo.
Requesting external images is a privacy risk, so route all external
images through Camo.

Tweaked by tabbott for better test coverage, more comments, and to fix
bugs.
2021-04-30 10:36:16 -07:00
Ganesh Pawar ddf2127035 widgets: Prevent edits to widgets.
As of now, editing a widget doesn't update the rendered content.
It's important to ensure that existing votes or options added later on
don't get deleted when rendered.
This seems more complex than it's worth.

For now, we just prevent edits to widgets.
This commit makes the UI clearer that editing widgets isn't allowed.

See also:
https://github.com/zulip/zulip/issues/14229
https://github.com/zulip/zulip/issues/14799

Fixes #17156
2021-04-30 09:55:25 -07:00
Vishnu KS 7f3fc3423b audit log: Create audit log when a realm is created.
This is mainly useful in recording the user who created the realm,
when possible.
2021-04-30 09:25:11 -07:00
Ganesh Pawar 830f1fa8c5 upload: Refactor and add tests for ensure_avatar_image in upload.py.
`ensure_basic_avatar_image` and `ensure_medium_avatar_image` are
essentially the same thing, except a size parameter.
So, refactor them into a single function.

This doesn't introduce any functional changes.
2021-04-29 21:18:13 -07:00
Tim Abbott 615ad2d5d8 middleware: Simplify logic for parsing user-agent.
This avoids calling parse_user_agent twice when dealing with official
Zulip clients, and also makes the logical flow hopefully easier to read.

We move get_client_name out of decorator.py, since it no longer
belongs there, and give it a nicer name.
2021-04-29 17:47:41 -07:00
orientor 6224d83dea middleware: Get client name in LogRequests instead of process_client.
This ensures it is present for all requests; while that was already
essentially true via process_client being called from every standard
decorator, this allows middleware and other code to rely on this
having been set.
2021-04-29 17:03:05 -07:00
Anders Kaseorg 871e73ab8f mypy: Don’t use Iterable for values iterated multiple times.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-29 16:06:17 -07:00
sahil839 bd78b1ff90 events: Add 'is_moderator' field to the page_params object sent to clients. 2021-04-29 15:18:49 -07:00
sahil839 dc771f3a14 users: Add support for changing user role to moderator in api.
This commit adds backend support for changing a user role to
moderator and also to change role from moderator to any other
role.
2021-04-29 15:17:45 -07:00
sahil839 34f134d58d users: Add role field to user objects returned by format_user_row.
This commit modifies the user objects returned by 'GET /users',
'GET /users/me', 'GET /users/{user_id}' and 'GET /users/{email}'
endpoints to include role field.

We also include role field in the page_params['realm_users'] dict
and in the person object sent in (type="realm_user", op="add")
event.
2021-04-29 15:13:50 -07:00
Alex Vandiver 8711ab7676 outgoing_webhook: Add a logging statement for each outgoing webhook.
This will help determine potentail timeout lengths, as well as serve
as a generally-useful log for locations which do not have Smokescreen
enabled.

In service of #17742.
2021-04-29 12:37:14 -07:00
m-e-l-u-h-a-n 65c400e06d api: Add zulip_version and zulip_feature_level in restart event.
This help mobile and terminal clients understand whether a server
restart changed API feature levels or not, which in turn determines
whether they will need to resynchronize their data.

Also add tests and documentation for this previously undocumented
event type.

Fixes: #18205.
2021-04-29 12:08:15 -07:00
m-e-l-u-h-a-n d2c18e28a4 api: Handle restart events in apply_events.
Event of type restart could not be handled properly, because of
its special behavior. For handling this event in most natural way
we recursively call `do_events_register` when restart event is
recieved, based on custom error created for this event.

Testing: Second call to get_user_events due to recursive calling
of do_event_register, is expected to not contain the restart event.
So new test added in test_event_system.py are based on above behavior
of get_user_events.

Fixes: #15541.
2021-04-29 11:40:59 -07:00
Anders Kaseorg 31105b78f2 test_signup: Test List-Unsubscribe POST request.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-28 17:19:35 -07:00
Anders Kaseorg f4b3c15fe8 test_signup: Fix copy-paste mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-28 17:19:35 -07:00
strifel 02c5676a67 ldap: Add advanced LDAP realm access control.
This allows access to be more configurable than just setting one
attribute.  This can be configured by setting the setting
AUTH_LDAP_ADVANCED_REALM_ACCESS_CONTROL.
2021-04-28 09:08:13 -07:00
PIG208 d20deccee7 tests: Add a test for fixtures for non-webhook integrations.
This commit create a directory to store the mock message for nagios and
more will be added.

The json files in this directory will be used to config the screenshot
generating script for the documentations of non-webhook integrations.
2021-04-28 08:16:51 -07:00
im-adithya 009b7bca24 alert_words: Fix highlighting of adjacent alert words.
This prevents the regex from requiring multiple spaces between
adjacent alert words by using lookahead and lookbehind (rather than
the before/after checks each needing to eat a whitespace character) so
that consecutive alert words (if any) can be highlighted.

With a frontend test covering adjacent corner cases by tabbott.

Fixes #17320
2021-04-28 07:54:50 -07:00
Tim Abbott a0e18cb06a send_custom_email: Improve printing of outgoing emails.
The fake email addresses are not useful for debugging/confirmation,
and additionally it's nice to have the realms listed as well.
2021-04-27 21:36:17 -07:00
Tim Abbott 50f00d3e97 emails: Add unsubscribe link infrastructure for marketing emails.
This is intended to be used in any marketing emails that we send with
the send_custom_email infrastructure.
2021-04-27 21:36:09 -07:00
Dinesh 27e4f5da92 typing: Support sending stream/topic typing status.
This extends the /json/typing endpoint to also accept
stream_id and topic. With this change, the requests
sent to /json/typing should have these:
* `to`: a list set to
    - recipients for a PM
    - stream_id for a stream message
* `topic`, in case of stream message
along with `op`(start or stop).

On receiving a request with stream_id and topic, we send
typing events to clients with stream_typing_notifications set
to True for all users subscribed to that stream.
2021-04-27 20:52:21 -07:00
Dinesh 734d935d4a client_capabilities: Add stream_typing_notifications. 2021-04-27 20:52:21 -07:00
Tim Abbott 05f2ebb2b8 settings: Include units in SERVER_UPGRADE_NAG_DEADLINE_DAYS name.
When naming things with units of time involved, it's always good to
include units in the name, so that the meaning is clear.
2021-04-27 17:15:16 -07:00
Nyior 8e2042d378 security: Add tooling to nag users if a Zulip server is very old.
This will help ensure that users upgrade their Zulip server.

Essentially rewritten by tabbott.

Fixes part of #17826.
2021-04-27 12:10:48 -07:00
Siddharth Asthana a81c4b5e4c send_custom_email: Add dry-run flag to verify recipients and email.
Add a `--dry-run` flag to send_custom_email management command
in order to provide a mechanism to verify the emails of the recipients
and the text of the email being sent before actually sending them.

Add tests to:
- Check that no emails are actually sent when we are in the dry-run mode.
- Check if the emails are printed correctly when we are in the dry-run mode.

Fixes #17767
2021-04-26 17:31:34 -07:00
Cyril Pletinckx 9afde790c6 email: Open a single SMTP connection to send email batches.
Previously the outgoing emails were sent over several SMTP
connections through the EmailSendingWorker; establishing a new
connection each time adds notable overhead.

Redefine EmailSendingWorker worker to be a LoopQueueProcessingWorker,
which allows it to handle batches of events. At the same time, persist
the connection across email sending, if possible.

The connection is initialized in the constructor of the worker
in order to keep the same connection throughout the whole process.
The concrete implementation of the consume_batch function is simply
processing each email one at a time until they have all been sent.

In order to reuse the previously implemented decorator to retry
sending failures a new method that meets the decorator's required
arguments is declared inside the EmailSendingWorker class. This
allows to retry the sending process of a particular email inside
the batch if the caught exception leaves this process retriable.

A second retry mechanism is used inside the initialize_connection
function to redo the opening of the connection until it works or
until three attempts failed. For this purpose the backoff module
has been added to the dependencies and a test has been added to
ensure that this retry mechanism works well.

The connection is closed when the stop method is called.

Fixes: #17672.
2021-04-26 17:27:22 -07:00
Mateusz Mandera b7b1ec0aeb outgoing_webhook: Improve invalid json handling when parsing response.
It's better to just raise JsonableError here, as that makes this error
processed in the central place for this kind of thing in do_rest_call:
---------
except JsonableError as e:
    response_message = e.msg
    logging.info("Outhook trigger failed:", stack_info=True)
    fail_with_message(event, response_message)
    response_message = f"The outgoing webhook server attempted to send a message in Zulip, but that request resulted in the following error:\n> {e}"
    notify_bot_owner(event, failure_message=response_message)
    return None
----------

which does all the things that are supposed to happen -
fail_with_message, appropriate logging and notifying the bot owner.
2021-04-26 09:32:35 -07:00
Mateusz Mandera f1a3f6056a outgoing_webhook: Fix invalid ResponseMock(200) mocks in tests.
These aren't good mocks of a good reponse - a good response is supposed
to contain valid json that doesn't trigger error-handling in the
codepath. Without this change, all these actually trip up on
json.loads(response.text) in process_success_response.
2021-04-26 09:32:35 -07:00
Mateusz Mandera b998138d3a outgoing_webhook: Handle valid, but unexpected json in response.
Responses such as "null" or "true" are valid json, but json.loads
returns different objects than dicts that the codepath expects.

Fixes #18223.
2021-04-26 09:32:35 -07:00
Anders Kaseorg 6060d0d364 docs: Add missing space to compound verbs “log in”, “set up”, etc.
Noun: backup, checkout, cleanup, login, logout, setup, shutdown, signup,
timeout.

Verb: back up, check out, clean up, log in, log out, set up, shut
down, sign up, time out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg 178736c8eb docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Aman Agrawal ebe822341d message_edit: Don't add content edit to all messages.
Remove content edit keys if present in edit_history_event
when passing to update_messages_for_topic_edit.

Since content edit is only applied to the edited_message,
this shouldn't be part of the rest of the messages for which
topic was edited. This was a bug identified by
editing topic and content of a message at the same time
when more than 1 message is affected.
2021-04-24 13:51:49 -07:00
Anders Kaseorg cef2ecf004 i18n: Pass language name with every response.
With a comment added by tabbott on live-update considerations.

See https://chat.zulip.org/#narrow/stream/9-issues/topic/FormatJS.20error.20in.20standalone.20pages/near/1168413 for context.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-24 13:00:27 -07:00
Riken Shah 0e5f6c21ad refactor: Split dev related code from `auth.py` into `dev_login.py`.
This refactor will help us make our development-related authentication
code to avoid production security impact.
2021-04-23 15:11:44 -07:00
Mateusz Mandera 977a2f7fa0 emails: Rename "missed message email" to "message notification email". 2021-04-21 10:10:54 -07:00
Mateusz Mandera cc11575b21 emails: Rename "New missed messages" email subjects to "New messages". 2021-04-21 10:10:54 -07:00
Mateusz Mandera 716449030d emails: Rename missed message email sender to "Zulip notifcations".
It was decided that this is more appropriate naming. "Missed message"
gives it a bit of a sound like something went wrong.
2021-04-21 10:10:54 -07:00
Anders Kaseorg dd3fa4ac52 templates: Mark all void tags as self-closing.
This reverses the policy that was set, but incompletely enforced, by
commit 951514dd7d.  The self-closing tag
syntax is clearer, more consistent, simpler to parse, compatible with
XML, preferred by Prettier, and (most importantly now) required by
FormatJS.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 09:49:34 -07:00