Commit Graph

13279 Commits

Author SHA1 Message Date
Mateusz Mandera b7b1ec0aeb outgoing_webhook: Improve invalid json handling when parsing response.
It's better to just raise JsonableError here, as that makes this error
processed in the central place for this kind of thing in do_rest_call:
---------
except JsonableError as e:
    response_message = e.msg
    logging.info("Outhook trigger failed:", stack_info=True)
    fail_with_message(event, response_message)
    response_message = f"The outgoing webhook server attempted to send a message in Zulip, but that request resulted in the following error:\n> {e}"
    notify_bot_owner(event, failure_message=response_message)
    return None
----------

which does all the things that are supposed to happen -
fail_with_message, appropriate logging and notifying the bot owner.
2021-04-26 09:32:35 -07:00
Mateusz Mandera f1a3f6056a outgoing_webhook: Fix invalid ResponseMock(200) mocks in tests.
These aren't good mocks of a good reponse - a good response is supposed
to contain valid json that doesn't trigger error-handling in the
codepath. Without this change, all these actually trip up on
json.loads(response.text) in process_success_response.
2021-04-26 09:32:35 -07:00
Mateusz Mandera b998138d3a outgoing_webhook: Handle valid, but unexpected json in response.
Responses such as "null" or "true" are valid json, but json.loads
returns different objects than dicts that the codepath expects.

Fixes #18223.
2021-04-26 09:32:35 -07:00
Anders Kaseorg 6060d0d364 docs: Add missing space to compound verbs “log in”, “set up”, etc.
Noun: backup, checkout, cleanup, login, logout, setup, shutdown, signup,
timeout.

Verb: back up, check out, clean up, log in, log out, set up, shut
down, sign up, time out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg e3f2ffa681 docs: Capitalize “Markdown” consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg 178736c8eb docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Aman Agrawal ebe822341d message_edit: Don't add content edit to all messages.
Remove content edit keys if present in edit_history_event
when passing to update_messages_for_topic_edit.

Since content edit is only applied to the edited_message,
this shouldn't be part of the rest of the messages for which
topic was edited. This was a bug identified by
editing topic and content of a message at the same time
when more than 1 message is affected.
2021-04-24 13:51:49 -07:00
Anders Kaseorg cef2ecf004 i18n: Pass language name with every response.
With a comment added by tabbott on live-update considerations.

See https://chat.zulip.org/#narrow/stream/9-issues/topic/FormatJS.20error.20in.20standalone.20pages/near/1168413 for context.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-24 13:00:27 -07:00
Aman Agrawal 79d748ba7d message_edit: Use update_edit_history to update message history. 2021-04-23 15:12:09 -07:00
Aman Agrawal de50f4ae25 message_edit: Extract update_edit_history. 2021-04-23 15:12:09 -07:00
Aman Agrawal 736fdcda49 update_messages_for_topic_edit: Remame `message` variable. 2021-04-23 15:12:09 -07:00
Riken Shah 0e5f6c21ad refactor: Split dev related code from `auth.py` into `dev_login.py`.
This refactor will help us make our development-related authentication
code to avoid production security impact.
2021-04-23 15:11:44 -07:00
Mateusz Mandera 1a8ad796f8 models: Replace __id syntax with _id where possible.
model__id syntax implies needing a JOIN on the model table to fetch the
id. That's usually redundant, because the first table in the query
simply has a 'model_id' column, so the id can be fetched directly.
Django is actually smart enough to not do those redundant joins, but we
should still avoid this misguided syntax.

The exceptions are ManytoMany fields and queries doing a backward
relationship lookup. If "streams" is a many-to-many relationship, then
streams_id is invalid - streams__id syntax is needed. If "y" is a
foreign fields from X to Y:
class X:
  y = models.ForeignKey(Y)

then object x of class X has the field x.y_id, but y of class Y doesn't
have y.x_id. Thus Y queries need to be done like
Y.objects.filter(x__id__in=some_list)
2021-04-22 14:53:00 -07:00
Alex Vandiver 11177a40da soft_deactivate: Log and continue on failure to catch up a user.
There exists a logic bug (see #18236) which causes duplicate
usermessage rows to be inserted.  Currently, this stops catch-up for
all users.

Catch and record the exception for each affected user, so we at least
make catch-up progress on other users.
2021-04-22 14:38:03 -07:00
Mateusz Mandera 8d4ab69a46 docs: Move the /configure-missed-message-emails help page.
configure-message-notification-emails is the correct name now.
2021-04-21 10:10:54 -07:00
Mateusz Mandera 977a2f7fa0 emails: Rename "missed message email" to "message notification email". 2021-04-21 10:10:54 -07:00
Mateusz Mandera cc11575b21 emails: Rename "New missed messages" email subjects to "New messages". 2021-04-21 10:10:54 -07:00
Mateusz Mandera 716449030d emails: Rename missed message email sender to "Zulip notifcations".
It was decided that this is more appropriate naming. "Missed message"
gives it a bit of a sound like something went wrong.
2021-04-21 10:10:54 -07:00
Anders Kaseorg dd3fa4ac52 templates: Mark all void tags as self-closing.
This reverses the policy that was set, but incompletely enforced, by
commit 951514dd7d.  The self-closing tag
syntax is clearer, more consistent, simpler to parse, compatible with
XML, preferred by Prettier, and (most importantly now) required by
FormatJS.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-21 09:49:34 -07:00
Tim Abbott 6346b9d3eb models: Replace user_profile__is_active queries with is_user_active.
This saves a couple database queries by using the recently added
denormalization for Subscription objects.
2021-04-19 18:30:31 -07:00
Tim Abbott a1cfe25f8d streams: Move can_access_stream_user_ids into streams.py.
This belongs either here or in stream_subscription.py, which arguably
should just be merged into streams.py anyway.
2021-04-19 18:30:31 -07:00
Tim Abbott e4932bd952 actions: Fixed deactivated user IDs being included for stream events.
This was a mostly harmless bug, since those users cannot have active
clients, but fixing it will improve performance in any Zulip
organization where the vast majority of users are deactivated.
2021-04-19 18:30:31 -07:00
akshatdalton 6509c4f8f4 linkifiers: Add an API to support the editing of linkifier.
This commit adds an API to `zproject/urls.py` to edit/update
the realm linkifier. Its helper function to update the
database is added in `zerver/lib/actions.py`.

`zulip.yaml` is documented accordingly as well, clearly
stating that this API updates one linkifier at a time.

The tests are added for the API and helper function which
updates the realm linkifier.

Fixes #10830.
2021-04-19 18:01:45 -07:00
akshatdalton c180cd5fa1 Refactor: Use backend-endpoint function instead of helper function.
Use backend-endpoint function instead of helper function in
`test_realm_linkifiers.py` so that tests are more end-to-end.

The removed helper function: `do_add_linkifier` is tested in
`zerver/tests/test_events.py`.
2021-04-19 18:01:45 -07:00
akshatdalton b29bd71a9c Refactor: Use `id` instead of `pk` as key.
Use `id` instead of `pk` as key to get RealmFilter
object in `do_remove_linkifier` function in `actions.py`.
2021-04-19 18:01:45 -07:00
akshatdalton 5bbfa24d74 Refactor: Rename `Filter` to `Linkifier`.
Linkifier error message: `Filter not found` is
updated to `Linkifier not found.`.

Similarly, `filter_id` description is updated to:
`The ID of the linkifier that you want to remove.`,
renamed the term `filter` with `linkifier`, in `zulip.yaml`.
2021-04-19 18:01:45 -07:00
Aman Agrawal 70d444a8eb compose: Move to `hbs` file and append it to DOM on init.
We move compose.html to compose.hbs file while keeping
`#compose` still in `home.html` as a hanger
where append rest of the elements.

This will provide us with two benefits:
* We could share common elements between message_edit_form and
compose.
* We can insert compose directly in any element. We may decide to
do it for recent topics.
2021-04-19 16:53:30 -07:00
Tim Abbott 3810eaaa25 api docs: Document that servers ignore unexpected event types.
Clients like the mobile and terminal apps will want to rely on this
detail of our API, so we should write it down.
2021-04-19 14:14:14 -07:00
Sumanth V Rao f9b79999ed playgrounds: Avoid json encoding each field in POST /realm/playgrounds.
This prevents us from having to json encode every field in the POST
request to /realm/playgrounds, and keeps the client logic simpler
when adding a playground.
2021-04-19 11:23:09 -07:00
Riken Shah 1d9417f9a1 refactor: Rename `dev_panel.html` for more clarity.
This commit renames `dev_panel.html` to `integrations_dev_panel.html`.
2021-04-19 10:25:05 -07:00
Riken Shah bd9e6ae97e refactor: Move dev-only templates to `templates/zerver/development`.
This a prep commit for adding a check to lint to ensure
only development-related templates entrypoint name
starts with `dev`.
2021-04-19 10:25:05 -07:00
Mateusz Mandera ccfcc186ad subs: Fix subscriber_..._history_access to not exclude subbed guests.
Guests are supposed to have stream history access to public streams
they're subscribed to.
2021-04-19 10:10:51 -07:00
Mateusz Mandera 68d1f2d7ef streams: Add realm check in can_access_stream_history.
The caller is supposed validate the stream and user realm match, but
since this is a security-sensitive function, we should have this
defensive code to protect against some validation bugs in the caller
leading to this being called incorrectly and returning True.
2021-04-19 10:10:51 -07:00
Mateusz Mandera f5c4005f8a actions: Fix some lists incorrectly named "subscribers".
These contain subscriptions, not subscribers.
2021-04-19 10:10:51 -07:00
Mateusz Mandera 4e26a9e9d6 subs: Fix codepaths incorrectly fetching subs of deactivated users.
Fixes #17922.

These two places fetch subscriptions for the sake of getting user ids to
send events to. Clearly deactivated users should be excluded from that.
2021-04-19 10:10:51 -07:00
Mateusz Mandera 50bfbb588e subs: Allow filtering by is_user_active in get_active_subscriptions.
get_active_subscriptions_for_stream_id should allow specifying whether
subscriptions of deactivated users should be included in the result.
Active subs of deactivated users are  a subtlety that's easy to miss
when writing relevant code, so we make include_deactivated_users a
mandatory kwarg - this will force callers to definitely give thought to
whether such subs should be included or not.

This commit is just a refactoring, we keep original behavior everywhere
- there are places where subs of deactivates users should probably be
excluded but aren't - we don't fix that here, it'll be addressed in
follow-up commits.
2021-04-19 10:10:51 -07:00
sahil839 4ac3fabadd models: Add new helper can_move_messages_between_streams.
This commit adds new helper can_move_messages_between_streams
which will be used to check whether a user is allowed to move
messages from one stream to another according to value of
'move_messages_between_streams_policy'.
2021-04-16 15:16:08 -07:00
sahil839 2dc99aa90f settings: Add new setting for controlling who can move msgs to stream.
This commit adds a new setting 'move_messages_between_streams_policy`
for controlling who can move messages from one stream to other.
2021-04-16 15:10:39 -07:00
Vishnu KS e72dea1de6 models: Create get_first_human_user function in Realm. 2021-04-16 13:22:01 -07:00
Mateusz Mandera 2983a7e799 test_home: Fix user_activity queue event format in a test.
Current production code uses client_id in the event dict and this test
should be updated to reflect that. Old format event can still be
consumed by the worker, but that is already tested by
WorkerTest.test_UserActivityWorker.
2021-04-16 11:02:48 -07:00
Tim Abbott 48d99886fd event_queue: Update rule for when we can remove compatibility code.
Since c3a8a15bae removed the last
instance of code using the dictionary code path, we actually need to
wait until one can no longer upgrade directly from 4.x to master in
order to avoid breakage should we remove this compatibility code,
since only today did we stop generating the old event format.
2021-04-16 09:58:21 -07:00
Tim Abbott 260861426c queue_processors: Document when can remove compatibility code. 2021-04-16 09:55:14 -07:00
Mateusz Mandera c3a8a15bae delete_messages: Pass a list of user ids in the event in all cases.
The bulk deletion codepath was using dicts instead of user ids in the
event, as opposed to the other codepath which was adjusted to pass just
user ids before. We make the bulk codepath consistent with the other
one. Due to the dict-type events happening in 3.*, we move the goal for
deleting the compat code in process_notification to 5.0.
2021-04-16 09:54:14 -07:00
Anders Kaseorg f59f2ca165 requirements: Re-drop direct dependency on mock.
This was dropped in commit 840cf4b885
(#15091), but commit 1432067959
(#17047) mistakenly reintroduced it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 21:47:33 -07:00
Anders Kaseorg bdb20a8002 integrations: Convert deprecated Django url to path.
django.conf.urls.url is actually a deprecated alias of
django.urls.re_path, but we want path instead of re_path.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Anders Kaseorg 2939d29b6d python: Convert deprecated Django smart_text alias to smart_str.
django.utils.encoding.smart_text is a deprecated alias of
django.utils.encoding.smart_str as of Django 3.0, and will be removed
in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Anders Kaseorg dcdb00a5e6 python: Convert deprecated Django is_safe_url.
django.utils.http.is_safe_url is a deprecated alias of
django.utils.http.url_has_allowed_host_and_scheme as of Django 3.0,
and will be removed in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Anders Kaseorg e7ed907cf6 python: Convert deprecated Django ugettext alias to gettext.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Adam Birds 545cd961f4 integrations: Add docs for GitHub Actions integration.
I have added a documentation page for the GitHub Actions integration to
`/integrations/doc/github-actions` with a link to the Zulip GitHub
Actions repository.

Tweaked by tabbott to add cross-links with the main GitHub integration.
2021-04-15 16:42:31 -07:00
Tim Abbott 208721b3d7 left sidebar: Fix misleading "search streams" label.
This widget only filters the user's subscription -- it's only suggest
public streams that the user is not subscribed to.  "Filter" is the
correct label for a widget with this use case.
2021-04-15 15:36:05 -07:00