Commit Graph

1566 Commits

Author SHA1 Message Date
Anders Kaseorg a610bd19a1 python: Simplify away various unnecessary lists and list comprehensions.
Loosely inspired by the flake8-comprehensions plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg ab120a03bc python: Replace unnecessary intermediate lists with generators.
Mostly suggested by the flake8-comprehension plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Mateusz Mandera 9b50c49ea7 streams: Mark all messages as read when deactivating a stream.
The query to finds and marks all unread UserMessages in the stream as read
can be quite expensive, so we'll move that work to the deferred_work
queue and split it into batches.

Fixes #15770.
2020-09-01 11:24:27 -07:00
Alex Vandiver 81893c9dbb actions: Invalid flag operation is a user error. 2020-08-29 11:38:59 -04:00
orientor 372e010dbb events: Add `op` field to `update_message_flags` events.
`update_message_flags` events used `operation` instead of `op`, the
latter being the standard field used in other events. So add `op`
field to `update_message_flags` and mark `operation` as deprecated,
so that it can be removed later.
2020-08-24 12:42:03 -07:00
Clara Dantas 05bf72a75c attachments: Add is_web_public field.
This commit adds the is_web_public field in the AbstractAttachment
class. This is useful when validating user access to the attachment,
as otherwise we would have to make a query in the db to check if
that attachment was sent in a message in a web-public stream or not.
2020-08-12 17:26:03 -07:00
Alex Vandiver 153f16ee6a links: Flatten the set into a list before serializing into the queue.
orjson does not transparently do this set-to-list translation, unlike
ujson.
2020-08-12 11:42:24 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Anders Kaseorg 768f9f93cd docs: Capitalize Markdown consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Anders Kaseorg 60a25b2721 docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00
Alex Vandiver 2928bbc8bd logging: Report stack_info on logging.exception calls.
The exception trace only goes from where the exception was thrown up
to where the `logging.exception` call is; any context as to where
_that_ was called from is lost, unless `stack_info` is passed as well.
Having the stack is particularly useful for Sentry exceptions, which
gain the full stack trace.

Add `stack_info=True` on all `logging.exception` calls with a
non-trivial stack; we omit `wsgi.py`.  Adjusts tests to match.
2020-08-11 10:16:54 -07:00
Alex Vandiver 39368cad3a tornado: Extract functions called from django into one module.
This makes clearer the separation of concerns.
2020-08-10 16:55:56 -07:00
Tim Abbott e12225ace7 realm_audit_log: Fix malformed RealmAuditLog entries.
In f8bcf39014, we fixed buggy
marshalling of Streams and similar data structures where we were
including the Stream object rather than its ID in dictionaries passed
to ujson, and ujson happily wrote that large object dump into the
RealmAuditLog.extra_data field.

This commit includes a migration to fix those corrupted RealmAuditLog
entries, and because the migration loop is the same, also fixes the
format of similar RealmAuditLog entries to be in a more natural format
that doesn't weirdly nest and duplicate the "property" field.

Fixes #16066.
2020-08-09 16:51:29 -07:00
Anders Kaseorg b250e42f61 realm_user_count_by_role: Use strings as dict keys.
JSON keys must be strings, and orjson enforces this.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Anders Kaseorg f8bcf39014 actions: Do not attempt JSON serialization of a Stream object.
It doesn’t end well.  Or sometimes it doesn’t end (OverflowError:
Maximum recursion level reached).

Introduced by commits ccdf52fef6 and
94d2de8b4a (#15601).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 10:53:06 -07:00
Tim Abbott 00fd9afad5 embed: Remove useless 'sender' field.
The variant `update_message` events have this extra sender field not
present in normal update_message events; this field has no purpose, so
we remove it.
2020-08-03 18:04:38 -07:00
Clara Dantas 6b011b241b actions: Remove duplicated conditional.
In the gather_subscriptions_helper functions, there was a repeated
conditional inside one another.
2020-07-30 16:59:51 -07:00
Clara Dantas a9af80d7a2 streams: Make /streams endpoint return also web-public streams.
This commit modifies the /streams endpoint so that the web-public
streams are included in the default list of streams that users
have access to.

This is part of PR #14638 that aims to allow guest users to
browse and subscribe themselves to web public streams.
2020-07-29 17:52:36 -07:00
Clara Dantas 523bb30f33 actions: Modify validate_user_access_to_subscribers.
This modification allows guest users to have access to web-public
streams subscribers, even if they aren't subscribed or never
subscribed to that stream.

This commit is part of PR #14638 that aims to allow guest users to
browser and subscribe to web-public streams.
2020-07-29 17:52:36 -07:00
Clara Dantas 6a1d9480bc actions: Modify gather_subscriptions() to also gather web-public streams.
Now, gather_subscriptions include web-public streams in the 3 sets
of streams that it returns, subscribed, unsubscribed and never
subscribed.

This is part of PR #14638 that aims to allow guest users to browse and
subscribe to web-public streams.
2020-07-29 17:52:36 -07:00
Sumanth V Rao 9b6de63afe stream/docs: Add date_created to Stream.API_FIELDS.
The parameter Stream.date_created is now sent down to the clients
for both:

    - client.get_streams()
    - client.list_subscriptions()

API docs updated for stream and subscriptions.

Fixes #15410
2020-07-27 16:33:36 -07:00
Tim Abbott 29c66cf7c2 actions: Remove log_event and its legacy settings.
Now that we've finally converted these to use RealmAuditLog, we can
remove this ultra-legacy bit of code.
2020-07-24 12:13:16 -07:00
arpit551 0d79b55b2e audit_log: Record RealmAuditLog in do_change_notification_settings.
Removed logging with log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:04:40 -07:00
arpit551 54df9290b9 audit_log: Record RealmAuditLog in do_rename_stream.
Removed logging with  log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:00:31 -07:00
arpit551 94d2de8b4a audit_log: Record RealmAuditLog while changing default streams.
Removed logging with log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:00:31 -07:00
arpit551 753713fde7 audit_log: Record RealmAuditLog in do_change_subscription_property.
Removed logging with log_subscription_property_change
and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:00:29 -07:00
arpit551 e6f54a3278 audit_log: Record RealmAuditLog in do_change_icon_source.
Removed logging with  log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 11:56:40 -07:00
arpit551 12a19b0ff1 actions.py: Removed unnecessary logging in notify_subscription_added/removed.
We already record RealmAuditLog in bulk_add/remove_subscription
so there is no need to log while notifying.
2020-07-24 11:51:47 -07:00
arpit551 ccdf52fef6 audit_log: Log RealmAuditLog for realm notification properties.
Log RealmAuditLog for do_set_realm_notifications_stream and
do_set_realm_signup_notifications_stream function.
Added tests for the same.
2020-07-24 11:51:47 -07:00
arpit551 8f6a1c3f40 actions.py: Removed do_set_realm_message_deleting function.
Used do_set_realm_property function instead of
do_set_realm_message_deleting and removed it.
2020-07-24 11:51:47 -07:00
arpit551 422fa0ff7d audit_log: Log RealmAuditLog for do_set_realm_message_editing.
Log RealmAuditLog for do_set_realm_message_editing. Added tests for
same.
2020-07-24 11:51:46 -07:00
Steve Howell c44500175d database: Remove short_name from UserProfile.
A few major themes here:

    - We remove short_name from UserProfile
      and add the appropriate migration.

    - We remove short_name from various
      cache-related lists of fields.

    - We allow import tools to continue to
      write short_name to their export files,
      and then we simply ignore the field
      at import time.

    - We change functions like do_create_user,
      create_user_profile, etc.

    - We keep short_name in the /json/bots
      API.  (It actually gets turned into
      an email.)

    - We don't modify our LDAP code much
      here.
2020-07-17 11:15:15 -07:00
Steve Howell 2a9cb4c8cf dead code: Remove legacy reaction actions. 2020-07-17 04:40:01 -04:00
Aman Agrawal f1cc2ab926 delete_event_notify_user_ids: Fix UnboundLocalError. 2020-07-07 10:33:50 -07:00
arpit551 a2428017e9 audit_log: Log RealmAuditLog in do_set_realm_authentication_methods.
Log RealmAuditLog in do_set_realm_authentication_methods and added
tests for it.
2020-07-06 17:38:20 -07:00
arpit551 ba92666dbd audit_log: Log RealmAuditLog in do_set_realm_property.
Log RealmAuditLog in do_set_realm_property and do_remove_realm_domain.
Tests for the changes are written in test_events because it will save
duplicate code for test_change_realm_property.
2020-07-06 17:38:19 -07:00
arpit551 6af337c07e audit_log: Log RealmAuditLog when stream is deactivated. 2020-07-06 17:32:11 -07:00
arpit551 e0c9b0fdff audit_log: Log RealmAuditLog when Stream is created.
Added new Event Type in AbstractRealmAuditLog STREAM_CREATED.
Since we finally create streams in create_stream_if_needed function
in zerver/lib/streams.py so logged realm_audit there.
Passed acting_user when create_stream_if_needed or ensure_stream
function  is called.
Added tests in test_audit_log.
2020-07-06 17:32:11 -07:00
arpit551 8807cddc90 audit_log: Correctly add acting_user in bulk_remove_subscriptions.
Earlier we were not adding acting_user while creating RealmAuditLog
objects in bulk_remove_subscriptions. Improved tests_subscriptions
to check it.
2020-07-06 17:32:11 -07:00
arpit551 87aaa84b42 audit_log: Log acting_user in do_change_user_role. 2020-07-06 17:32:11 -07:00
arpit551 01f12b9fc2 audit_log: Log acting_user in user creation and user activation. 2020-07-06 17:32:09 -07:00
arpit551 653928bdfe audit_log: Log acting_user in do_change_avatar_fields. 2020-07-06 17:24:18 -07:00
arpit551 2279fef316 audit_log: Log acting_user in do_change_logo_source. 2020-07-06 17:24:18 -07:00
arpit551 19a8841a9e audit_log: Log acting_user in do_scrub_realm. 2020-07-06 17:24:18 -07:00
Anders Kaseorg c0bbdbcadf test_messages: Fix strict_optional errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 11:25:48 -07:00
Anders Kaseorg 3885fdadce realm: Fix strict_optional errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 11:25:48 -07:00
Anders Kaseorg c08ee904d8 models: Add explicit id fields for better type checking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-02 13:28:10 -07:00
Tim Abbott 0c48c75920 push_notifications: Fix events with no message_ids.
This fixes our triggering a RabbitMQ event to send a push notification
to remove the empty set of push notifications, resulting from not
using the correct data structure to determine which message IDs to look at.

This was causing a lot of visible exceptions when running
the `test_messages.py` test suite.
2020-07-01 14:58:43 -07:00
Tim Abbott c6638a62dc message_edit: Clean up comments around moving topics. 2020-07-01 14:55:17 -07:00
Tim Abbott 03cf3edf42 message_edit: Fix typo in old_stream_unsubbed_guests. 2020-07-01 14:54:36 -07:00