Commit Graph

17898 Commits

Author SHA1 Message Date
Sahil Batra 0669e4f631 realm: Allow everyone to delete their own messages by default.
This commit updates default for delete_own_message_policy
setting to "Everyone" as it is helpful to allow everyone
to delete their own messages in a new organization where
users might be using Zulip for the first time.
2024-02-28 13:13:46 -08:00
Sahil Batra 4fbc137c2b realm: Allow members to move msgs between streams by default.
This commit updates default for move_messages_between_streams_policy
setting to "Members and above" as it is helpful to allow members
to move messages between streams in new organizations where users
might be using Zulip for first time.
2024-02-28 13:13:46 -08:00
Alex Vandiver a808c730bc deferred_work: Use an id high-water-mark instead of offsets.
This solves the problem listed in the now-removed comment.
2024-02-27 17:02:34 -08:00
Alya Abbott ad8bdfbe20 portico: Consistently describe Zulip as organized team chat. 2024-02-27 17:01:05 -08:00
Alex Vandiver 58f0669997 deferred_work: Re-queue remaining "mark all as read" work after 30s. 2024-02-27 10:21:04 -08:00
Alex Vandiver 75e9903be5 deferred_work: Move all queries into the transaction.
The presence of `len(messages)` outside the transaction caused the
full resultset to be fetched outside of the transaction.  This should
ideally be inside the transaction, and also only need be the count.

However, also note that the process of counting matching rows, and
then executing a second query which embeds the same query, is
susceptible to phantom reads, where a query with the same conditions
returns different resultsets, under PostgreSQL's default transaction
isolation of "read committed."  While this is possible to resolve by
pulling the returned IDs into a Python list, it would not address the
issue that concurrent updates which change the resultset would make
the overall algorithm still incorrect.

Add a comment clarifying the conditions under which the algorithm is
correct.  A more correct algorithm would walk the UserMessage rows
which are unread and in the stream, but this requires a
whole-UserMessage index which would be quite large for such an
infrequent use case.
2024-02-27 10:21:04 -08:00
Alex Vandiver 4712031528 email_validation: Merge two implementations of a disposable-email check.
Only one of them properly checked for improperly-formed email
addresses, thanks to #22723.
2024-02-27 10:19:23 -08:00
Alex Vandiver 4b512b3409 email_validation: Rename to validate_is_not_disposable.
This clarifies what we are asserting -- namely, that the email is
_not_ disposable.
2024-02-27 10:19:23 -08:00
Alex Vandiver c1aadbe52e events: Do nothing in apply_event for restart events.
These signal that the Tornado process restarted, which in itself is
not notable for apply_events.
2024-02-27 10:18:11 -08:00
Alex Vandiver be59ef9c02 runtornado: Swap deferred reload events to the default.
This makes no immediate reloads the default for runtornado, matching
the production configuration, and changes the development incantation
to be the one to specify the departure from the norm, with
--immediate-reloads.
2024-02-26 22:29:19 -08:00
Lauryn Menard cf82d3316b push-bouncer: Exclude LoggingCountStats with partial data.
LoggingCountStats with a daily duration and that are directly stored
on the RealmCount table (not via aggregation in process_count_stat),
can be in a state, after the hourly cron job to update analytics
counts, where the logged value will be live-updated later, because
the end time for the stat is still in the future.

As these logging counts are designed to be used on the self-hosted
installation for either debugging or rate limiting, sending these
partial/incomplete counts to the bouncer has low value.
2024-02-26 17:53:12 -08:00
Lauryn Menard 3a03e14938 integrations: Update Slack webhook to use check_send_webhook_message.
Due to the channel_map_to_topics URL parameter in the Slack webhook,
it was not migrated to use the check_send_webhook_message.

By using check_send_webhook_message, any topic parameter in the
webhook URL will be prioritized over mapping Slack channels to
topics, e.g. when channel_map_to_topics is true. This is because
the default behaviour for incoming webhooks is to send a default
topic as a parameter to check_send_webhook_message in case there
is no topic specified in the URL.

In contrast, we can override the stream passed in the URL when
channel_map_to_topics is false by passing the Slack channel name
to check_send_webhook_message. The default behaviour for incoming
webhooks is to send a direct message if there is no specified
stream in the URL, so a default stream is not generally passed
to check_send_webhook_message.

Fixes #27601.
2024-02-25 16:47:34 -08:00
Alex Vandiver 8e7ab57bf2 export_search: Support exporting DMs or huddles. 2024-02-25 09:34:17 -08:00
Prakhar Pratyush fe1a20ebb3 settings: Add realm-level setting 'zulip_update_announcements_stream'.
This commit adds a realm-level setting named
'zulip_update_announcements_stream' that configures the
stream to which zulip updates should be posted.

Fixes part of #28604.
2024-02-25 09:33:00 -08:00
Mateusz Mandera d7cf4336e4 zilencer: Flip realm_locally_deleted when realm re-appears during sync. 2024-02-24 08:52:11 -08:00
David Rosa 5215e3963c help: Clean up beta app installation instructions.
Converts the beta app installation instructions into our standard
tabbed instructions format.

Fixes #28885.
2024-02-24 08:12:34 -08:00
David Rosa 948b427de7 api-docs: Add instructions to download `zuliprc` files.
- Adds instructions for downloading a zuliprc file for a bot or for
  yourself.
- Updates the button label to "Download zuliprc", since that's the
  filename it downloads.

Fixes #28881.
2024-02-23 09:43:57 -08:00
Tim Abbott d0c276d863 corporate: Fix billing_session variable reuse confusion.
The previous logic incorrectly used the server-level number of users
even when a (presumably smaller) realm-level count was available.

Fixes a bug introduced in 2e1ed4431a.
2024-02-21 17:51:30 -08:00
Tim Abbott 6ee123be21 migrations: Renumber misnumbered migrations. 2024-02-21 09:19:34 -08:00
Prakhar Pratyush ee612dafac settings: Rename signup_notifications_stream realm setting.
This commit renames the realm-level setting
'signup_notifications_stream' to 'signup_announcements_stream'.

The new name reflects better what the setting does.
2024-02-21 09:04:23 -08:00
Prakhar Pratyush ab453fbe20 settings: Rename notifications_stream to new_stream_announce..._stream.
This commit renames the realm-level setting 'notifications_stream'
to 'new_stream_announcements_stream'.

The new name reflects better what the setting does.
2024-02-21 09:04:23 -08:00
Prakhar Pratyush 0c83bca81e events: Use 'get_notifications_stream' realm method.
This commit updates the code in 'fetch_initial_state_data'
function to use the 'get_notifications_stream' method
instead of re-writing the logic.
2024-02-21 09:04:23 -08:00
Alex Vandiver 7e9d8cd192 tornado: Fix GC of old queues. 2024-02-20 22:07:25 -08:00
Alex Vandiver b747ea285f topic: Fix history order for topic moves.
5c96f94206 mistakenly appended, rather than prepended, the edit to
the history.  This caused AssertionErrors when attempting to view the
history of moved messages, which check that the `last_edit_time`
matches the timestamp of the first edit in the list.

Fix the ordering, and update the `edit_history` for messages that were
affected.  We limit to only messages edited since the commit was
merged, since that helps bound the affected messages somewhat.
2024-02-20 21:30:32 -08:00
Mateusz Mandera be03dabf76 zilencer: Implement do_reactivate_remote_server utility function.
The inverse of do_deactivate_remote_server. It's just flipping the
.deactivated flag, but we also should have an AuditLog for these events.
2024-02-19 20:26:47 -08:00
Tim Abbott 2e1ed4431a corporate: Fix plan precedence issues with expired plans.
RemoteRealm customer takes precedence over RemoteServer
in general. But if an inactive plan is associated with
RemoteRealm and an active plan with RemoteServer, the
ACTIVE plan takes precendence.

Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
2024-02-19 17:58:49 -08:00
Anders Kaseorg e568064f4e page_params: Set and expect state_data = null for spectators.
Fix a ZodError when spectators load the app, introduced by commit
a4938d3760 (#28971).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-19 03:03:58 -05:00
Anders Kaseorg a4938d3760 page_params: Parse page_params and state_data with Zod.
This establishes a runtime check that their types continue to reflect
reality going forward.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-17 00:02:38 -08:00
Alya Abbott 4f1659fe8f billing: Make push notifications error message more clear. 2024-02-16 16:01:53 -08:00
sayyedarib 5c993f0479 direct_message: Allow DMs to bots and self regardless of policy.
Previously, in DM disabled org messaging to bot was not working when
starting new conversation and adding bot as recipient because of not
updating on recipient change. And secondly, self messaging was not
allowed.
This commit ensures that the DM to bot and self are allowed irrespective
of dm restrictions.

tests: Verify DMs adhere to DM restriction policy.

Fixes #28412

Signed-off-by: sayyedarib <sayyedaribhussain4321@gmail.com>
2024-02-15 16:13:36 -08:00
Alex Vandiver 0079688c49 tornado: Drop WebReloadClientError logic.
The widening of the time between when a process is marked for
reload (at Tornado startup) and when it sends reload events makes it
unlikely-to-impossible that a single `/` request will span both of
them, and thus hit the WebReloadClientError corner case.

Remove it, as it is not worth the complication.  The bad behaviour it
is attempting to prevent (of a reload right after opening `/`) was
always still possible  -- if the `/` request completed right before
Tornado restarted -- so it is not clear that it was ever worth the
complication.
2024-02-15 15:42:50 -08:00
Alex Vandiver da6b0b1cc6 tornado: Add a web_reload_clients endpoint to notify web clients. 2024-02-15 15:42:50 -08:00
Alex Vandiver b25d966352 tornado: Add a --no-immediate-reloads flag to leave web clients alone. 2024-02-15 15:42:50 -08:00
Alex Vandiver 1d3813ec4f tornado: Track which queues were inherited from old Tornado instances. 2024-02-15 15:42:50 -08:00
Alex Vandiver fc41d6085b tornado: Split server restart events from web client reload events. 2024-02-15 15:42:50 -08:00
Alex Vandiver a6287faea4 tornado: Stop collapsing "restart" events via virtual events.
Collapsing was done incorrectly, as 65c400e06d added `zulip_version`
and `zulip_feature_level`, but did not update the virtual event logic
to copy those new values into the virtual event.

However, it is unlikely that a server will be upgraded multiple times
in quick enough succession for this to ever be relevant.  Remove the
logic, which is additional complication for little or no gain.
2024-02-15 15:42:50 -08:00
Alex Vandiver 52a80e1d78 tornado: Switch to time.perf_counter for timing data. 2024-02-15 15:42:50 -08:00
Alex Vandiver ab683fac29 decorator: Rename internal_notify_view.
As noted in the previous commit, this decorator is not just used for
"notify" endpoints anymore.
2024-02-15 15:42:50 -08:00
Alex Vandiver be75736a95 decorator: Rename authenticate_notify.
This was originally used for more Django -> Tornado communication than
it now does; it is primarily not used to "notify" Tornado anymore.
2024-02-15 15:42:50 -08:00
Alex Vandiver e6fa1f5436 email_mirror_message: Move endpoint under /api/internal/.
This also has the side effect that it saves the endpint from being
improperly redirected by the HostDomainMiddleware's canonicalization
code.
2024-02-15 15:42:50 -08:00
Alex Vandiver 84fa9be73a tornado: Move the /notify-tornado endpoint, and document it better. 2024-02-15 15:42:50 -08:00
Alya Abbott 8bebabab8f api docs: Clarify Zulip Cloud requirements for user creation endpoint. 2024-02-15 15:08:53 -08:00
Anders Kaseorg 823e8d8716 state_data: Fix custom_profile_fields type for CHOICE → SELECT rename.
Commit bd6471f0e3 (#28691) added this
reference to the old name, even though it had already been renamed in
commit b220d29fed (#17775), presumably
because that had failed to update the OpenAPI description.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-15 16:00:25 -05:00
Anders Kaseorg bf6978185d home: Send state_data in a separate object.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-15 10:22:52 -08:00
Alex Vandiver 23baabba86 message: Merge unnecessary cache_transformer step.
Having a non-identity `cache_transformer` is no different from running
it on every row of the query_function.  Simplify understanding of the
codepath used in caching by merging the pieces of code.
2024-02-14 12:27:03 -08:00
Alex Vandiver 6597c560cb message_cache: Rename "to_dict" functions which deal with bytes. 2024-02-14 12:27:03 -08:00
Alex Vandiver b3e8878252 message_cache: Rename update_to_dict_cache to update_message_cache.
This better describes what it does.
2024-02-14 12:27:03 -08:00
Alex Vandiver 737a751f5c message: Split MessageDict and friends into its own file. 2024-02-14 12:27:03 -08:00
Alex Vandiver 0f9b7f112b message: Move render_markdown into zerver.lib.markdown. 2024-02-14 12:27:03 -08:00
Alex Vandiver 22837fc1b4 message_edit: Carry the QuerySet through as much as possible.
Rather than pass around a list of message objects in-memory, we
instead keep the same constructed QuerySet which includes the later
propagated messages (if any), and use that same query to pick out
affected Attachment objects, rather than limiting to the set of ids.
This is not necessarily a win -- the list of message-ids *may* be very
long, and thus the query may be more concise, easier to send to
PostgreSQL, and faster for PostgreSQL to parse.  However, the list of
ids is almost certainly better-indexed.

After processing the move, the QuerySet must be re-defined as a search
of ids (and possibly a very long list of such), since there is no
other way which is guaranteed to correctly single out the moved
messages.  At this point, it is mostly equivalent to the list of
Message objects, and certainly takes no less memory.
2024-02-14 12:27:03 -08:00