Commit Graph

17753 Commits

Author SHA1 Message Date
Tim Abbott fbd8ed1425 import_realm: Move push bouncer ping after realm reactivation. 2023-12-04 07:08:39 -08:00
Tim Abbott f6c7eaf1e5 models: Add push_notifications_enabled & corresponding end_timestamp.
Add two fields to Realm model:
*push_notifications_enabled
*push_notifications_enabled_end_timestamp

Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
2023-12-04 07:08:39 -08:00
Prakhar Pratyush 6aa911a9b2 remote_server_post_analytics: Return remote realms data in response.
This is a prep commit to return, for each remote realm, the 'uuid',
'can_push', and 'expected_end_timestamp'.

This data will be used in 'initialize_push_notifications'.
2023-12-04 07:08:39 -08:00
Lauryn Menard 0679bc044a corporate: Make references to billing modality consistent.
The "send_invoice" and "charge_automatically" strings used by stripe
for the `collection_method` are referred to both as the "billing
method" and "billing modality" in the billing code.

Because we send this as data to stripe as either `collection_method`
or `billing_modality`, renames any references that are any form of
"billing method".
2023-12-03 14:27:07 -08:00
Mateusz Mandera ec7245d4e1 remote_billing: Add redirect flow for users with expired session.
Implements a nice redirect flow to give a good UX for users attempting
to access a remote billing page with an expired RemoteRealm session e.g.
/realm/some-uuid/sponsorship - perhaps through their browser
history or just their session expired while they were doing things in
this billing system.

The logic has a few pieces:
1. get_remote_realm_from_session, if the user doesn't have a
   identity_dict will raise RemoteBillingAuthenticationError.
2. If the user has an identity_dict, but it's expired, then
   get_identity_dict_from_session inside of get_remote_realm_from_session
   will raise RemoteBillingIdentityExpiredError.
3. The decorator authenticated_remote_realm_management_endpoint
   catches that exception and uses some general logic, described in more
   detail in the comments in the code, to figure out the right URL to
   redirect them to. Something like:
   https://theirserver.example.com/self-hosted-billing/?next_page=...
   where the next_page param is determined based on parsing request.path
   to see what kind of endpoint they're trying to access.
4. The remote_server_billing_entry endpoint is tweaked to also send
   its uri scheme to the bouncer, so that the bouncer can know whether
   to do the redirect on http or https.
2023-12-03 10:39:56 -08:00
Mateusz Mandera 7f33d6f0ea zilencer: Tie RemotePushDeviceToken to RemoteRealm at registration.
This consists of the following pieces:
1. Makes servers using the bouncer send realm_uuid in requests for token
   registration. (Sidenote: realm_uuid is already sent in the "send
   notification" codepath as of
   48db4bf854)
2. This allows the bouncer to tie RemotePushDeviceToken to the
   RemoteRealm with matching realm_uuid at registration time.
3. Introduce handling of some potential weird edge cases around the
   realm_uuid and RemoteRealm objects in get_remote_realm_helper.
2023-12-03 09:51:45 -08:00
Mateusz Mandera c9b0602320 tests: Create default RemoteRealms in populate_db.
This default setup will be more realistic, matching the ordinary
conditions for a modern server.
Especially needed as we add bouncer code that will expect to have
RemoteRealm entries for realm_uuid values for which it receives
requests.
2023-12-03 09:51:45 -08:00
Mateusz Mandera a67dd6dc1f realms: Call send_realms_only_to_push_bouncer at realm creation/import. 2023-12-03 08:49:58 -08:00
Anders Kaseorg 1efc6efd23 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-01 18:52:00 -08:00
Alya Abbott 6f796e0cbd plans_page: Adjust /plans content.
[squash]: Update sponsorsip and question boxes for Cloud.

[squash]: Update tabs subtitles.

[squash]: Content for info boxes for self-hosted plans.

[squash]: Adjust content to fit design.

portico: Tweak /plans text.
2023-12-01 15:12:31 -08:00
Prakhar Pratyush b32950d790 notifications: Revert API changes for push_notifications_enabled.
This commit reverts the API changes in 56ec1c2.
2023-12-01 08:14:14 -08:00
Alex Vandiver 569c364392 users: Fetch and lock the user row before updating its role.
We want to both (a) take a lock on the UserProfile row, and (b)
modify the passed-in UserProfile object, so that callers see the
changes in the object they hold.  Unfortunately,
`select_for_update` cannot be combined with `refresh_from_db`
(https://code.djangoproject.com/ticket/28344).  Call
`select_for_update` and throw away the result, so that we know we have
the lock on the row, then re-fill the `user_profile` object with the
values now that the lock exists.
2023-11-30 16:15:23 -08:00
Alex Vandiver 7f96bed17b stream_traffic: Use the realm_id to get a much better-indexed query.
This reduces the query time by an order of magnitude, since it is able
to switch from a raw `stream_id` index to an index over all of
`realm_id, property, end_time`.
2023-11-30 12:32:30 -08:00
Aman Agrawal 1df8e00d7c remote_billing: Redirect to upgrade/sponsorship page based on next.
We pass `next` parameter with /self-hosted-billing to redirect
users to the intended page after login.

Fixed realm_uuid incorrectly required in remote_realm_upgrade_page.
2023-11-30 11:22:19 -08:00
Alya Abbott 7b71ea3314 help: Add tab for Zulip 8.0+ features. 2023-11-29 23:44:00 -08:00
Tim Abbott 610338d192 billing: Add BILLING_SCHEDULE_ prefix to values. 2023-11-29 23:32:56 -08:00
Tim Abbott 5d6b635efe billing: Use better variable names for plan tiers.
The existing values didn't have our standard type-prefixing naming
scheme.

Add some extra unused placeholder values while we're at it.
2023-11-29 23:32:56 -08:00
David Rosa af3956e1a9 help: Document upgrade flow, billing, and sponsorship requests.
Documents how to upgrade to a paid plan, manage billing, and
apply for sponsorship.

Fixes #27946.
2023-11-29 21:50:44 -08:00
Mateusz Mandera 63618e93f0 zilencer: Sync new RemoteRealm fields also for existing registrations. 2023-11-29 15:54:38 -08:00
Mateusz Mandera 9b1a495e2c zilencer: Sync name and authentication_methods on RemoteRealm. 2023-11-29 15:54:38 -08:00
Tim Abbott 4ef6b7cc44 notifications: Rename apns_enabled and gcm_enabled.
The new names are a lot clearer when thinking about self-hosted
systems that might indirectly use these via a bouncer.
2023-11-29 15:06:46 -08:00
Tim Abbott b71c5746ab notifications: Rename push_notifications_enabled for clarity.
This doesn't actually check if push notifications are working, just
whether there is configuration for them.
2023-11-29 15:06:46 -08:00
Tim Abbott 7db15176f3 push bouncer: Submit basic metadata unconditionally.
These metadata are essentially all publicily available anyway, and
making uploading them unconditional will simplify some things.

The documentation is not quite accurate in that it claims the server
will upload some metadata that is not actually uploaded yet (but will
by soon). This seems harmless.
2023-11-29 14:45:53 -08:00
Prakhar Pratyush d8cf12eaaa send_email: Improve configurability for outgoing email sender name.
Currently, the sender names for outgoing emails sent by Zulip
are hardcoded. It should be configurable for self-hosted systems.

This commit makes the 'Zulip' part a variable in the following
email sender names: 'Zulip Account Security', 'Zulip Digest',
and 'Zulip Notifications' by introducing a settings variable
'SERVICE_NAME' with the default value as f"{EXTERNAL_HOST} Zulip".

Fixes: #23857
2023-11-29 14:20:01 -08:00
Prakhar Pratyush 78a75ab28c digest: Fix the `_` used for an unused variable.
This prep commit replaces `_` with `ignored` to represent
an unused variable.

In later commits, we are going to use `_` for translation,
which leads to a lint error.
2023-11-29 14:20:01 -08:00
Sahil Batra a6fa875c23 models: Refactor get_fake_email_domain to take realm.host as arg.
This commit updates get_fake_email_domain to accept realm.host as
argument instead of the Realm object since we only use realm.host
to get the fake email domain.

This is a preparatory commit for the limited guest feature as we
would be sending the fake email of the message sender in message
event object to a guest user who cannot access the sender and
there we would need to compute the fake email.
2023-11-29 12:01:37 -08:00
Alex Vandiver 3eb5b20c41 register_server: Redact the key when printing it. 2023-11-29 09:56:14 -08:00
Alex Vandiver 737de6d4cd user_settings: Re-verify email addresses when enacting them. 2023-11-28 22:11:41 -08:00
Alex Vandiver 037eaa07e2 user_settings: Revoke previous email changes on new one. 2023-11-28 22:11:41 -08:00
Alex Vandiver f7990ad175 user_settings: Lock user row before changing email address. 2023-11-28 22:11:41 -08:00
Alex Vandiver 6ecfbd2ae8 create_realm: Fix copy/paste error in assertion message. 2023-11-28 18:15:30 -08:00
Alex Vandiver 61fc838179 python: Switch mocking of timezone_now to time_machine. 2023-11-28 15:10:39 -08:00
Alex Vandiver 82c08dd153 python: Prevent bare timedelta(), which defaults to days. 2023-11-28 15:10:39 -08:00
Alex Vandiver 0f132cef4d test_presence: Fix test which moved days, not seconds. 2023-11-28 15:10:39 -08:00
Alex Vandiver 8c8dbb3d66 markdown: Stop attempting to expand/collapse re2 regex.
549dd8a4c4 changed the regex that we build to contain whitespace for
readability, and strip that back out before returning it.
Unfortunately, this also serves to strip out whitespace in the source
linkifier, causing it to not match expected strings.

Revert 549dd8a4c4.

Fixes: #27854.
2023-11-28 15:07:23 -08:00
Mateusz Mandera 02d5740f0f remote_realm: Add syncing of org_type. 2023-11-28 14:41:16 -08:00
Mateusz Mandera e276812e42 models: Create OrgTypeDict for Realm.ORG_TYPES typing. 2023-11-28 14:41:16 -08:00
Lauryn Menard 276ceb46e2 emails: Add organization details to confirm registration for new org.
Adds details about the requested organization URL and type to the
registration confirmation email that's sent when creating a new
Zulip organization.

Fixes #25899.
2023-11-28 09:52:31 -08:00
Alex Vandiver 82960d9bc2 upload: Redirect unauthorized anonymous requests to login.
Note that this also redirects rate-limited anonymous requests to the
login page, as we do not currently differentiate the cases.
2023-11-28 09:44:55 -08:00
Alex Vandiver f9884af114 upload: Return images for 404/403 responses with image Accept: headers.
If the request's `Accept:` header signals a preference for serving
images over text, return an image representing the 404/403 instead of
serving a `text/html` response.

Fixes: #23739.
2023-11-28 09:44:55 -08:00
Alya Abbott fcd0783202 upgrade: Simplify free trial upgrade page. 2023-11-28 08:57:27 -08:00
roanster007 8bd92fe801 bots: Fix muting of cross realm bots.
Previously, we weren't able to mute the cross realm bots. This was
because, for muting the users, we access only those profiles which are
in realm, excluding the cross realm system bots.

This is fixed by replacing the access_user_by_id method with a new
method access_user_by_id_including_cross_realm for this specific test.

Fixes #27823
2023-11-27 16:16:23 -08:00
Alya Abbott 9788e7514f billing: Edit text of free trial upgrade page. 2023-11-27 14:17:18 -08:00
kimry02 adc11e5ba2
email: Add a space after the time and AM/PM in the login email.
This is more typical formatting for times.

Fixes #27727.
2023-11-27 09:47:30 -08:00
Aman Agrawal a96619639c events_register: Set home view for spectators as recent view. 2023-11-27 09:41:52 -08:00
Aman Agrawal 603aa3db01 user_base_settings: Change web_home_view default to inbox view.
It would probably provide a better introduction to Zulip
than Recent conversations.
2023-11-27 09:41:52 -08:00
Prakhar Pratyush 6f3b25d749 push_notification: Fix bad rendering of math formulas.
Earlier, for the push notifications having latex math
like "$$1 \oplus 0 = 1$$, the notification had the math
included multiple times.

This commit fixes the incorrect behavior by replacing
the KaTeX with the raw LaTeX source.

Fixes part of #25289.
2023-11-26 23:30:24 -08:00
Aman Agrawal 354330d81b decorator: Move self_hosting_management_endpoint wrapper to corporate. 2023-11-26 20:23:24 -08:00
Mateusz Mandera 2f935290f6 tests: Extract BouncerTestCase to test_classes.
This allows re-use in other test_*.py files, which may also want to test
bouncer-reliant logic.
2023-11-26 19:57:12 -08:00
Mateusz Mandera 860f94e599 tests: Remove useless line from test_send_realms_only_to_push_bouncer. 2023-11-26 19:57:12 -08:00
Mateusz Mandera 2a65183991 tests: Add test for nocoverage userAccountControl case in ldap auth.
This logic has been #nocoverage since its implementation, but since this
is an authentication codepath, it seems important for it to have a test.
2023-11-24 13:38:23 -08:00
Aman Agrawal f006be0cdf initial_upgrade: Rename to upgrade_page.
Replaced for "(initial_upgrade)", " initial_upgrade"
`'initial_upgrade'` and `"initial_upgrade"`.
2023-11-24 09:22:02 -08:00
Lalit 1089e13529 hotspots: Refactor hotspots system to use a new type `Hotspot`.
This commit refactors the current hotspot subsytem to use a more
robust dataclass `Hotspot` defined in `lib/hotspots.py`. This fixes
mypy errors as well as make code more readable.
2023-11-24 07:49:24 -08:00
Riken Shah b74f6afeb1 hotspot: Add frontend changes for non-intro hotspots.
This commit also solves a bug where it displayed
multiple copies of the hotspots when
`ALWAYS_SEND_ALL_HOTSPOTS` is set to true.
2023-11-24 07:49:24 -08:00
Riken Shah 8d633cc368 hotspot: Add backend changes for non-intro hotspots.
This commit introduces non-intro hotspots.
They are a bit different than intro hotspots in the
following ways:

* All the non-intro hotspots are sent at once instead of
sending them one by one like intro hotspots.

* They only activate when a specific event occurs,
unlike intro hotspot where they activate after the
previous hotspot is read.
2023-11-24 07:49:24 -08:00
Prakhar Pratyush 49388d5d3d topic_mentions: Fix restriction rule for @-topic mentions.
Now, the topic wildcard mention follows the following
rules:
* If the topic has less than 15 participants , anyone
can use @ topic mentions.
* For more than 15, the org setting 'wildcard_mention_policy'
determines who can use @ topic mentions.

Earlier, topic wildcard mentions followed the same restriction
as stream wildcard mentions, which was incorrect.

Fixes part of #27700.
2023-11-23 12:52:25 -08:00
Sahil Batra 189718dc64 settings: Add support to change user-access setting in development.
This commit updates the backend code to allow changing
can_access_all_users_group setting in development environment
and also adds a dropdown in webapp UI which is only shown in
development environment.
2023-11-23 10:40:42 -08:00
David Rosa 505a591179 help: Rename view-and-browse-images.md -> view-images-and-videos.md
- Updates filename and adds URL redirect.
- Crosslinks related articles.
2023-11-23 10:37:31 -08:00
Prakhar Pratyush 51b39cb682 stripe: Add 'do_update_plan' method to the 'BillingSession' class.
This commit moves a major portion of the 'update_plan`
view to a new shared 'BillingSession.do_update_plan' method.

This refactoring will help in minimizing duplicate code
while supporting both realm and remote_server customers.
2023-11-23 09:01:45 -08:00
Mateusz Mandera 3958743b33 corporate: Add prototype authentication system for self-hosters.
This makes it possible for a self-hosted realm administrator to
directly access a logged-page on the push notifications bouncer
service, enabling billing, support contacts, and other administrator
for enterprise customers to be managed without manual setup.
2023-11-22 17:03:47 -08:00
Mateusz Mandera 1ec0d5bd9d requests: Add SELF_HOSTING_MANAGEMENT_SUBDOMAIN. 2023-11-22 14:22:26 -08:00
Mateusz Mandera 2149cd236f settings: Add new SIGNED_ACCESS_TOKEN_VALIDITY_IN_SECONDS setting. 2023-11-22 14:22:26 -08:00
Mateusz Mandera 8187d6b963 home: Remove redundant condition in get_billing_info.
has_billing_access already has the is_realm_owner check:

    @property
    def has_billing_access(self) -> bool:
        return self.is_realm_owner or self.is_billing_admin
2023-11-22 14:22:26 -08:00
Mateusz Mandera 8695ffba49 remote_server_post_analytics: Change RealmDataForAnalytics.uuid type.
pydantic allows us to use UUID4 as a more accurate type.
2023-11-22 14:22:26 -08:00
Sahil Batra b1d5cd6bf6 realm: Allow setting notification settings to unsubscribed private streams.
We previously did not allow setting signup_notifications_stream and
notifications_stream settings to private streams that admin is not
subscribed to, even when admins have access to metadata of all the
streams in the realm and can see them in the dropdown options as well.

This commit fixes it to allow admins to set these settings to private
streams that the admin is not subscribed to.
2023-11-22 10:01:19 -08:00
Satyam Bansal bf2d216100 integrations: Add action to GitHub discussion comment notifications.
Previously, the notifications had "commented" as the action word
for every event.

As part of these changes, we extract a shared comment action function
in GitHub Integration that's used for both issue and discussion
comment events.
2023-11-22 08:28:03 -08:00
Satyam Bansal 62ec51f715 integrations: Add GitHub discussion comment edit fixture. 2023-11-22 08:28:03 -08:00
Satyam Bansal 9f01876de8 integrations: Update GitHub discussion related fixtures. 2023-11-22 08:28:03 -08:00
Satyam Bansal 22fa5a7ee3 integrations: Use consistent code boundaries in GitHub notifications.
In other templates we have used "~~~" to start and end a code block.
2023-11-22 08:28:03 -08:00
Lauryn Menard a4d1211ec6 integrations: Reformat Github pull request assigned message body.
Instead of adding the assignee to the end of the message body,
we update the message body where the verb is so that the link
formatting at the end of the message is not broken, for example:
"user_a assigned user_b to [issue #XXX title text is here](link)."

This matches the formatting for the issue assigned message body.
2023-11-22 08:26:09 -08:00
Lauryn Menard f6e17fa972 integrations: Reformat Github issue assigned message body.
Instead of adding the assignee to the end of the message body,
we update the message body where the verb is so that the link
formatting at the end of the message is not broken, for example:
"user_a assigned user_b to [issue #XXX title text is here](link)."

Also updates the issue title in the test fixture so that it tests
that only the first instance of "assigned" or "unassigned" in the
issue title is updated for the assignee text.

Also adds punctuation to the issue title in the test fixture to
test the expected behavior for titles that end in a value from
`string.punctuation`.
2023-11-22 08:26:09 -08:00
Aman Agrawal d82efbd503 free_trial: Remove extra onboarding flow.
We still redirect free trial users to upgrade page on first
signup but no longer pass the onboarding param.
2023-11-22 08:06:22 -08:00
Sahil Batra f75b4f65c1 streams: Send user remove events when deactivating streams. 2023-11-21 23:58:45 -08:00
Sahil Batra 4f58733d82 events: Remove deactivated streams from subscriptions field.
We did not remove the objects for deactivated streams from
subscriptions field in apply_event. We need to do this because
we do not send "subscription/remove" events to subscribers
when deactivating streams.
2023-11-21 23:58:45 -08:00
Sahil Batra 45e1b32447 users: Send user remove events on user deactivation.
Guests might lose access to deactivated users if the user
is not involved in any DM with guest. This commit adds
code to send "realm_user/remove" events for such cases.
2023-11-21 23:58:45 -08:00
Sahil Batra 58461660c3 users: Restrict accessing avatar for inaccessible users.
We now return the special avatar used for inaccessible users
when a guest user tries to access avatar of an inaccessibe
user using "/avatar" endpoint.
2023-11-21 23:58:45 -08:00
Sahil Batra 3d96969398 avatar: Update avatar used for inaccessible users.
This commit adds a new avatar image which will be shown for
all inaccessible users.
2023-11-21 23:58:45 -08:00
Sahil Batra 32c15d67b5 users: Send user creation events when sending DMs.
We now send user creation events to recipient users
when sending DMs if recipients gain access to either
sender or other pariticpating users in the DM.
2023-11-21 23:58:45 -08:00
Sahil Batra e4a97dd3ac message: Restrict sending DMs to inaccessible users.
This commit adds code to not allow guest users to send DMs
to users they cannot access.
2023-11-21 23:58:45 -08:00
Sahil Batra 39a31170ee streams: Send event when guest loses access to a user.
This commit adds code to send "realm_user/remove" event
when a guest user loses access to a user due to the user
being unsubscribed from one or more streams.
2023-11-21 23:58:45 -08:00
Sahil Batra d394cfc4db streams: Send user creation events on adding subscribers.
This commit adds code to send user creation events to
guests who gain access to new subscribers and to the
new guest subscribers who gain access to existing
stream subscribers.
2023-11-21 23:58:45 -08:00
Sahil Batra dbcc9ea826 users: Update presence and user status code to support restricted users.
The presence and user status update events are only sent to accessible
users, i.e. guests do not receive presence and user status updates for
users they cannot access.
2023-11-21 23:58:45 -08:00
Sahil Batra 650e55fef8 users: Send events only to users who can access the modified user.
This commit adds code to make sure that update events for changing
a user's role, email, etc. are not sent to guests who cannot access
the modified user.
2023-11-21 23:58:45 -08:00
Sahil Batra 6f14d105a7 create_user: Update data in user creation events for guests.
We do not send the original user data in user creation events
to guests if user access is restricted in realm, as they would
receive the information about user if user is subscribed to some
common streams after account creation.
2023-11-21 23:58:45 -08:00
Sahil Batra a23eff20fe users: Restrict read access to users in access_user_by_id.
This commit adds code to update access_user_by_id to raise
error if guest tries to access an inaccessible user.

One notable behavioral change due to this is that we do
not allow guest to mute or unmute a deactivated user if
that user was not involved in DMs.
2023-11-21 23:58:45 -08:00
Satyam Bansal cda7ed7101 integrations: Use correct type for GitHub pull request comment messages. 2023-11-21 21:05:26 -08:00
Satyam Bansal d2589a5bd1 integrations: Send GitHub pull request comment alerts to correct topic.
Pull request comment alerts were previously sent to a topic for an issue,
which resulted in two different topics for the same PR.

Fixes: #26086.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2023-11-21 21:05:26 -08:00
Satyam Bansal 7ebf572b8e integrations: Add pull request comment fixture to GitHub Integration.
Updated the repo name and pull request number/title for the new
pull request commit fixture to be the same as the one used for the
other pull request test fixtures (e.g. pull_request__opened) so
that the TOPIC_PR can be used in the subsequent updates.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2023-11-21 21:05:26 -08:00
Alex Vandiver 8b0cecc7e4 migrations: Fix revert migration to not lose all preferences.
Renumbering 4 -> 3, and then 3 -> 2 leads to everyone having their
preferences set to 2.  Swap the order, so that we renumber 3 -> 2,
then 4 -> 3.
2023-11-21 17:56:53 -08:00
Alex Vandiver dd954749be zilencer: Log, and drop, duplicated rows from remote servers.
This may happen if there are multiple servers with the same UUID
submitting data (e.g. if they were cloned after initial creation), or
if there is one server, but `./manage.py clear_analytics_tables` was
used to truncate the analytics tables.

In the case of `clear_analytics_tables`, the data submitted likely has
identical historical values with new remote `id` values; preserving
the originally-submitted contemporaneous data is the best option.  For
the case of submissions from multiple servers, there is no completely
sensible outcome, so the best we can do is detect the case and move
on.

Since we have a lock on the RemoteZulipServer, we know that no other
inserts are happening, so counting before and after will return the
true number of rows inserted (which `bulk_create` cannot do in the
face of `ignore_conflicts`[^1]).  We compare this to the expected
number of new inserted rows to detect dropped duplicates.

[^1]: See https://code.djangoproject.com/ticket/30138.
2023-11-21 11:44:55 -08:00
Alex Vandiver b363999d19 analytics: Slew record reporting by up to 10 minutes.
This reduces the giant load spike at 5 minute past the hour, when all
remote servers currently attempt to submit their records.

We do not wish to slew over a full hour, because we want to ensure
that we do not hold the lock when the next hour's analytics runs.  It
is also not necessary to have that much variation; 10 minutes is
picked as an arbitrary "long enough" time to spread requests over.
2023-11-21 10:49:57 -08:00
Alex Vandiver 85cc8b6a20 remote_server: Use analytics logger when reporting analytics. 2023-11-21 10:49:57 -08:00
Aman Agrawal 69d8442ab4 billing: Allow user to switch between billing frequencies. 2023-11-21 10:42:12 -08:00
Alex Vandiver 9bc41ca040 zilencer: Store the last-reported server version when storing analytics.
Servers since 216d2ec1bf (version 2.0.0)
have submitted this, but we have never stored it.
2023-11-20 14:36:27 -08:00
Prakhar Pratyush f7f5131aa8 email_notification: Fix bad rendering of math formulas.
Earlier, for the emails having latex math like
"$$d^* = +\infty$$", the bad rendering led to the math
being included multiple times in the email body.

This was due to displaying KaTeX HTML without the CSS.

This commit fixes the incorrect behavior by replacing
the KaTeX with the raw LaTex source.

Fixes part of #25289.
2023-11-20 10:41:02 -08:00
Aman Agrawal 71ea6e8863 realm_inline_image_preview: Use it to toggle video previews too.
This setting now also works to decide whether to show previews of
uploaded or linked videos.
2023-11-20 08:48:39 -08:00
Mateusz Mandera 357dceb05a typing: Rewrite remote_server_post_analytics to use @typed_endpoint.
The main point is the RealmDataForAnalytics structure, which we can next
re-use for other endpoints that will take it in in their params.
2023-11-17 18:32:49 -08:00
Mateusz Mandera ab633f4557 analytics: Add send_realms_only_to_push_bouncer function.
This is a useful helper using the same API as
send_analytics_to_push_bouncer(), but uploading only realms info. This
is useful to upload realms info without the risk of taking a long time
to process the request due to too much of the *Count analytics data.
2023-11-17 18:32:49 -08:00
Mateusz Mandera 1800b2c797 ldap: Tweak AUTH_LDAP_ADVANCED_REALM_ACCESS_CONTROL behavior.
The original behavior of this setting was to disable LDAP
authentication for any realms not configured to use it. This was an
arbitrary choice, and its only value was to potentially help catch
typos for users who are lazy about testing their configuration.

Since it makes it a very inconvenient to potentially host multiple
organizations with different LDAP configurations, remove that
behavior.
2023-11-17 14:40:26 -08:00
Prakhar Pratyush cc934429fe settings: Add option for followed topics to unread count badge setting.
This commit adds a new option 'DMs, mentions, and followed topics'
to 'desktop_icon_count_display' setting.

The total unread count of DMs, mentions, and followed topics appears
in desktop sidebar and browser tab when this option is configured.

Some existing options are relabeled and renumbered. We finally have:
* All unread messages
* DMs, mentions, and followed topics
* DMs and mentions
* None

Fixes #27503.
2023-11-17 14:07:20 -08:00
Tim Abbott 34f4622091 api_docs: Add missing followed topic visibility policy.
While the server implementation has accepted this value for a few
months as part of building the feature, following topics was not a
fully supported feature of the Zulip server before
3f2ab44f94, just before feature
level 219.

So that's probably the correct level to document as the first feature
level at which we recommend that clients supporting the followed
topics feature process the value.
2023-11-17 13:47:08 -08:00
Lauryn Menard 11cb37c9a4 billing: Add prototype remote billing sessions.
These new models are incomplete and totally untested, but merging this
will provide valuable scaffolding for doing smaller PRs working on
individual gaps, and reveals a clear set of TODOs/refactoring/model
changes needed to support where want to end up.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2023-11-17 12:58:37 -08:00
David 81f99dde55
gear_menu: Rename "Manage streams" -> "Stream settings".
Fixes #27754.
2023-11-17 08:53:41 -08:00
Tim Abbott c71f62089b api_docs: Clarify found_oldest/newest descriptions. 2023-11-16 18:00:37 -08:00
Mateusz Mandera 13aa24a56a drafts: Update some stale docstrings regarding draft data validation.
These were written before the draft endpoints were converted to use
@typed_endpoint and pydantic-based DraftData(BaseModel) for param
validation. Update them to avoid the confusion of talking about dicts
and dict_validator functions when those are no longer a thing.
2023-11-16 17:29:12 -08:00
Tim Abbott 3cfe4b720c Revert "linkifiers: Match JS implementation to server implementation."
This reverts commit 091e2f177b.

This version of python_to_js_linkifier fails for at least some real
linkifiers. We'll likely re-introduce this after a bit more debugging.
2023-11-16 14:59:48 -08:00
Greg Price 528a76a419 push_notifs: Set APNs topic individually on each notification.
This makes it possible to send notifications to more than one app ID
from the same server: for example, the main Zulip mobile app and the
new Flutter-based app, which has a separate app ID for use through its
beta period so that it can be installed alongside the existing app.
2023-11-16 12:03:30 -08:00
Greg Price ff32d51700 push_notifs: Support APNs token auth, as well as cert auth.
This will make it possible to send notifications to multiple
distinct app IDs over the same connection.
2023-11-16 12:03:30 -08:00
Alex Vandiver d6e809d185 streams: Fix unsorted import. 2023-11-16 19:02:13 +00:00
Alex Vandiver 5e4bb69498 subscription_info: Fix typo caught by codespell. 2023-11-16 18:55:31 +00:00
Sahil Batra dc98136346 streams: Send stream deletion events on unsubscribing users.
This commit adds code to send stream deletion events when
unsubscribing non-admin users from private streams and
when unsubscribing guests from public streams since
non-admins cannot access unsubscribed private streams
and guests cannot access unsubscribed public streams.
2023-11-16 13:25:33 -05:00
Sahil Batra d4fb244d2d CVE-2023-47642: Invalid metadata access for formerly subscribed streams.
It was discovered by the Zulip development team that active users who
had previously been subscribed to a stream incorrectly continued being
able to use the Zulip API to access metadata for that stream. As a
result, users who had been removed from a stream, but still had an
account in the organization, could still view metadata for that
stream (including the stream name, description, settings, and an email
address used to send emails into the stream via the incoming email
integration). This potentially allowed users to see changes to a
stream’s metadata after they had lost access to the stream.

This bug was present in all Zulip releases prior to today's Zulip
Server 7.5.
2023-11-16 13:25:33 -05:00
Sahil Batra 3c8701ee36 streams: Add API endpoint to get stream email.
This commit adds new API endpoint to get stream email which is
used by the web-app as well to get the email when a user tries
to open the stream email modal.

The stream email is returned only to the users who have access
to it. Specifically for private streams only subscribed users
have access to its email. And for public streams, all non-guest
users and only subscribed guests have access to its email.
All users can access email of web-public streams.
2023-11-16 13:25:33 -05:00
Sahil Batra 432001656e streams: Remove "email_address" field from Subscription objects.
This commit removes "email_address" field from Subscription objects
and we would instead a new endpoint in next commit to get email
address for stream with proper access check.

This change also fixes the bug where we would include email address
for the unsubscribed private stream as well when user did not have
permission to send message to the stream, and having email allowed
the unsubscribed user to send message to the stream.

Note that the unsubscribed user can still send message to the stream
if the user had noted down the email before being unsubscribed
and the stream token is not changed after unsubscribing the user.
2023-11-16 13:25:33 -05:00
Mateusz Mandera 1819b85b85 management: Allow changing is_billing_admin using change_user_role. 2023-11-15 18:36:07 -08:00
Anders Kaseorg c8784634e1 openapi: Remove unnecessary cast.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 17:10:48 -08:00
Anders Kaseorg f4e7a11c35 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Anders Kaseorg a688e753de test_helpers: Fix logging in cursor_executemany mock.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Anders Kaseorg cb9a04d3e3 test_create_video_call: Add missing not None assertions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Anders Kaseorg 2c9c105bbb db: Check connection type in wrapper_execute.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Anders Kaseorg 248ee82db0 migrations: Remove useless null argument for ManyToManyField.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Anders Kaseorg 9552014047 models: Include explicit app name in ManyToManyField reference strings.
https://github.com/typeddjango/django-stubs/issues/1802

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-15 15:27:54 -08:00
Tim Abbott 2e2997bd7d typing: Limit typing notifications in large streams. 2023-11-15 09:42:25 -08:00
Alex Vandiver 091e2f177b linkifiers: Match JS implementation to server implementation.
Since the server-side implementation no longer uses look-ahead
or (more importantly) look-behind, it is possible to exactly implement
in Javascript.  This removes a common class which would prevent local
echo.

This requires reworking the topic linking algorithm, to march the
server's as well.  The tests and behaviour are adjusted in so doing --
previously, the JS implementation would have linked `#foo` with a
`foo` regex on the linkifier, but the server implementation would not
have.
2023-11-14 20:43:39 -08:00
Alex Vandiver 70b20e9d2b markdown: Use \p{White_Space} equivalent for linkifier boundaries.
We do not use \p{White_Space} itself because re2 does not support it.
2023-11-14 20:43:39 -08:00
Alex Vandiver 549dd8a4c4 markdown: Expand prepare_linkifier_pattern to make it more legible. 2023-11-14 20:43:39 -08:00
Sahil Batra 532293c4cf streams: Unset is_web_public and is_realm_public fields on attachments.
This commit adds code to unset is_web_public and is_realm_public fields
on attachments when deactivating a stream as we do not want to allow
spectators to access them after the stream is deactivated.

This commit also adds a comment explaining why we don't use
do_change_stream_permission to set the privacy fields on deactivating
a stream.

Fixes #27634.
2023-11-13 10:26:14 -08:00
Sahil Batra 30e2c14f4a streams: Unset is_realm_public on attachment when unarchiving streams.
We did not unset is_realm_public field on attachements when unarchiving
streams, but we do unset is_web_public field. This commit adds code to
unset the is_realm_public field as well as we make the stream private
while unarchiving it.
2023-11-13 10:26:14 -08:00
Alex Vandiver 1f3a53db2d cache: Drop the huddle cache.
This cache was only used in one place, which is infrequently
called (only when sending messages, or searching explicitly for a list
of users) and the overhead of maintaining the cache is not worth
trying to avoid the well-indexed lookup of the huddle.
2023-11-13 10:01:29 -08:00
Sahil Batra 8f79cec51a users: Pass bogus data for inaccessible users.
We now pass bogus data for inaccessible users when sending
the users data in "realm_users" field of "register" response
or when using endpoints like "GET /users" to get data of
all the users in realm.

We would add a client capability field in future commits
such that new clients would receive data only for accessible
users and they can form the bogus data by themselves.
2023-11-13 08:04:45 -08:00
Sahil Batra aedb9b23b4 models: Pass "realm__can_access_all_users_group" to select_related. 2023-11-13 08:04:45 -08:00
Sahil Batra c82bb3ec76 models: Add can_access_all_users_group setting.
This commit adds new setting for controlling who can access
all users in the realm which would have "Everyone" and
"Members only" option.

Fixes part of #10970.
2023-11-13 08:04:45 -08:00
Sahil Batra f20d427a4d users: Remove unnecessary comment.
This comment can be removed as the code was updated in 2ab71fb67d.
2023-11-13 08:04:45 -08:00
Sahil Batra df01d0725c message_send: Remove select_related call for stream.
We do not access any related field from the stream object,
so there is no need for select_related.
2023-11-13 08:04:45 -08:00
Mateusz Mandera 48db4bf854 counts: Add new mobile_pushes RemoteRealmCount stats.
This requires a bit of complexity to avoid a name collision in
COUNT_STATS with the RemoteInstallationCount stats with the same name.
2023-11-10 16:09:11 -08:00
Mateusz Mandera 8a6d5b4997 counts: Add new Add new mobile_pushes_sent::day LoggingCountStat.
This is a CountStat for tracking how many mobile notifications the
server requested.
1. On a self-hosted server, that means requesting from the push bouncer.
2. On a server that's its own push bouncer, that's just the number
   directly sent.

This number has room for inaccuracy due to incrementing by the number of
user devices on a self-hosted server, as it doesn't account for errors
that may occur in the GCM/APNs low-level sending codepaths on the bouncer.

Also tests that a server that's its own push bouncer correctly
increments its mobile_pushes_sent::day CountStat, by basing it on the
values returned from the send_apple/android_push_notification functions
which tell us the actual number of successfully sent notifications.

Since the return values of send_..._push_notification are now
used in those codepaths, we need to tweak our mocks in some unrelated
tests to set up some return value to avoid errors.
2023-11-10 16:09:11 -08:00
Prakhar Pratyush c597de6a1d topic_mentions: Rename wildcard_mentioned to stream_wildcard_mentioned.
Rename the existing 'wildcard_mentioned' flag to
'stream_wildcard_mentioned'.

The 'wildcard_mentioned' flag is deprecated and exists for
backwards compatibility.

We have two separate flags for stream and topic wildcard mentions,
i.e., 'stream_wildcard_mentioned' and 'topic_wildcard_mentioned',
respectively.

* stream wildcard mentions: `@all`, `@everyone`, and `@stream`
* topic wildcard mentions: `@topic`

The `wildcard_mentioned` flag is included in the events and
API response if either `stream_wildcard_mentioned` or
`topic_wildcard_mentioned` is set.
2023-11-10 11:06:26 -08:00
Prakhar Pratyush 24fa361f40 migration: Clear old data for unused usermessage flags.
In c37871ac3a, we renamed the
two unused and historical bits of the 'flags' bitfield of
the 'UserMessage' table:

* 'summarize_in_home' to 'topic_wildcard_mentioned'
* 'summarize_in_stream' to 'group_mentioned'

This commit clears out the old data for those bits.

Additionally, we are clearing 'force_expand' and 'force_collapse'
unused flags to save future work.
2023-11-10 11:06:26 -08:00
Mateusz Mandera 1312c7ccd7 zilencer: Add mechanism to update RemoteRealm when Realm is changed.
This requires a migration to allow RemoteRealmAuditLog.remote_id to be
NULL, and to add a RemoteRealmAuditLog.remote_realm.
2023-11-08 15:54:22 -08:00
Mateusz Mandera 76e0511481 zilencer: Add new model RemoteRealm and send the data to the bouncer.
Add the new model for recording basic information about Realms on remote
server, to go with the other analytics data. Also adds necessary changes
to the bouncer endpoint and the send_analytics_to_push_bouncer()
function to submit such Realm information.
2023-11-08 15:54:22 -08:00
Tim Abbott 0d525cf644 integrations: Recommend new integration URL tool.
Fixes part of #25976.
2023-11-08 14:19:05 -08:00
David Rosa 0c4ef58acd help: Document new help menu.
Updates help center docs to reflect the new dedicated help menu.

- Adds `keyboard_tip` to document the `?` keyboard shortcut.

Fixes #27384.
2023-11-08 14:17:13 -08:00
Alex Vandiver acb4e9e783 cache_helpers: Show item and DB query counts. 2023-11-08 11:00:00 -08:00
roanster007 dc492867af user_mention: Fix mentions of deactivated users.
Previously, when a deactivated user was mentioned, he wasn't
rendered as a Pill. This is because the dataset for validating mentions
only included active users, which is fixed by removing that filter.

To allow only silent mentions of them, an extra is_active property
added to FullNameInfo class, which is populated from the query,
which tells if user is deactivated. This is used to convert any
mentions of them to silent mentions in the backend markdown.

Fixes #26857
2023-11-08 09:48:31 -08:00
Sahil Batra 617d2d509c users: Update format_user_row to return a TypedDict.
This commit updates format_user_row to return a TypedDict.

This commit is a prep commit for feature of restricting user
access such that code can be easy to read and understand when
we add that feature.
2023-11-08 09:22:26 -08:00
Sahil Batra 2ab71fb67d users: Update user_profile_to_user_row to return a TypedDict.
This commit updates user_profile_to_user_row to return a TypedDict
and also updates the return type of get_realm_user_dicts to be a
TypedDict.

This commit is a prep commit for feature of restricting user
access such that code can be easy to read and understand when
we add that feature.
2023-11-08 09:22:26 -08:00
Sahil Batra deb8431a47 users: Rename get_raw_user_data to get_users_for_api.
This is a prep commit for adding feature of restricting
user access to guests such that we can keep the code
easy to read and understand when that feature is added.
2023-11-08 09:22:26 -08:00
Greg Price 7a0ebc0d18 push_notifs: Log an error if ios_app_id missing on any APNs token. 2023-11-07 16:19:42 -08:00
Greg Price f109e3b598 push_notifs: Backfill ios_app_id on bouncer. 2023-11-07 16:19:42 -08:00
Greg Price be2a9a03d0 push_notifs: Check app ID has a plausible shape at bouncer. 2023-11-07 16:19:42 -08:00
Greg Price 1b2178f558 push_notifs: Require ios_app_id on register at bouncer, too.
We'll need this information in order to properly direct APNs
notifications.  Happily, the Zulip server always sends it when
registering an APNs token; and it appears it always has done so
since the commit:

cddee49e7 Add support infrastructure for push notification bouncer service.

back in 2016.  So there's no compatibility issue from requiring it.
2023-11-07 16:19:42 -08:00
Greg Price 9c2d53bcef push_notifs: Stop dropping ios_app_id on floor at bouncer.
This missing `REQ` call has meant we just drop this parameter:
even though the remote Zulip server passes it (for all APNs tokens),
we never notice and never store it.  Fix that.
2023-11-07 16:19:42 -08:00
Greg Price 9072ddda05 push_notifs: Remove remaining references to ZULIP_IOS_APP_ID setting. 2023-11-07 16:19:42 -08:00
Greg Price a9796ec503 push_notifs: Check app ID has a plausible shape. 2023-11-07 16:19:42 -08:00
Greg Price fb3af7fbcb push_notifs: Make appid required on add_apns_device_token.
We're going to need to use this information, so we shouldn't just
assume a value; the client should tell us the actual value.

Conveniently, the Zulip mobile app does already pass this parameter
and has since forever.  So we can just start requiring it, with no
compatibility constraint.
2023-11-07 16:19:42 -08:00
Greg Price 5d0d35d3f5 push_notifs: Always supply appid to add_apns_device_token in tests.
We already always pass this parameter from the mobile client,
so this makes the tests more realistic already.  And we'll shortly
be making this parameter required.
2023-11-07 16:19:42 -08:00
Greg Price 3893a70888 push_notifs: Cut spurious token_kind arguments in some test data.
These endpoints distinguish the token kinds by having different
URL paths, and don't take any argument by this name.
2023-11-07 16:19:42 -08:00
Alya Abbott 8fd631bf06 integrations-docs: Update intro content for Slack incoming webhooks. 2023-11-07 12:59:24 -08:00
Lauryn Menard 1618060a3e integrations-docs: Update both Slack integration articles.
Updates the Slack integration page to not describe adding a stream
or topic parameter to the URL query since that's not supported by
the current integration implementation.

Updates the Slack-compatible webhook integration page to have the
extra notes about the integration at the top of the page. Also,
removes the reference to a screenshot of the webhook since there
isn't one.
2023-11-07 12:59:24 -08:00
Prakhar Pratyush 94679d590f email_notification: Include prior message context only when mentioned.
Earlier, email message notifications included prior messages sent
to the same topic for context. This is more confusing than helpful
for messages that the user is likely to have received notifications
for all the prior messages in the conversation already (or read them
in the Zulip UI).

Now, we include prior context only when the user is mentioned via
personal, group, stream or topic wildcard mention.

Fixes #27479.
2023-11-07 11:44:38 -08:00
Prakhar Pratyush c0f445294c test_message_notif_email: Verify email has multiple stream messages.
This commit improves the test to explicitly verify that multiple
messages that were sent in quick succession to a topic are included
in the email body when we have email notifications enabled for a
given stream.

Earlier, the test was only verifying the email subject and the fact
that only one email was sent.

It is important to verify the fact that all the messages sent to a
topic in quick succession should be included in the email body.
2023-11-07 11:44:38 -08:00
Lauryn Menard fff2c9e47b integrations-docs: Update GitHub doc for new URL instructions. 2023-11-07 11:21:05 -08:00
Prakhar Pratyush e6e156709a typing_notifications: Don't notify long_term_idle subscribers.
The event for stream typing notifications is no longer sent
to the long_term_idle subscribers of the stream.

This helps to reduce the tornado's work of parsing super-long
JSON-encoded lists of user IDs in large streams. Now the lists
are shorter.
2023-11-07 09:30:27 -08:00
Aman Agrawal be6f467f42 home: Add `sponsorship_pending` page_param.
This will be used in gear menu to inform admin of their
sponsorship application status.

This includes some additional tweaks for when to show
billing and plans to users.
2023-11-06 14:55:14 -08:00
Aman Agrawal 00df9c6c18 urls: Show sponsorship page on a new URL.
Fixes #27377

Replaced #sponsorship links with sponsorship/ links and in other
places liking gear menu.
2023-11-06 14:55:14 -08:00
David Rosa 56f725e39b help: Document ability to drag-and-drop anywhere to upload a file.
- Replaces the "Via Markdown" tab with "Via drag-and-drop", and
modifies the instructions to explain that you can drag and drop
anywhere in the app, whether or not the compose box is open.
- Adds "Via paste" tab for the copy-pasting instructions.

Fixes #26894.
2023-11-04 12:49:54 -07:00
Aman Agrawal ddab0e6cd2 test_user_groups: Fix flaky tests.
There order of group ids doesn't matter here and thus the
compared values can have the ids in different order and test
should still pass. So, using `set` for comparing unordered
lists seems like the right fix here.
2023-11-04 12:46:06 -07:00
roanster007 ac084ddb0e mention_dataset: Add query to mention cross realm bots as pill.
Previously, cross realm bots were not displayed as mention Pills.

This is because, the data set for validating mentions considers
only the realm id which is None in case of cross realm bots.

Hence, adding an or Q object to it, to also check if
the email is a part of the cross realm bots email, in case the
realm id returns None.

Fixes #26913
2023-11-03 12:29:20 -07:00
Aman Agrawal 9ebb3f06b3 populate_billing_realms: Add script to populate different realms.
These realms are based on different cases of realms which can
exist while billing them and hence are useful for testing.
2023-11-03 08:44:23 -07:00
David Rosa c486ae37d3 help: Document option to paste URL to create a named link.
- Adds a "Via paste" tab to document the new functionality.

Fixes #26892.
2023-11-02 12:44:33 -07:00
Prakhar Pratyush b0ef76bf27 topic_mentions: Set 'topic_wildcard_mentioned' flag on @topic mention.
Earlier, the 'wildcard_mentioned' flag was set for both the
stream and topic wildcard mentions.

Now, the 'topic_wildcard_mentioned' flag is set for topic
wildcard mentions, and the 'wildcard_mentioned' flag is set for
stream wildcard mentions.

We will rename the 'wildcard_mentioned' flag to
'stream_wildcard_mentioned' in a later commit.
2023-11-02 09:25:51 -07:00
Prakhar Pratyush c37871ac3a user_message: Rename unused flags and create an index.
This commit renames the two unused and historical bits of the
'fields' bitfield of the 'UserMessage' and 'ArchivedUserMessage'
tables.

* 'summarize_in_home' to 'topic_wildcard_mentioned'
* 'summarize_in_stream' to 'group_mentioned'

The 'group_mentioned' flag doesn't affect the feature,
but completing the work here helps to save future migration
and indexing efforts on the UserMessage table, as we plan to
use this flag in the future for group mentions.

The unused bits may have old data; we'll clear that in
a separate commit.

It creates the 'zerver_usermessage_any_mentioned_message_id'
index concurrently.
2023-11-02 09:25:51 -07:00
Alex Vandiver 5a44a6624b gitlab: Description can be none. 2023-11-02 09:24:31 -07:00
Sahil Batra bb15b2d708 users: Send "update" events when deactivating or reactivating users.
We now send "realm_user/update" (and "realm_bot/update" for bots)
events with "is_active" field when deactivating and reactivating
users, including bots.

We would want to use "remove" event for a user losing access
to another user for #10970, so it is better to use "update"
event for deactivation as we only update "is_active" field
in the user objects and the clients still have the data for
deactivated users.

Previously, we used to send "add" event for reactivation along
with complete user objects, but clients should have the data
for deactivated users as well, so an "update" event is enough
like we do when deactivating users.
2023-11-02 08:31:12 -07:00
Mateusz Mandera 6819ecee92 zilencer: Add new LoggingCountStat mobile_pushes_forwarded.
This one counts actual successful deliveries.
2023-11-01 17:26:10 -07:00
Mateusz Mandera b7117d51b2 zilencer: Don't allow syncing mobile_pushes_received::day count. 2023-11-01 17:26:10 -07:00
Mateusz Mandera 7eb24ff237 send_test_push_notification_api: Fixes to the API endpoint doc.
Fixes misspellings and bad wording in the original zulip.yaml doc for
this endpoint.
2023-11-01 16:57:04 -07:00
Alex Vandiver 779268c3a4 sentry: Circuit-break on 429s and 500s from Sentry. 2023-11-01 16:16:37 -07:00
Alex Vandiver 631a1def13 portico: Skip footer and adjust header on invalid realm. 2023-11-01 11:24:12 -07:00
Alex Vandiver 54605e54b6 tests: Switch test_i18n fetch helper to use client_foo methods.
38f2a2f475 updated the comment but not the code.  Using
`self.client.post` instead of `self.client_post` means that we do not
set the host headers correctly.
2023-11-01 11:24:12 -07:00
Sahil Batra 47c8e369cf register: Pass the configuration objects for group settings in response.
This commit adds code to pass configuration objects for group
permission settings in register response to clients such that
we do need to duplicate that data in clients and can avoid
future bugs due to inconsistency.

The "server_supported_permission_settings" field is included
in the response if "realm" is present in "fetch_event_types",
as this is what we do for other server-related fields.
2023-11-01 10:42:56 -07:00
Sahil Batra e458b73a01 user_groups: Move constants for system group names to a new class.
This commit moves constants for system group names to a new
"SystemGroups" class so that we can use these group names
in multiple classes in models.py without worrying about the
order of defining them.
2023-11-01 10:42:56 -07:00
Sahil Batra 2c43c2a4db user_groups: Pass config object to access_user_group_for_setting.
We now pass the complete configuration object for a setting to
access_user_group_for_setting instead of passing the configuration
object's fields as different variables.
2023-11-01 10:42:56 -07:00
Sahil Batra 08d41f9eb3 settings: Rename permissions_configuration variable.
This commit renames permissions_configuration variable to
permission_configuration since the object contains config for
a single permission setting and thus permission_configuration
seems like a better name.
2023-11-01 10:42:56 -07:00
Lauryn Menard 6671a4a3f5 api-docs: Update description of update_message_flags op: add event.
Updates the description of update_message_flags op: add event for
details about actions that send/trigger the event and other details
that are useful for client implementations.

Also, links to the above updates in the op: remove variant of the
update_message_flags event.
2023-10-31 14:28:30 -07:00
Lauryn Menard afa5119303 api-docs: Update delete_message event for moved messages case. 2023-10-31 14:28:30 -07:00
Sahil Batra 9a6cf82adc streams: Fix sending stream-related events to guests.
Previous behavior-
- Guest did not receive stream creation events for new
web-public streams.
- Guest did not receive peer_add and peer_remove events
for web-public and subscribed public streams.

This commit fixes the behavior to be -
- Guests now receive stream creation events for new
web-public streams.
- Guest now receive peer_add and peer_remove events for
web-public and subscribed public streams.
2023-10-31 10:54:21 -07:00
Sahil Batra 71b8f49614 streams: Return early if there is no change in subscriptions.
This commit updates code in bulk_remove_subscriptions and
bulk_add_subscriptions to return early if there are no
subscribers to remove or add to the streams.

This change helps us in avoiding unnecessary queries like the
one used to get subscribers list of streams, which is then used
to send events but we would not send any events if no subscribers
are added or removed and some more similar queries.
2023-10-31 10:54:21 -07:00
Sahil Batra cdd15b4a69 test_events: Fix typo in comments. 2023-10-31 10:54:21 -07:00
Chandan Yadav 951e31a154 user_status: Remove `status` field from `UserInfoDict` type. 2023-10-30 11:10:47 -07:00
Anders Kaseorg 14b7d097a0 compilemessages: Normalize language names to NFC.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-26 16:27:35 -07:00
Hemant Umre d27a50852e onboarding: Internationalize the introductory messages in streams.
We use `Realm.default_language` value, which is set by selecting
the 'Organization language', to internationalize the introductory
messages of the initial streams.

Fixes #25729.
2023-10-26 16:27:35 -07:00
Hemant Umre ac1f711fef registration: Set the organization language at creation time.
In this commit, we add a new dropdown 'Organization language' on
the `/new` and `/realm/register` pages. This dropdown allows setting
the language of the organization during its creation. This allows
messages from Welcome Bot and introductory messages in streams to be
internationalized.

Fixes a part of #25729.
2023-10-26 16:27:35 -07:00
Sahil Batra c9fccf476a settings: Rename default_view and escape_navigates_to_default_view.
This commit renames default_view and escape_navigates_to_default_view
settings to web_home_view and web_escape_navigates_to_home_view in
database and API to match with our recent renaming of user facing
strings related to this.

We also rename the variables, functions, comments in code and class
names and IDs for elements related to this.
2023-10-26 11:00:12 -07:00
Lauryn Menard 02d6b3e16d onboarding-emails: Add new onboarding email for organization creator.
Adds a new onboarding email `onboarding_team_to_zulip` for the user
who created the new Zulip organization.

Co-authored by: Alya Abbott <alya@zulip.com>
2023-10-25 13:51:03 -07:00
Lauryn Menard f755fd1a3a api-docs: Check for deprecated at top level of endpoint docs.
Previously, we had checked that deprecated parameters and return
values had been marked as `deprecated: true` in the OpenAPI
documentation and had a description with a deprecated note.

Here we extend that check at the top level to deprecated endpoints.

The backend test that catches a failed assertion for this check
is `test_api_doc_endpoints` in zerver/tests/test_docs.py as that
test checks for a success response all pages linked the sidebar of
the API documentation.
2023-10-25 12:21:53 -07:00
Lauryn Menard 94088a7709 api-docs: Add deprecated true to /api/mute-topic endpoint. 2023-10-25 12:21:53 -07:00
Lauryn Menard 8905d918ae api-docs: Document bulk mark as read endpoints as deprecated. 2023-10-25 12:21:53 -07:00
Mateusz Mandera bb63aca8bb send_analytics_to_push_bouncer: Fix a stale comment.
The comment has drifted away from where it should be placed within the
code and also talks about RealmCounts specifically, while we have other
object types that this equally applies to.
2023-10-25 11:09:49 -07:00
Mateusz Mandera 986f2fd962 send_analytics_to_remote_server: Rename to ..._to_push_bouncer.
The former name is kind of misleading - this function is for the remote
server to send analytics to the push bouncer. Under our usual
terminology, a "remote server" is a self-hosted Zulip server. So data is
sent FROM not TO a remote server.
2023-10-25 11:09:49 -07:00