Commit Graph

8044 Commits

Author SHA1 Message Date
Prakhar Pratyush 32a5c422e9 migration: Make 'rename_indexes_constraints' a lib function.
This prep commit moves the 'rename_indexes_constraints'
function to 'lib/migrate' as we're going to re-use it for
the 'UserHotspot' to 'OnboardingStep' table rename operation.

In general, this function would be helpful in migrations
involving table rename operations, subject to the caution
mentioned in the function via comments.
2023-12-06 18:19:20 -08:00
Prakhar Pratyush 777398fc82 test_helpers: Maintain alphabetical order of models in 'use_db_models'. 2023-12-06 18:19:20 -08:00
Alex Vandiver 8d5573b395 narrow: Prevent contradicting DM and stream narrows.
These confused `ok_to_include_history` and caused exceptions looking
for the "flags" column.
2023-12-06 16:06:15 -08:00
Sahil Batra 6c3fbcc383 push_notifications: Do not include details of inaccessible users.
This commit adds code to include original name, email and avatar
for inaccessible users which can happen when a user sends message
to an unsubscribed stream.
2023-12-06 00:09:53 -08:00
Sahil Batra 965869d3f8 register: Add client capability to not receive unknown users data.
This commit adds a new client capability to decide whether the
client needs unknown users data or not.
2023-12-06 00:09:53 -08:00
Sayam Samal bcc58edfda hotspots: Fix misplaced onboarding hotspots.
We rename "intro_gear" to "intro_personal" because after the menu
was split into help menu, main menu and personal menu, the "Settings"
option now resides inside the personal menu.

Fixes #27878.
2023-12-05 17:37:47 -08:00
Anders Kaseorg 223b626256 python: Use urlsplit instead of urlparse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg 3853fa875a python: Consistently use from…import for urllib.parse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg 8a7916f21a python: Consistently use from…import for datetime.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 12:01:18 -08:00
Mateusz Mandera 8b55d60f9e populate_db: Create RemoteZulipServer with proper details from settings.
This creates a valid registration, for two reasons:
1. Avoid the need to run "manage.py register_server" in dev env to
   register, when wanting to to test stuff with
   `PUSH_NOTIFICATION_BOUNCER_URL = "http://localhost:9991"`.
2. Avoid breaking RemoteRealm syncing, due to duplicate registrations
   (first set of registrations that gets set up with the dummy
   RemoteZulipServer in populate_db, and the second that gets set up via
   the regular syncing mechanism with the new RemoteZulipServer created
   during register_server).
2023-12-05 11:34:57 -08:00
Tim Abbott fbd8ed1425 import_realm: Move push bouncer ping after realm reactivation. 2023-12-04 07:08:39 -08:00
Tim Abbott f6c7eaf1e5 models: Add push_notifications_enabled & corresponding end_timestamp.
Add two fields to Realm model:
*push_notifications_enabled
*push_notifications_enabled_end_timestamp

Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
2023-12-04 07:08:39 -08:00
Prakhar Pratyush 6aa911a9b2 remote_server_post_analytics: Return remote realms data in response.
This is a prep commit to return, for each remote realm, the 'uuid',
'can_push', and 'expected_end_timestamp'.

This data will be used in 'initialize_push_notifications'.
2023-12-04 07:08:39 -08:00
Mateusz Mandera ec7245d4e1 remote_billing: Add redirect flow for users with expired session.
Implements a nice redirect flow to give a good UX for users attempting
to access a remote billing page with an expired RemoteRealm session e.g.
/realm/some-uuid/sponsorship - perhaps through their browser
history or just their session expired while they were doing things in
this billing system.

The logic has a few pieces:
1. get_remote_realm_from_session, if the user doesn't have a
   identity_dict will raise RemoteBillingAuthenticationError.
2. If the user has an identity_dict, but it's expired, then
   get_identity_dict_from_session inside of get_remote_realm_from_session
   will raise RemoteBillingIdentityExpiredError.
3. The decorator authenticated_remote_realm_management_endpoint
   catches that exception and uses some general logic, described in more
   detail in the comments in the code, to figure out the right URL to
   redirect them to. Something like:
   https://theirserver.example.com/self-hosted-billing/?next_page=...
   where the next_page param is determined based on parsing request.path
   to see what kind of endpoint they're trying to access.
4. The remote_server_billing_entry endpoint is tweaked to also send
   its uri scheme to the bouncer, so that the bouncer can know whether
   to do the redirect on http or https.
2023-12-03 10:39:56 -08:00
Mateusz Mandera 7f33d6f0ea zilencer: Tie RemotePushDeviceToken to RemoteRealm at registration.
This consists of the following pieces:
1. Makes servers using the bouncer send realm_uuid in requests for token
   registration. (Sidenote: realm_uuid is already sent in the "send
   notification" codepath as of
   48db4bf854)
2. This allows the bouncer to tie RemotePushDeviceToken to the
   RemoteRealm with matching realm_uuid at registration time.
3. Introduce handling of some potential weird edge cases around the
   realm_uuid and RemoteRealm objects in get_remote_realm_helper.
2023-12-03 09:51:45 -08:00
Mateusz Mandera c9b0602320 tests: Create default RemoteRealms in populate_db.
This default setup will be more realistic, matching the ordinary
conditions for a modern server.
Especially needed as we add bouncer code that will expect to have
RemoteRealm entries for realm_uuid values for which it receives
requests.
2023-12-03 09:51:45 -08:00
Mateusz Mandera a67dd6dc1f realms: Call send_realms_only_to_push_bouncer at realm creation/import. 2023-12-03 08:49:58 -08:00
Anders Kaseorg 1efc6efd23 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-01 18:52:00 -08:00
Alex Vandiver 7f96bed17b stream_traffic: Use the realm_id to get a much better-indexed query.
This reduces the query time by an order of magnitude, since it is able
to switch from a raw `stream_id` index to an index over all of
`realm_id, property, end_time`.
2023-11-30 12:32:30 -08:00
Alya Abbott 7b71ea3314 help: Add tab for Zulip 8.0+ features. 2023-11-29 23:44:00 -08:00
Tim Abbott 610338d192 billing: Add BILLING_SCHEDULE_ prefix to values. 2023-11-29 23:32:56 -08:00
Tim Abbott 5d6b635efe billing: Use better variable names for plan tiers.
The existing values didn't have our standard type-prefixing naming
scheme.

Add some extra unused placeholder values while we're at it.
2023-11-29 23:32:56 -08:00
David Rosa af3956e1a9 help: Document upgrade flow, billing, and sponsorship requests.
Documents how to upgrade to a paid plan, manage billing, and
apply for sponsorship.

Fixes #27946.
2023-11-29 21:50:44 -08:00
Mateusz Mandera 9b1a495e2c zilencer: Sync name and authentication_methods on RemoteRealm. 2023-11-29 15:54:38 -08:00
Tim Abbott 4ef6b7cc44 notifications: Rename apns_enabled and gcm_enabled.
The new names are a lot clearer when thinking about self-hosted
systems that might indirectly use these via a bouncer.
2023-11-29 15:06:46 -08:00
Tim Abbott b71c5746ab notifications: Rename push_notifications_enabled for clarity.
This doesn't actually check if push notifications are working, just
whether there is configuration for them.
2023-11-29 15:06:46 -08:00
Tim Abbott 7db15176f3 push bouncer: Submit basic metadata unconditionally.
These metadata are essentially all publicily available anyway, and
making uploading them unconditional will simplify some things.

The documentation is not quite accurate in that it claims the server
will upload some metadata that is not actually uploaded yet (but will
by soon). This seems harmless.
2023-11-29 14:45:53 -08:00
Prakhar Pratyush d8cf12eaaa send_email: Improve configurability for outgoing email sender name.
Currently, the sender names for outgoing emails sent by Zulip
are hardcoded. It should be configurable for self-hosted systems.

This commit makes the 'Zulip' part a variable in the following
email sender names: 'Zulip Account Security', 'Zulip Digest',
and 'Zulip Notifications' by introducing a settings variable
'SERVICE_NAME' with the default value as f"{EXTERNAL_HOST} Zulip".

Fixes: #23857
2023-11-29 14:20:01 -08:00
Prakhar Pratyush 78a75ab28c digest: Fix the `_` used for an unused variable.
This prep commit replaces `_` with `ignored` to represent
an unused variable.

In later commits, we are going to use `_` for translation,
which leads to a lint error.
2023-11-29 14:20:01 -08:00
Sahil Batra a6fa875c23 models: Refactor get_fake_email_domain to take realm.host as arg.
This commit updates get_fake_email_domain to accept realm.host as
argument instead of the Realm object since we only use realm.host
to get the fake email domain.

This is a preparatory commit for the limited guest feature as we
would be sending the fake email of the message sender in message
event object to a guest user who cannot access the sender and
there we would need to compute the fake email.
2023-11-29 12:01:37 -08:00
Alex Vandiver 82c08dd153 python: Prevent bare timedelta(), which defaults to days. 2023-11-28 15:10:39 -08:00
Alex Vandiver 8c8dbb3d66 markdown: Stop attempting to expand/collapse re2 regex.
549dd8a4c4 changed the regex that we build to contain whitespace for
readability, and strip that back out before returning it.
Unfortunately, this also serves to strip out whitespace in the source
linkifier, causing it to not match expected strings.

Revert 549dd8a4c4.

Fixes: #27854.
2023-11-28 15:07:23 -08:00
Mateusz Mandera 02d5740f0f remote_realm: Add syncing of org_type. 2023-11-28 14:41:16 -08:00
roanster007 8bd92fe801 bots: Fix muting of cross realm bots.
Previously, we weren't able to mute the cross realm bots. This was
because, for muting the users, we access only those profiles which are
in realm, excluding the cross realm system bots.

This is fixed by replacing the access_user_by_id method with a new
method access_user_by_id_including_cross_realm for this specific test.

Fixes #27823
2023-11-27 16:16:23 -08:00
Aman Agrawal a96619639c events_register: Set home view for spectators as recent view. 2023-11-27 09:41:52 -08:00
Prakhar Pratyush 6f3b25d749 push_notification: Fix bad rendering of math formulas.
Earlier, for the push notifications having latex math
like "$$1 \oplus 0 = 1$$, the notification had the math
included multiple times.

This commit fixes the incorrect behavior by replacing
the KaTeX with the raw LaTeX source.

Fixes part of #25289.
2023-11-26 23:30:24 -08:00
Mateusz Mandera 2f935290f6 tests: Extract BouncerTestCase to test_classes.
This allows re-use in other test_*.py files, which may also want to test
bouncer-reliant logic.
2023-11-26 19:57:12 -08:00
Lalit 1089e13529 hotspots: Refactor hotspots system to use a new type `Hotspot`.
This commit refactors the current hotspot subsytem to use a more
robust dataclass `Hotspot` defined in `lib/hotspots.py`. This fixes
mypy errors as well as make code more readable.
2023-11-24 07:49:24 -08:00
Riken Shah b74f6afeb1 hotspot: Add frontend changes for non-intro hotspots.
This commit also solves a bug where it displayed
multiple copies of the hotspots when
`ALWAYS_SEND_ALL_HOTSPOTS` is set to true.
2023-11-24 07:49:24 -08:00
Riken Shah 8d633cc368 hotspot: Add backend changes for non-intro hotspots.
This commit introduces non-intro hotspots.
They are a bit different than intro hotspots in the
following ways:

* All the non-intro hotspots are sent at once instead of
sending them one by one like intro hotspots.

* They only activate when a specific event occurs,
unlike intro hotspot where they activate after the
previous hotspot is read.
2023-11-24 07:49:24 -08:00
Prakhar Pratyush 49388d5d3d topic_mentions: Fix restriction rule for @-topic mentions.
Now, the topic wildcard mention follows the following
rules:
* If the topic has less than 15 participants , anyone
can use @ topic mentions.
* For more than 15, the org setting 'wildcard_mention_policy'
determines who can use @ topic mentions.

Earlier, topic wildcard mentions followed the same restriction
as stream wildcard mentions, which was incorrect.

Fixes part of #27700.
2023-11-23 12:52:25 -08:00
David Rosa 505a591179 help: Rename view-and-browse-images.md -> view-images-and-videos.md
- Updates filename and adds URL redirect.
- Crosslinks related articles.
2023-11-23 10:37:31 -08:00
Mateusz Mandera 3958743b33 corporate: Add prototype authentication system for self-hosters.
This makes it possible for a self-hosted realm administrator to
directly access a logged-page on the push notifications bouncer
service, enabling billing, support contacts, and other administrator
for enterprise customers to be managed without manual setup.
2023-11-22 17:03:47 -08:00
Mateusz Mandera 1ec0d5bd9d requests: Add SELF_HOSTING_MANAGEMENT_SUBDOMAIN. 2023-11-22 14:22:26 -08:00
Mateusz Mandera 8187d6b963 home: Remove redundant condition in get_billing_info.
has_billing_access already has the is_realm_owner check:

    @property
    def has_billing_access(self) -> bool:
        return self.is_realm_owner or self.is_billing_admin
2023-11-22 14:22:26 -08:00
Mateusz Mandera 8695ffba49 remote_server_post_analytics: Change RealmDataForAnalytics.uuid type.
pydantic allows us to use UUID4 as a more accurate type.
2023-11-22 14:22:26 -08:00
Sahil Batra 4f58733d82 events: Remove deactivated streams from subscriptions field.
We did not remove the objects for deactivated streams from
subscriptions field in apply_event. We need to do this because
we do not send "subscription/remove" events to subscribers
when deactivating streams.
2023-11-21 23:58:45 -08:00
Sahil Batra 45e1b32447 users: Send user remove events on user deactivation.
Guests might lose access to deactivated users if the user
is not involved in any DM with guest. This commit adds
code to send "realm_user/remove" events for such cases.
2023-11-21 23:58:45 -08:00
Sahil Batra 3d96969398 avatar: Update avatar used for inaccessible users.
This commit adds a new avatar image which will be shown for
all inaccessible users.
2023-11-21 23:58:45 -08:00
Sahil Batra 32c15d67b5 users: Send user creation events when sending DMs.
We now send user creation events to recipient users
when sending DMs if recipients gain access to either
sender or other pariticpating users in the DM.
2023-11-21 23:58:45 -08:00