Commit Graph

797 Commits

Author SHA1 Message Date
Mateusz Mandera 0cbbb98cc2 zilencer: Have server send realm_uuid to remaining bouncer endpoints.
Requests to these endpoint are about a specified user, and therefore
also have a notion of the RemoteRealm for these requests. Until now
these endpoints weren't getting the realm_uuid value, because it wasn't
used - but now it is needed for updating .last_request_datetime on the
RemoteRealm.

(cherry picked from commit e2291b7656)
2024-01-15 16:51:09 -08:00
Mateusz Mandera 3a4d45022f zilencer: Add last_request_datetime to RemoteRealm + RemoteZulipServer.
For the RemoteRealm case, we can only set this in endpoints where the
remote server sends us the realm_uuid. So we're missing that for the
endpoints:

- remotes/push/unregister and remotes/push/unregister/all
- remotes/push/test_notification

This should be added in a follow-up commit.

(cherry picked from commit 3dca333b8d)
2024-01-15 16:51:09 -08:00
Mateusz Mandera 7a4a8f55c0 zilencer: Set .remote_realm for existing RemotePushDeviceToken.
Old RemotePushDeviceTokens were created without this attribute. But when
processing a notification, if we have remote_realm, we can take the
opportunity to to set this for all the registrations for this user.

(cherry picked from commit 82ee93d967)
2024-01-15 16:51:09 -08:00
Anders Kaseorg 25592eba22 models: Extract zerver.models.alert_words.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 26811c5049 models: Extract zerver.models.clients.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 776a0eeae8 models: Extract zerver.models.streams.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 56a09f426b models: Extract zerver.models.recipients.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 086df4a81e models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg ee85ac5433 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Alex Vandiver 48ee7506ba zilencer: Log the can_push value that we returned. 2023-12-15 11:03:42 -08:00
Sahil Batra 03323b0124 push_notifications: Enforce max user count on self managed plan.
We do not support sending push notifications for realms having
more than 10 users on self managed plan.
2023-12-15 11:03:42 -08:00
Tim Abbott 10862451ef billing: Refactor code to compute push status.
This moves the function which computes can_push and
expected_end_timestamp outside RemoteRealmBillingSession
because we might use this function for RemoteZulipServer
as well and also renames it.
2023-12-15 11:03:42 -08:00
Mateusz Mandera fb5137f8b5 zilencer: Handle deleted realms nicely at server/analytics. 2023-12-15 09:18:26 -08:00
Mateusz Mandera d351353126 zilencer: Fix server/register failing to check if server is deactivated.
A deactivated server shouldn't be updated like that - we want to return
an error.
2023-12-15 08:06:58 -08:00
Aman Agrawal 11dff6b4d8 zilencer: Add tests for migrating customer from server to realm. 2023-12-14 14:31:28 -08:00
Aman Agrawal b4e92ecc48 zilencer: Don't consider deactivated realms to transfer server plan. 2023-12-14 14:31:28 -08:00
Tim Abbott 1757b88760 billing: Offer release announcement subscriptions.
Also avoid prompting for full name time more than once.
Adds TOS version field to Remote server user.

Co-authored-by: Karl Stolley <karl@zulip.com>
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
2023-12-14 10:51:16 -08:00
Alex Vandiver 4ef93de128 zilencer: Set the remote server as the user in Sentry. 2023-12-14 09:46:21 -08:00
Alex Vandiver 863c1c28f7 zilencer: Log why the server got a 401. 2023-12-14 09:46:21 -08:00
Tim Abbott 036b1156f2 zilencer: Avoid using stale last_audit_log_update. 2023-12-14 09:26:45 -08:00
Mateusz Mandera 651590c49a remote_billing: Store acting users in remote user audit logs. 2023-12-14 08:11:04 -08:00
Aman Agrawal 10ef308848 migrate_server_to_realm: Reset server tier during migration. 2023-12-13 22:29:30 -08:00
Tim Abbott bfa3d144eb zilencer: Don't try to migrate ended plans. 2023-12-13 17:55:00 -08:00
Tim Abbott b8deb44a05 zilencer: Fix mismigrated plan type assertion. 2023-12-13 17:55:00 -08:00
Tim Abbott 6308e07e53 billing: Standardize remote server plan type IDs.
This will likely save us at least one headache.
2023-12-13 16:40:44 -08:00
Aman Agrawal 35b644c564 populate_billing_realms: Don't create plan for sponsored remote realm. 2023-12-13 08:13:33 -08:00
Aman Agrawal 63f4fc51de server_deactivate: Show error message for server on active plan. 2023-12-12 21:00:42 -08:00
Aman Agrawal e9ce69aaf6 populate_billing_realms: Call sessions with billing_user. 2023-12-12 17:31:41 -08:00
Prakhar Pratyush 4ad03804cf stripe: Add `sync_license_ledger...` for RemoteServerBillingSession.
This commit implements the 'sync_license_ledger_if_needed' method
for RemoteServerBillingSession.
2023-12-12 12:15:32 -08:00
Prakhar Pratyush 88fe0a7561 zilencer: Update last_audit_log_update before `sync_license_ledger..`.
We need to update 'last_audit_log_update' before calling the
'sync_license_ledger_if_needed' method to avoid 'MissingDataError'
due to 'has_stale_audit_log' being True.

Also, we made the code block that creates audit logs,
updates 'last_audit_log_update', and syncs LicenseLedger in
an atomic operation.

This helps to rely on 'last_audit_log_update' to assume
'RemoteRealmAuditLog' and 'LicenseLedger' are up-to-date.
2023-12-12 12:15:31 -08:00
Mateusz Mandera c1988a14a7 zilencer: Return can_push info at the push/notify endpoint.
This provides the remote server this information to refresh it on its
Realm attributes whenever it sends a push notification.

Fixes #27483.
2023-12-12 08:06:12 -08:00
Vector73 2e71ec78e3 settings: Add "Automatically follow topics where I'm mentioned" setting.
Fixes: #26795
2023-12-11 23:26:11 -08:00
Mateusz Mandera b82ea179ac zilencer: Have push/notify endpoint return registrations to delete.
- The server sends the list of registrations it believes to have with
  the bouncer.
- The bouncer includes in the response the registrations that it doesn't
  actually have and therefore the server should delete.
2023-12-11 18:36:23 -08:00
Aman Agrawal 23d712391e post_analytics: Migrate plan from server to realm after upgrade. 2023-12-11 13:23:49 -08:00
Aman Agrawal b4e4ca14d5 models: Store `is_system_bot_realm` information for `RemoteRealm`.
This will help us filter out system bot realm and control
feature access to it.
2023-12-11 13:23:49 -08:00
Aman Agrawal df23701475 stripe: Rename add_server_to_legacy_plan.
This function will not also be used to migrate remote realms
to legacy plan.
2023-12-11 13:23:49 -08:00
Mateusz Mandera c800951966 remote_billing: Add some useful fields to Remote...User models.
These are useful for auditing and follow what we have for UserProfile.
And is_active will be used in the future when we add user deactivation.
2023-12-11 09:39:24 -08:00
Aman Agrawal ac8d5a5f0b remote_billing_page: Show error page for registration mismatch.
When a self-hosted Zulip server does a data export and then import
process into a different hosting environment (i.e. not sharing the
RemoteZulipServer with the original, we'll have various things that
fail where we look up the RemoteRealm by UUID and find it but the
RemoteZulipServer it is associated with is the wrong one.

Right now, we ask user to contact support via an error page but
might develop UI to help user do the migration directly.
2023-12-10 19:33:48 -08:00
Mateusz Mandera 423aebf98e remote_billing: Implement confirmation flow for RemoteRealm auth.
The way the flow goes now is this:
1. The user initiaties login via "Billing" in the gear menu.
2. That takes them to `/self-hosted-billing/` (possibly with a
   `next_page` param if we use that for some gear menu options).
3. The server queries the bouncer to give the user a link with a signed
   access token.
4. The user is redirected to that link (on `selfhosting.zulipchat.com`).
Now we have two cases, either the user is logging in for the first time
and already did in the past.
If this is the first time, we have:
5. The user is asked to fill in their email in a form that's shown,
   pre-filled with the value provided inside the signed access token.
   They POST this to the next endpoint.
6. The next endpoint sends a confirmation email to that address and asks
   the user to go check their email.
7. The user clicks the link in their email is taken to the
   from_confirmation endpoint.
8. Their initial RemoteBillingUser is created, a new signed link like in
   (3) is generated and they're transparently taken back to (4),
   where now that they have a RemoteBillingUser, they're handled
   just like a user who already logged in before:
If the user already logged in before, they go straight here:
9. "Confirm login" page - they're shown their information (email and
   full_name), can update
   their full name in the form if they want. They also accept ToS here
   if necessary. They POST this form back to
   the endpoint and finally have a logged in session.
10. They're redirected to billing (or `next_page`) now that they have
    access.
2023-12-10 16:15:28 -08:00
Aman Agrawal a1fa969693 populate_billing_realms: Add temporary fix for MissingDataError.
We need to add proper logs here, which can probably be copied from
test_current_count_for_billed_licenses.
2023-12-09 12:50:05 -08:00
Anders Kaseorg f86becfc94 remote_server: Send API feature level along with Zulip version.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-09 12:01:22 -08:00
Anders Kaseorg 0400614a48 remote_server: Serialize analytics requests with Pydantic.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-09 12:01:22 -08:00
Mateusz Mandera abdfdeffe4 remote_billing: Implement confirmation flow for legacy servers.
For the last form (with Full Name and ToS consent field), this pretty
shamelessly re-uses and directly renders the
corporate/remote_realm_billing_finalize_login_confirmation.html
template. That's probably good in terms of re-use, but calls for a
clean-up commit that will generalize the name of this template and the
classes/ids in the HTML.
2023-12-08 23:49:10 -08:00
Tim Abbott f355b76024 zilencer: Only accept SYNCED_BILLING_EVENTS.
I expect we would ignore extra events anyway, but this just makes it
easier to reason about how the system works.
2023-12-08 23:12:29 -08:00
Prakhar Pratyush bf4fdbff12 stripe: Update LicenseLedger for remote realms.
When a remote server uploads statistics, we update the
LicenseLedger using the audit logs uploaded.

We iterate over the RemoteRealmAuditlog data for the concerned
realm starting from the event_time of the last LicenseLedger
created for that customer and update the ledger based on each event.
2023-12-08 12:58:21 -08:00
Prakhar Pratyush ed9b0d330d stripe: Raise 'MissingDataError' while fetching license count.
If the RemoteRealmAuditLog has stale data, it means the server
stopped or never uploaded data. We raise MissingDataError in such
cases when a user action led to calculating licenses count from
stale data.
2023-12-08 12:58:21 -08:00
Prakhar Pratyush 40621478cb zilencer: Add `get_remote_realm_guest_and_non_guest_count`.
We add a 'get_remote_realm_guest_and_non_guest_count'
function that queries 'RemoteRealmAuditLog' to get
the guest and non_guest count for that remote_realm.

This function is used in 'RemoteRealmBillingSession'
to calculate the current count of billed licenses.
2023-12-08 12:58:21 -08:00
Aman Agrawal c49fc8c6fe populate_billing_realms: Delete existing remote realms.
Running script multiple times created multiple `RemoteRealm`
objects resulting in processing error.
2023-12-08 11:24:15 -08:00
Aman Agrawal 9594e635cd populate_billing_realms: Simulate 2 months free trial.
Makes the billing page for realms on free trial more realistic.
2023-12-08 09:21:18 -08:00
Aman Agrawal 1a60697bd5 populate_billing_realms: Use correct plan tier for sponsored realms.
In cloud:

Sponsored organizations have plan_type=STANDARD_FREE, don't have
a CustomerPlan object and thus no tier value.

With self-hosting:
Sponsored organizations have a CustomerPlan object with tier
TIER_SELF_HOSTED_COMMUNITY and a plan_type of PLAN_TYPE_COMMUNITY.
2023-12-06 12:04:10 -08:00