Commit Graph

788 Commits

Author SHA1 Message Date
Mateusz Mandera 96001b19fc update_analytics_count: Use a correct lock mechanism.
Adds a re-usable lockfile_nonblocking helper to context_managers.

Relying on naive `os.mkdir` is not enough especially now that the
successful operation of this command is necessary for push notifications
to work for many servers.

We can't use `lockfile` context manager from
`zerver.lib.context_managers`, because we want the custom behavior of
failing if the lock can't be acquired, instead of waiting.
That's because if an instance of this gets stuck, we don't want to start
queueing up more processes waiting forever whenever the cronjob runs
again and fail->exit is preferrable instead.

(cherry picked from commit f61ed58c8f)
2024-03-07 15:01:15 -08:00
Alex Vandiver aee36b2b49 tests: Clear in-memory Client caches before testing query counts.
This makes counts more apples-to-apples comparable when run
back-to-back.

(cherry picked from commit a84de411a9)
2024-03-07 09:48:20 -08:00
Lauryn Menard 85bd8dd33d push-bouncer: Exclude LoggingCountStats with partial data.
LoggingCountStats with a daily duration and that are directly stored
on the RealmCount table (not via aggregation in process_count_stat),
can be in a state, after the hourly cron job to update analytics
counts, where the logged value will be live-updated later, because
the end time for the stat is still in the future.

As these logging counts are designed to be used on the self-hosted
installation for either debugging or rate limiting, sending these
partial/incomplete counts to the bouncer has low value.

(cherry picked from commit 40daf10a74)
2024-03-01 09:26:25 -08:00
Anders Kaseorg 2e4785f51c requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
(cherry picked from commit 93198a19ed)
2024-02-16 12:17:34 -08:00
Anders Kaseorg d07d773809 models: Extract zerver.models.scheduled_jobs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 26811c5049 models: Extract zerver.models.clients.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg 086df4a81e models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg a8b5a99281 models: Extract zerver.models.groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Anders Kaseorg ee85ac5433 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-05 10:32:54 -05:00
Sahil Batra 03323b0124 push_notifications: Enforce max user count on self managed plan.
We do not support sending push notifications for realms having
more than 10 users on self managed plan.
2023-12-15 11:03:42 -08:00
Aman Agrawal 34a1be80a4 model: Use same name for legacy plan across the app. 2023-12-14 08:22:25 -08:00
Tim Abbott 6308e07e53 billing: Standardize remote server plan type IDs.
This will likely save us at least one headache.
2023-12-13 16:40:44 -08:00
Lauryn Menard a897d68d93 support: Add active plan information to remote server activity.
Adds three columns to the remote server activity chart and updates
the chart key for the third of those columns.

The first is the plan name. If there are multiple plans with a
status under the live threshhold, then we send "See support view".

The second is the plan status. If there are multiple plans, then
we send "Multiple plans".

The third is the estimated annual revenue for the plan. Note that
for free trials, this will be calculated as if the plan was paid
for 12 months (so a full year).

If there is no plan for the server under the live threshold or at
all then "---" is inserted into the table row. Note that 100%
sponsored servers/realms would fall into this category.
2023-12-13 13:43:00 -08:00
Lauryn Menard 68d117e60a support: Add sponsorship request information to remote support view. 2023-12-13 10:28:11 -08:00
Lauryn Menard e8500fbdb0 support: Include sponsorship request for remote server support view.
Preparation for updating the sponsorship forms template to include
information about the latest sponsorship request if sponsorship is
pending.
2023-12-13 10:28:11 -08:00
Alex Vandiver 19575b46f7 remote_activity: Hide deactivated remote servers. 2023-12-12 20:36:03 -08:00
Alex Vandiver 6d7037e867 analytics: Show latest number in "Analytics users", not max. 2023-12-12 12:09:51 -08:00
Lauryn Menard 449714245e support: Exclude system bot realm from remote realm query. 2023-12-12 08:04:02 -08:00
Prakhar Pratyush c1daabd3c0 remote_server: Rename to 'send_server_data_to_push_bouncer'.
This commit renames 'send_analytics_to_push_bouncer'
to 'send_server_data_to_push_bouncer'.
2023-12-11 14:07:39 -08:00
Prakhar Pratyush d763fae9d0 remote_server: Eliminate separate realms-only code path.
Given that most of the use cases for realms-only code path would
really like to upload audit logs too, and the others would likely
produce a better user experience if they upoaded audit logs, we
should just have a single main code path here i.e.
'send_analytics_to_push_bouncer'.

We still only upload usage statistics according to documented
option, and only from the analytics cron job.

The error handling takes place in 'send_analytics_to_push_bouncer'
itself.
2023-12-11 14:07:39 -08:00
Lauryn Menard 7186f75b58 support: Add sponsorship/discount support forms to remote realms.
If there are remote realms attached to the remote server, adds
the forms to be able to update sponsorship requests, approve
sponsorship and update a discount.
2023-12-11 10:33:50 -08:00
Lauryn Menard c2a76ccb3e support: Clean up template variables for remote server support. 2023-12-11 10:33:50 -08:00
Lauryn Menard d079a13760 support: Use shared template for current plan details on support views.
Moves the section in support views for any current plan details
to a new template: `templates/analytics/current_plan_details.html`.

Also, updates the PlanData dataclass to have a boolean that checks
if the current plan tier is the self-hosted legacy plan.
2023-12-06 11:01:26 -08:00
Anders Kaseorg 223b626256 python: Use urlsplit instead of urlparse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg 3853fa875a python: Consistently use from…import for urllib.parse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Lauryn Menard ba3279e959 support: Add remote realm stub to remote server information.
Adds a small section for any remote realms attached to the remote
server in the support view.
2023-12-05 12:48:42 -08:00
Lauryn Menard cfd61669e0 support: Add plan type to remote server information.
Updates `get_plan_type_string` for RemoteZulipServer plan types and
capitalizes the strings used for Realm plan types.

Also changes the string for Realm.PLAN_TYPE_STANDARD_FREE to be
"Standard free" instead of "open source" as that is used for any
100% sponsored organization, which is not restricted to open-source
projects.
2023-12-05 12:48:42 -08:00
Lauryn Menard 6c5b419267 support: Add downgrade current plan actions to remote servers. 2023-12-05 12:48:42 -08:00
Lauryn Menard f5ab9419d2 support: Update billing modality for remote server current plan. 2023-12-05 12:48:42 -08:00
Anders Kaseorg 8a7916f21a python: Consistently use from…import for datetime.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 12:01:18 -08:00
Tim Abbott 95b39bc8bf analytics: More fixes to zilencer urls import logic. 2023-12-05 10:52:00 -08:00
Tim Abbott 28d736d63b analytics: Don't import zilencer-specific code unconditionally.
This fixes the production server not working when zilencer is not
enabled.
2023-12-05 10:36:35 -08:00
Prakhar Pratyush 14b1539647 remote_activity: Add 'guest users' and 'non guest users' count column.
This commit adds two columns named 'Guest users' and
'Non guest users' to respresent count of such users.

We query 'RemoteRealmAuditLog' to get the data.
2023-12-05 10:04:41 -08:00
Lauryn Menard dc0b8bcb6b analytics: Update query for remote_push_devices in remote activity.
Updates query to count the distinct pairs of user_id and user_uuid
on the RemotePushDeviceToken table.
2023-12-04 11:50:18 -08:00
Lauryn Menard 69f6d3dcb1 support: Approve sponsorship for remote server customer.
Adds ability to approve a sponsorship request for a customer
attached to a remote server via the remote server support view.
2023-12-03 14:27:07 -08:00
Lauryn Menard 2218e667a6 support: Update remote server customer sponsorship pending status.
Adds ability to update the sponsorship pending status of a customer
attached to a remote server via the remote server support view.
2023-12-03 14:27:07 -08:00
Lauryn Menard 4250e9c1c9 support: Update remote server customer default discount.
Adds ability to update the default discount for a customer attached
to a remote server via the remote server support view.
2023-12-03 14:27:07 -08:00
Lauryn Menard 38eaa4b958 support: Add plan data to remote server support view context. 2023-12-03 14:27:07 -08:00
Lauryn Menard 1d685bd198 support: Add success message to remote servers support view context. 2023-12-03 14:27:07 -08:00
Lauryn Menard 6d66248d3d support: Get plan data via BillingSession for support views. 2023-12-03 14:27:07 -08:00
Lauryn Menard 8d992405a6 analytics: Get customer discount for support views. 2023-12-03 14:27:07 -08:00
Lauryn Menard 5135acd9e3 support: Use process_support_view_request for plan modifications.
Updates the support view to use process_support_view_request to
process upgrade or downgrade modifications currently implemented
for active plans.
2023-12-03 14:27:07 -08:00
Lauryn Menard 4fb564026d corporate: Move void_all_open_invoices to BillingSession. 2023-12-03 14:27:07 -08:00
Lauryn Menard 97d33a4363 support: Update billing modality via process_support_view_request. 2023-12-03 14:27:07 -08:00
Lauryn Menard 5d25cab42b analytics: Create process_support_view_request BillingSession method.
Creates a process_support_view_request method for BillingSession
to process the various support requests that relate to the billing
system.

Moves approve_realm_sponsorship, update_realm_sponsorship_status,
and attach_discount_to_realm to this new BillingSession method.

Adds a new abstract property to BillingSession to have a string
value, billing_entity_display_name, to use for support messages
sent when these requests are processed.
2023-12-03 14:27:07 -08:00
Lauryn Menard 0679bc044a corporate: Make references to billing modality consistent.
The "send_invoice" and "charge_automatically" strings used by stripe
for the `collection_method` are referred to both as the "billing
method" and "billing modality" in the billing code.

Because we send this as data to stripe as either `collection_method`
or `billing_modality`, renames any references that are any form of
"billing method".
2023-12-03 14:27:07 -08:00
Mateusz Mandera c9b0602320 tests: Create default RemoteRealms in populate_db.
This default setup will be more realistic, matching the ordinary
conditions for a modern server.
Especially needed as we add bouncer code that will expect to have
RemoteRealm entries for realm_uuid values for which it receives
requests.
2023-12-03 09:51:45 -08:00
Tim Abbott 610338d192 billing: Add BILLING_SCHEDULE_ prefix to values. 2023-11-29 23:32:56 -08:00
Tim Abbott 5d6b635efe billing: Use better variable names for plan tiers.
The existing values didn't have our standard type-prefixing naming
scheme.

Add some extra unused placeholder values while we're at it.
2023-11-29 23:32:56 -08:00
Tim Abbott 7db15176f3 push bouncer: Submit basic metadata unconditionally.
These metadata are essentially all publicily available anyway, and
making uploading them unconditional will simplify some things.

The documentation is not quite accurate in that it claims the server
will upload some metadata that is not actually uploaded yet (but will
by soon). This seems harmless.
2023-11-29 14:45:53 -08:00