To estimate the annual recurring revenue for remote server and
remote realm CustomerPlans, we prefetch the current license
ledger as part of the CustomerPlan query.
Also adds a select related to the remote realm audit log query
so that we don't go to the database for the remote realm ID.
With the test added in the previous commit, the query count for
the remote activity view goes from 27 to 11, as we are no longer
hitting the database multiple times for every current plan or
for every remote realm with audit log data.
Refactors get_customer_plan_renewal_amount so that a license
ledger is always passed and make_end_of_cycle_updates_if_needed
does not need to be called.
Creates multiple remote servers and remote realms with active
plan data and audit logs.
Shows multiple small queries to get revenue data from plan license
ledger, as well as remote server and remote realm IDs.
Combines the IDs for the remote realm and remote server in the
first column.
Moves the Realm name to the second column.
For the host/hostname column, displays the remote realm host if
it is a remote realm row. Otherwise, the remote server hostname
is displayed.
Instead of showing the next invoice date for the plan, show the
date for the next billing cycle start (e.g. the next plan renewal
charge date), except for plans currently on a free trial.
For plans on a free trial, the next plan renewal date will be when
the free trial ends, which is stored as the next invoice date on
the plan.
Instead of querying the database for every remote server and realm
in the remote activity chart, we now get the server and realm data
for the installation in two queries.
Adds columns for remote realm ID, name and organization type. If
a remote server has remote realms attached that are not marked
as deactivated by the remote server, then there will be a row in
the chart for each remote realm (which duplicates some remote
server data).
Updates the plan data, revenue and user counts to be for the realm
if present and otherwise for the server.
Updates the user counts to be total users and guest users, instead
of non guest and guest users.
The total row for mobile data (users and pushes forwarded) sums
each remote server's data once, so while the column duplicates
data, the total row should be an accurate total for the installation.
Adds 5 queries to the remote activity page test. One is for the
additional query for the remote realm plans. The other four are
getting the remote realm object and then the user count data for
the two remote realms in the test.
This will help with updating this function for updating/changing
the columns in the chart as only the constant needs to be updated
and not every instance the integer is used in the loops over the
rows of data.
Merges the two charts remaining to have just one chart for the
realm activity view.
Removes the columns for different clients and adds a column to
show/sort by the user type (human or bot type).
Deletes templates/analytics/activity.html because it is no
longer used for any activity pages/views.
Removes the summary tab/chart from the user activity view.
Moves get_user_activity_summary from common_activity.py to
realm_activity.py since it's no longer used in the user
activity view.
Updates the title of the page to incluce the realm name since
an email address may be used for multiple user accounts.
Adds a support action for updating the minimum licenses on a
customer object once a default discount has also been set.
In the case that the current billing entity has a current active
plan or a scheduled upgrade to a new plan, then the minimum
licenses will not be updated.
Previously, the message string was sent as a success response to
the context, which could have been confusing or ignored when shown
in the support admin view.
Since the latest end time for reported active users on a remote
server is not used to determine if a remote server has current
data for billing purposes, we instead display the last audit log
update datetime for the remote server object.
Removes the note in the remote activity key about the Zulip
version.
Also adds a sum in the total row for mobile push notifications
sent.
Changes the emoji that highlights when a remote realm or server has
an active discount.
Adds a remote server UUID field to the remote server information.
Moves the 'has remote realms' field to be after the Zulip version
information, and adds an extra break there to visually highlight
that field as well as the following 'max monthly messages' field.
Creates some reusable helper functions and adds remote realms to
the search results that are checked, which gives coverage for the
remote realm user counts in the support view.
Also fixes formatting for per license price and moves the billing
schedule to be above this line so that it's clearer the per license
price is based on the billing schedule.
Adds three columns to the remote server activity chart and updates
the chart key for the third of those columns.
The first is the plan name. If there are multiple plans with a
status under the live threshhold, then we send "See support view".
The second is the plan status. If there are multiple plans, then
we send "Multiple plans".
The third is the estimated annual revenue for the plan. Note that
for free trials, this will be calculated as if the plan was paid
for 12 months (so a full year).
If there is no plan for the server under the live threshold or at
all then "---" is inserted into the table row. Note that 100%
sponsored servers/realms would fall into this category.
Given that most of the use cases for realms-only code path would
really like to upload audit logs too, and the others would likely
produce a better user experience if they upoaded audit logs, we
should just have a single main code path here i.e.
'send_analytics_to_push_bouncer'.
We still only upload usage statistics according to documented
option, and only from the analytics cron job.
The error handling takes place in 'send_analytics_to_push_bouncer'
itself.
If there are remote realms attached to the remote server, adds
the forms to be able to update sponsorship requests, approve
sponsorship and update a discount.
Moves the section in support views for any current plan details
to a new template: `templates/analytics/current_plan_details.html`.
Also, updates the PlanData dataclass to have a boolean that checks
if the current plan tier is the self-hosted legacy plan.
Updates `get_plan_type_string` for RemoteZulipServer plan types and
capitalizes the strings used for Realm plan types.
Also changes the string for Realm.PLAN_TYPE_STANDARD_FREE to be
"Standard free" instead of "open source" as that is used for any
100% sponsored organization, which is not restricted to open-source
projects.
This commit adds two columns named 'Guest users' and
'Non guest users' to respresent count of such users.
We query 'RemoteRealmAuditLog' to get the data.
Creates a process_support_view_request method for BillingSession
to process the various support requests that relate to the billing
system.
Moves approve_realm_sponsorship, update_realm_sponsorship_status,
and attach_discount_to_realm to this new BillingSession method.
Adds a new abstract property to BillingSession to have a string
value, billing_entity_display_name, to use for support messages
sent when these requests are processed.
The "send_invoice" and "charge_automatically" strings used by stripe
for the `collection_method` are referred to both as the "billing
method" and "billing modality" in the billing code.
Because we send this as data to stripe as either `collection_method`
or `billing_modality`, renames any references that are any form of
"billing method".
This default setup will be more realistic, matching the ordinary
conditions for a modern server.
Especially needed as we add bouncer code that will expect to have
RemoteRealm entries for realm_uuid values for which it receives
requests.
These metadata are essentially all publicily available anyway, and
making uploading them unconditional will simplify some things.
The documentation is not quite accurate in that it claims the server
will upload some metadata that is not actually uploaded yet (but will
by soon). This seems harmless.
Adds a column to the remote servers activity page for a 7 day
count of forwarded mobile push notifications from
RemoteInstallationCount for each server, which will be `None`
if there is no data for the remote server with that ID.
Refactors get_page helper function so that the updates to the
query data for each row is done in the function that processes
the request.
Adds columns to the remote installation page for both the support
and analytics links.
Adds `analytics/views/remote_activity.py` to the files without
100% backend test coverage.
Moves and generalizes `switch_realm_from_standard_to_plus_plan`
in stripe.py to be a more general function for changing a
CustomerPlan to a new and valid tier, `do_change_plan_to_new_tier`.
Adds a helper function with the previous function name to be used
for the support view and management command for changing a realm
from the Standard plan tier to the Plus plan tier.
This commit moves a major portion of the 'update_plan`
view to a new shared 'BillingSession.do_update_plan' method.
This refactoring will help in minimizing duplicate code
while supporting both realm and remote_server customers.
This reduces the giant load spike at 5 minute past the hour, when all
remote servers currently attempt to submit their records.
We do not wish to slew over a full hour, because we want to ensure
that we do not hold the lock when the next hour's analytics runs. It
is also not necessary to have that much variation; 10 minutes is
picked as an arbitrary "long enough" time to spread requests over.
Due to a bug[^1] in Django 4.2, fixed in 4.2.6, queries using
`__isnull` added an unnecessary cast. This cast was _also_ used in
`WHERE` clauses for partial indexes. This means that partial indexes
created before Zulip was using Django 4.2 (i.e. before Zulip Server
7.0 or 2c20028aa4) will not be used when the server is using Django
4.2.0 through 4.2.5 -- and, conversely, that indexes created while
Zulip had those versions of Django (i.e. Zulip Server 7.0 through 7.4
or 7807bff526) will not be used later.
We re-create the indexes, to ensure that users that installed Zulip
after Zulip Server 7.0 / 2c20028aa4 and before Zulip Server 7.5 /
7807bff526 have indexes which can be used by current Django. This
is useless work for some installations, but most analytics tables are
not large enough for this to take significant time.
[^1]: https://code.djangoproject.com/ticket/34840
Moves the 'make_end_of_cycle_updates_if_needed' function to
the 'BillingSession' abstract class.
This refactoring will help in minimizing duplicate code while
supporting both realm and remote_server customers.
Since the function is called from our main daily billing cron job
as well, we have changed 'RealmBillingSession' to accept 'user=None'
(our convention for automated system jobs).
This is a CountStat for tracking how many mobile notifications the
server requested.
1. On a self-hosted server, that means requesting from the push bouncer.
2. On a server that's its own push bouncer, that's just the number
directly sent.
This number has room for inaccuracy due to incrementing by the number of
user devices on a self-hosted server, as it doesn't account for errors
that may occur in the GCM/APNs low-level sending codepaths on the bouncer.
Also tests that a server that's its own push bouncer correctly
increments its mobile_pushes_sent::day CountStat, by basing it on the
values returned from the send_apple/android_push_notification functions
which tell us the actual number of successfully sent notifications.
Since the return values of send_..._push_notification are now
used in those codepaths, we need to tweak our mocks in some unrelated
tests to set up some return value to avoid errors.
This calculates the largest amount of messages sent within a month for
the last 3 months. The query is targeted for the specific use-case in
this function - for finding the count for a specific server. For
calculating this in bulk for a large number of remote server an
adapted, bulk query will be needed - rather than running this one in a
loop, which would likely be very inefficient.
Moves `update_billing_method_of_current_plan` to the BillingSession
abstract class.
Adds a helper function for support views for the realm case:
`update_realm_billing_method`.
Moves `update_sponsorship_status` to BillingSession abstract class
as `update_customer_sponsorship_status`.
Updates the support views to have a helper for updating this on a
realm: `update_realm_sponsorship_status`.
Lauryn Menard
Alya Abbott
Karl Stolley
Anders Kaseorg
Sahil Batra
Aman Agrawal
Tim Abbott
Alex Vandiver
Prakhar Pratyush
Mateusz Mandera