This commit moves a major portion of the 'update_plan`
view to a new shared 'BillingSession.do_update_plan' method.
This refactoring will help in minimizing duplicate code
while supporting both realm and remote_server customers.
This reduces the giant load spike at 5 minute past the hour, when all
remote servers currently attempt to submit their records.
We do not wish to slew over a full hour, because we want to ensure
that we do not hold the lock when the next hour's analytics runs. It
is also not necessary to have that much variation; 10 minutes is
picked as an arbitrary "long enough" time to spread requests over.
Due to a bug[^1] in Django 4.2, fixed in 4.2.6, queries using
`__isnull` added an unnecessary cast. This cast was _also_ used in
`WHERE` clauses for partial indexes. This means that partial indexes
created before Zulip was using Django 4.2 (i.e. before Zulip Server
7.0 or 2c20028aa4) will not be used when the server is using Django
4.2.0 through 4.2.5 -- and, conversely, that indexes created while
Zulip had those versions of Django (i.e. Zulip Server 7.0 through 7.4
or 7807bff526) will not be used later.
We re-create the indexes, to ensure that users that installed Zulip
after Zulip Server 7.0 / 2c20028aa4 and before Zulip Server 7.5 /
7807bff526 have indexes which can be used by current Django. This
is useless work for some installations, but most analytics tables are
not large enough for this to take significant time.
[^1]: https://code.djangoproject.com/ticket/34840
Moves the 'make_end_of_cycle_updates_if_needed' function to
the 'BillingSession' abstract class.
This refactoring will help in minimizing duplicate code while
supporting both realm and remote_server customers.
Since the function is called from our main daily billing cron job
as well, we have changed 'RealmBillingSession' to accept 'user=None'
(our convention for automated system jobs).
This is a CountStat for tracking how many mobile notifications the
server requested.
1. On a self-hosted server, that means requesting from the push bouncer.
2. On a server that's its own push bouncer, that's just the number
directly sent.
This number has room for inaccuracy due to incrementing by the number of
user devices on a self-hosted server, as it doesn't account for errors
that may occur in the GCM/APNs low-level sending codepaths on the bouncer.
Also tests that a server that's its own push bouncer correctly
increments its mobile_pushes_sent::day CountStat, by basing it on the
values returned from the send_apple/android_push_notification functions
which tell us the actual number of successfully sent notifications.
Since the return values of send_..._push_notification are now
used in those codepaths, we need to tweak our mocks in some unrelated
tests to set up some return value to avoid errors.
This calculates the largest amount of messages sent within a month for
the last 3 months. The query is targeted for the specific use-case in
this function - for finding the count for a specific server. For
calculating this in bulk for a large number of remote server an
adapted, bulk query will be needed - rather than running this one in a
loop, which would likely be very inefficient.
Moves `update_billing_method_of_current_plan` to the BillingSession
abstract class.
Adds a helper function for support views for the realm case:
`update_realm_billing_method`.
Moves `update_sponsorship_status` to BillingSession abstract class
as `update_customer_sponsorship_status`.
Updates the support views to have a helper for updating this on a
realm: `update_realm_sponsorship_status`.
Makes `approve_sponshorship` an abstract method in BillingSession
abstract base class and moves the implementation for realms to the
RealmBillingSession child class.
Adds `approve_realm_sponsorship` helper function that's used in
the support view and initiates the billing session.
This moves the logic for `attach_realm_discount`, which is used in
the support view, to be in the BillingSession class.
Updates the function name to be `attach_discount_to_customer` so
that the context is generalized vs realm specific.
Updates RealmBillingSession implementation to account for actions
that are initiated by a support admin user.
Also moves the helper function `get_discount_for_realm` that is
only used in support views to `corporate/lib/support.py`.
We are about to add support for having RemoteZulipServer here, which is
a zilencer, not zerver, object. So let's rename this argument to
something more appropriately general.
An assert is appropriate here to ensure that some future additions of
other frequencies don't make this if/else logic wrong without explicitly
failing.
This commit moves constants for system group names to a new
"SystemGroups" class so that we can use these group names
in multiple classes in models.py without worrying about the
order of defining them.
Moves two functions in corporate/lib/stripe.py that are used to
get data for the main installation activity analytics page to a
separate file: corporate/lib/analytics.py.
Also, updates these functions for the possibility of realm being
None for a Customer object.
Use analytics_realmcount to improve query runtime. There are two
known difference with previous query:
- Messages that are deleted after hourly stats are aggregated
are not decremented in new query.
- Messages sent since the hourly aggregation last ran are not counted.
The former name is kind of misleading - this function is for the remote
server to send analytics to the push bouncer. Under our usual
terminology, a "remote server" is a self-hosted Zulip server. So data is
sent FROM not TO a remote server.
As a preliminary step for including graphs of StreamCount data in our
analytics pages, add API support for fetching the chart data.
Care is taken to limit access to streams that the current user has
access to, which isn't necessary in similar views where the current
user is a server administrator by assumption.
Fixes part of #19653.
Co-authored-by: Tim Abbott <tabbott@zulip.com>
Moves the "Remote Zulip servers" tab in the "/activity" page for
an installation to a separate page, "/activity/remote".
Prototype for moving other tabs in "/activity" to separate pages.
Using `COUNT(*) FILTER (WHERE ...)` allows getting counts of different
subsets with only one giant join. This makes the query significantly
more performant.
_default_manager is the same as objects on most of our models. But
when a model class is stored in a variable, the type system doesn’t
know which model the variable is referring to, so it can’t know that
objects even exists (Django doesn’t add it if the user added a custom
manager of a different name). django-stubs used to incorrectly assume
it exists unconditionally, but it no longer does.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Prakhar Pratyush
Alex Vandiver
Mateusz Mandera
Tim Abbott
Lauryn Menard
Sahil Batra
Alya Abbott
Lakshay Mittal
Anders Kaseorg
Lauryn Menard
Steve Howell