Commit Graph

367 Commits

Author SHA1 Message Date
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
roanster007 02d0566dc5 refactor: Rename `Huddle` Django model class to `DirectMessageGroup`.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".

Fixes part of #28640.
2024-07-07 21:31:30 -07:00
Alex Vandiver 0a296b2a6e upload: Start storing content-type for new uploads. 2024-07-07 14:40:07 -07:00
Mateusz Mandera 00b8cce50e push_notifs: Rename PushDeviceToken.GCM to FCM. 2024-06-17 18:22:59 -07:00
Alex Vandiver 09e9c75ec6 analytics: Remove `active_users` and `active_users_log` metrics.
Both of these are inaccurate, not currently used anywhere, and have
been superseded by the `active_users_audit` metric.
2024-06-03 12:35:35 -07:00
Alex Vandiver 0100440a86 analytics: Make active_users_audit into a RealmCount.
With `realm_active_humans` no longer dependent on the per-user rows,
there is no reason to preserve them -- any measure of "was a user
active" should look directly at the much richer RealmAuditLog.  This
removes the bulk of the UserCount table, since the remaining rows all
require user interaction of some sort to produce rows.
2024-06-03 12:35:35 -07:00
Alex Vandiver 195defb031 analytics: Rewrite realm_active_humans::day query.
This makes it no longer dependent on active_users_audit:is_bot:day,
which subsequent commits will make a RealmCount, not UserCount, query.
This folds the same behaviour of `active_users_audit` directly into
the query; however, only running over active users, using the index
from the earlier commit, and using the new `DISTINCT ON` formulation
make this a fast query compared to `active_users_audit:is_bot:day` +
the old `realm_active_humans::day`.
2024-06-03 12:35:35 -07:00
Alex Vandiver 7ad967ebc7 test_counts: Create audit log entries when creating users. 2024-06-03 12:35:35 -07:00
Lauryn Menard 5892e48ba4 analytics: Update "Messages sent by client" chart for Flutter app.
Updates the labels in the "Messages sent by client" analytics chart
for the user-agent/client names for the Flutter mobile app, which
can be "ZulipFlutter" or "ZulipMobile/flutter".

Fixes #28220.
2024-05-28 10:18:40 -07:00
Sahil Batra 7b42c802b1 invites: Add include_realm_default_subscriptions parameter.
This commit adds include_realm_default_subscriptions parameter
to the invite endpoints and the corresponding field in
PreregistrationUser and MultiuseInvite objects. This field will
be used to subscribe the new users to the default streams at the
time of account creation and not to the streams that were default
when sending the invite.
2024-05-14 14:20:07 -07:00
Mateusz Mandera 9406bfbc0a analytics: Store realm disk space used as a CountStat.
Fixes #29632.

The issue description explains this well:

We currently recalculate `currently_used_upload_space_bytes` every file
upload, by dint of calling `flush_used_upload_space_cache`  on
save/delete, and then immediately calling
`user_profile.realm.currently_used_upload_space_bytes()` in
`notify_attachment_update`.  Since this walks the Attachments table,
recalculating this can take seconds in large realms.

Switch this to using a CountStat, so we don't need to walk significant
chunks of the Attachment table when we upload an attachment.  This will
also give us a historical daily graph of usage.
2024-05-09 10:54:44 -07:00
Mahhheshh 1198785c62 analytics: Improve do_increment_logging_stat performance.
The previous implementation using Django's `get_or_create` for
`do_increment_logging_stat` involved two separate database queries,
potentially leading to race conditions.

Use an `ON CONFLICT ... DO UPDATE` (aka "upsert") query, which
eliminates race conditions and improves performance.  This is mildly
complicated due to the different unique indexes across the various
tables, and the need for bug-for-bug compatibility with the previous
implementation.

Fixes #28947.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2024-05-06 16:34:01 -07:00
Mahhheshh 218c7ae8cd test_counts: Add test to do_incremental_logging_stat.
Adds an test, to count the number of database queries made by
`do_incremental_logging_stat` function.
2024-05-06 16:34:01 -07:00
Alex Vandiver 9dfaa83aa8 invites: Remove invites worker, make confirmation object in-process.
The "invites" worker exists to do two things -- make a Confirmation
object, and send the outgoing email.  Making the Confirmation object
in a background process from where the PreregistrationUser is created
temporarily leaves the PreregistrationUser in invalid state, and
results in 500's, and the user not immediately seeing the sent
invitation.  That the "invites" worker also wants to create the
Confirmation object means that "resending" an invite invalidates the
URL in the previous email, which can be confusing to the user.

Moving the Confirmation creation to the same transaction solves both
of these issues, and leaves the "invites" worker with nothing to do
but send the email; as such, we remove it entirely, and use the
existing "email_senders" worker to send the invites.  The volume of
invites is small enough that this will not affect other uses of that
worker.

Fixes: #21306
Fixes: #24275
2024-05-02 14:23:04 -07:00
Alex Vandiver d863aa56de invites: Lock the realm when determining invitation counts.
This prevents users from hammering the invitation endpoint, causing
races, and inviting more users than they should otherwise be allowed
to.

Doing this requires that we not raise InvitationError when we have
partially succeeded; that behaviour is left to the one callsite of
do_invite_users.

Reported by Lakshit Agarwal (@chiekosec).
2024-05-02 14:23:04 -07:00
Sahil Batra e78d0aacaf tests: Use NamedUserGroup for queries. 2024-04-26 17:03:09 -07:00
Lauryn Menard 91ffb548cc streams: Update translated errors for stream to channel rename.
Updates translated JsonableError strings that relate to streams
to use channel instead of stream. Separated from other error string
updates as this is a dense area of changes for this rename.

Part of stream to channel rename project.
2024-04-24 14:35:05 -07:00
Lauryn Menard d0a62020ff stats: Update translated strings for stream to channel rename.
Updates the labels for the "Messages sent by recipient type" chart
to use "channel", and updates the error message that would be sent
for the "messages_sent_by_stream" chart (that has not yet been
implemented) for a missing channel/stream.

Part of the stream to channel rename project.
2024-04-24 14:35:05 -07:00
John Lu a5cf0ec526
refactor: Replace HUDDLE with DIRECT_MESSAGE_GROUP.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.

Fixes part of #28640.

Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
2024-03-21 16:39:33 -07:00
Anders Kaseorg 570f3dd447 python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-29 17:07:16 -08:00
Tim Abbott d0c276d863 corporate: Fix billing_session variable reuse confusion.
The previous logic incorrectly used the server-level number of users
even when a (presumably smaller) realm-level count was available.

Fixes a bug introduced in 2e1ed4431a.
2024-02-21 17:51:30 -08:00
Lauryn Menard df2f4b6469 corporate: Move support and activity views to /corporate.
View functions in `analytics/views/support.py` are moved to
`corporate/views/support.py`.

Shared activity functions in `analytics/views/activity_common.py`
are moved to `corporate/lib/activity.py`, which was also renamed
from `corporate/lib/analytics.py`.
2024-01-30 10:06:48 -08:00
Prakhar Pratyush edec29e0b6 support: Add support to configure fixed_price plan. 2024-01-29 11:23:20 -08:00
Lauryn Menard dcae35196c remote-activity: Prefetch LicenseLedger entry for current plan.
To estimate the annual recurring revenue for remote server and
remote realm CustomerPlans, we prefetch the current license
ledger as part of the CustomerPlan query.

Also adds a select related to the remote realm audit log query
so that we don't go to the database for the remote realm ID.

With the test added in the previous commit, the query count for
the remote activity view goes from 27 to 11, as we are no longer
hitting the database multiple times for every current plan or
for every remote realm with audit log data.

Refactors get_customer_plan_renewal_amount so that a license
ledger is always passed and make_end_of_cycle_updates_if_needed
does not need to be called.
2024-01-25 15:35:44 -08:00
Lauryn Menard 842dcb6546 analytics-tests: Build robust test data for remote activity view.
Creates multiple remote servers and remote realms with active
plan data and audit logs.

Shows multiple small queries to get revenue data from plan license
ledger, as well as remote server and remote realm IDs.
2024-01-25 15:35:44 -08:00
Lauryn Menard fca9ff1ae7 support: Show date for start of next billing cycle for current plan.
Instead of showing the next invoice date for the plan, show the
date for the next billing cycle start (e.g. the next plan renewal
charge date), except for plans currently on a free trial.

For plans on a free trial, the next plan renewal date will be when
the free trial ends, which is stored as the next invoice date on
the plan.
2024-01-22 10:09:00 -08:00
Lauryn Menard 76b26612a0 remote-activity: Get user counts for all servers and realms.
Instead of querying the database for every remote server and realm
in the remote activity chart, we now get the server and realm data
for the installation in two queries.
2024-01-19 11:46:13 -08:00
Lauryn Menard 536aef854c remote-activity: Display rows for remote realms.
Adds columns for remote realm ID, name and organization type. If
a remote server has remote realms attached that are not marked
as deactivated by the remote server, then there will be a row in
the chart for each remote realm (which duplicates some remote
server data).

Updates the plan data, revenue and user counts to be for the realm
if present and otherwise for the server.

Updates the user counts to be total users and guest users, instead
of non guest and guest users.

The total row for mobile data (users and pushes forwarded) sums
each remote server's data once, so while the column duplicates
data, the total row should be an accurate total for the installation.

Adds 5 queries to the remote activity page test. One is for the
additional query for the remote realm plans. The other four are
getting the remote realm object and then the user count data for
the two remote realms in the test.
2024-01-19 11:46:13 -08:00
Lauryn Menard 030f899195 realm-activity: Merge chart for human and bot users.
Merges the two charts remaining to have just one chart for the
realm activity view.

Removes the columns for different clients and adds a column to
show/sort by the user type (human or bot type).

Deletes templates/analytics/activity.html because it is no
longer used for any activity pages/views.
2024-01-16 09:43:42 -08:00
Lauryn Menard 82e3d0388b realm-activity: Remove Clients and History charts.
Removes the Clients and History tabs/charts from the realm
activity view.
2024-01-16 09:43:42 -08:00
Lauryn Menard 7e1d0adb6e support-tests: Add coverage for discount actions when upgrade scheduled. 2024-01-15 03:10:04 -08:00
Lauryn Menard f27b6d896a audit-logs: Add generic property changed for Customer and CustomerPlan.
Change audit logs for changing a Customer or CustomerPlan property
to use a generic event type and specify the property name in the
extra data.
2024-01-15 03:10:04 -08:00
Lauryn Menard 2994685399 support: Add admin support for updating end date of active plan.
This currently will only apply to tier.SELF_HOSTED_LEGACY plans.
2024-01-09 10:13:21 -08:00
Lauryn Menard 71263ac2ab support: Include legacy plan without upgrade in remote support tests. 2024-01-09 10:13:21 -08:00
Lauryn Menard 92ebeb7baa support: Add a link to remote server's installation analytics page. 2024-01-05 16:55:38 -08:00
Lauryn Menard f3d0ec15ca support: Clarify what missing data caused MissingDataError.
Update strings for support views to indicate if analytics data or
audit log data was missing and caused a MissingDataError.
2024-01-05 16:55:38 -08:00
Alya Abbott 2a57aa6cea billing: Improve display of sponsorship requests. 2024-01-04 12:25:30 -08:00
Lauryn Menard ea1da738d6 support: Update how remote server information is organized.
Changes the emoji that highlights when a remote realm or server has
an active discount.

Adds a remote server UUID field to the remote server information.

Moves the 'has remote realms' field to be after the Zulip version
information, and adds an extra break there to visually highlight
that field as well as the following 'max monthly messages' field.
2024-01-02 13:16:35 -08:00
Lauryn Menard b9ca60ab9e support: Add date created field to remove server information.
Removes the last updated field and replaces it with the date the
remote server was created based on the audit log data.
2024-01-02 13:16:35 -08:00
Lauryn Menard 481828e556 support: Exclude deactivated servers from search results. 2024-01-02 07:51:50 -08:00
Lauryn Menard ffd708ecaf support: Display next plan data on remote support view.
Currently, this will only be the case for legacy self-managed plans
that have scheduled a switch to either the Basic or Business plan.
2023-12-30 11:22:02 -08:00
Lauryn Menard 653901fc30 support: Update remote server support search test.
Creates some reusable helper functions and adds remote realms to
the search results that are checked, which gives coverage for the
remote realm user counts in the support view.
2023-12-19 14:44:26 -08:00
Lauryn Menard 9e9fec68a1 support: Add plan's estimated annual revenue to remote support view.
Also fixes formatting for per license price and moves the billing
schedule to be above this line so that it's clearer the per license
price is based on the billing schedule.
2023-12-19 14:44:26 -08:00
Lauryn Menard a8172591ca support: Show current discount for current plan.
This should match any discount information shown for the remote
realm or server.
2023-12-19 14:44:26 -08:00
Lauryn Menard f92152fafb support: Add shared helper for customer/plan in support views tests.
Adds a shared helper function for support view tests that need
a customer and/or customer plan.
2023-12-19 14:44:26 -08:00
Anders Kaseorg c9c819e1d7 models: Extract zerver.models.scheduled_jobs.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg bac027962f models: Extract zerver.models.clients.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg cd96193768 models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00