Commit Graph

468 Commits

Author SHA1 Message Date
Lauryn Menard 70ab893d34 urls: Generate narrow links in backend with "channel" operator. 2024-10-11 17:00:23 -07:00
Shubham Padia 12ebd97f1f settings: Add group_creator as default for can_manage_group.
We create an unnamed user group with just the group creator as it's
member when trying to set the default. The pattern I've followed across
most of the acting_user additions is to just put the user declared
somewhere before the check_add_user_group and see if the test passes.
If it does not, then I'll look at what kind of user it needs to be set
to `acting_user`.
2024-10-01 17:35:14 -07:00
Anders Kaseorg f0f048de69 corporate: Import corporate.lib.stripe lazily.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-09-24 18:18:26 -07:00
Lauryn Menard 00ecd4c8f0 audit-log: Move remote realm event types to AuditLogEventType enum.
Event types moved: REMOTE_REALM_VALUE_UPDATED,
REMOTE_PLAN_TRANSFERRED_SERVER_TO_REALM, REMOTE_REALM_LOCALLY_DELETED
REMOTE_REALM_LOCALLY_DELETED_RESTORED
2024-09-09 11:50:13 -07:00
Lauryn Menard aad93e149e audit-log: Move remote server event types to AuditLogEventType enum.
Event types moved: REMOTE_SERVER_DEACTIVATED, REMOTE_SERVER_REACTIVATED
REMOTE_SERVER_PLAN_TYPE_CHANGED, REMOTE_SERVER_DISCOUNT_CHANGED
REMOTE_SERVER_SPONSORSHIP_APPROVED, REMOTE_SERVER_BILLING_MODALITY_CHANGED
REMOTE_SERVER_SPONSORSHIP_PENDING_STATUS_CHANGED, REMOTE_SERVER_CREATED
2024-09-09 11:50:13 -07:00
Lauryn Menard d2c32f23db audit-log: Move realm event types to AuditLogEventType enum.
Event types moved: REALM_DEACTIVATED, REALM_REACTIVATED, REALM_SCRUBBED
REALM_PLAN_TYPE_CHANGED, REALM_LOGO_CHANGED, REALM_EXPORTED
REALM_PROPERTY_CHANGED, REALM_ICON_SOURCE_CHANGED, REALM_DISCOUNT_CHANGED
REALM_SPONSORSHIP_APPROVED, REALM_BILLING_MODALITY_CHANGED
REALM_REACTIVATION_EMAIL_SENT, REALM_SPONSORSHIP_PENDING_STATUS_CHANGED
REALM_SUBDOMAIN_CHANGED
2024-09-09 11:50:13 -07:00
Lauryn Menard e5daa3470f audit-log: Move user event types to AuditLogEventType enum.
Event types moved: USER_CREATED, USER_ACTIVATED, USER_DEACTIVATED
USER_REACTIVATED, USER_ROLE_CHANGED, USER_DELETED
USER_DELETED_PRESERVING_MESSAGES
2024-09-09 11:50:13 -07:00
Prakhar Pratyush 8ddaa949fc user_settings: Send event on commit in do_regenerate_api_key.
Earlier, we were using 'send_event' in 'do_regenerate_api_key'
which can lead to a situation, if any db operation is added after
the 'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-23 09:10:56 -07:00
Prakhar Pratyush ed512f06bb message_edit: Queue event on commit in do_update_message codepath.
'do_update_message' is within a db transaction, this commit
updates the 'do_clear_mobile_push_notifications_for_ids' function
used in 'do_update_message' to queue event on commit.

Events should not be sent until we know we're not rolling back,
otherwise it can lead to a situation where we enqueue events but
the function fails at a later stage.
2024-08-19 10:14:49 -07:00
Mateusz Mandera 4a93149435 settings: Rework how push notifications service is configured.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
2024-07-17 17:14:06 -07:00
Vector73 7a80fcf042 events: Send `delete_message` event to user who deleted the message.
Fixes #29826.

Co-authored-by: Mukul Goyal <goyal.mukul7689@gmail.com>
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
2024-07-16 23:29:02 -07:00
Anders Kaseorg 1e9b6445a9 ruff: Fix PLR6104 Use `+=` to perform an augmented assignment directly.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:49:51 -07:00
Anders Kaseorg b96feb34f6 ruff: Fix SIM117 Use a single `with` statement with multiple contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:48:32 -07:00
Anders Kaseorg 48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
roanster007 52692a6448 refactor: Rename `huddle` to `direct_message_group` in non API.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.

It also renames variables and methods of type -
"huddle_message" to "group_direct_message".

This is a part of #28640
2024-07-04 07:56:31 -07:00
Lauryn Menard 673a01ea0c realm-deactivation: Send email to owners as part of deactivation.
Creates a new "realm_deactivated" email that can be sent to realm
owners as part of `do_deactivate_realm`, via a boolean flag,
`email_owners`.

This flag is set to `False` when `do_deactivate_realm` is used for
realm exports or changing a realm's subdomain, so that the active
organization owners are not emailed in those cases.

This flag is optional for the `deactivate_realm` management command,
but as there is no active user passed in that case, then the email
is sent without referencing who deactivated the realm.

It is passed as `True` for the support analytics view, but the email
that is generated does not include information about the support
admin user who completed the request for organization deactivation.

When an active organization owner deactivates the organization, then
the flag is `True` and an email is sent to them as well as any other
active organization owners, with a slight variation in the email text
for those two cases.

Adds specific tests for when `email_owners` is passed as `True`. All
existing tests for other functionality of `do_deactivate_user` pass
the flag as `False`.

Adds `localize` from django.util.formats as a jinja env filter so
that the dates in these emails are internationlized for the owner's
default language setting in the "realm_deactivated" email templates.

Fixes #24685.
2024-06-26 16:48:18 -07:00
Mateusz Mandera 9f24b30e78 test_push_notifs: Adjust .filter() in test_analytics_api.
The test asserts entries about the zephyr realm anyway. The reason the
filter hasn't been limiting the query to zephyr is that we might
simultaneously want to ensure no other realms received changes - but
that doesn't seem quite right, given that the test doesn't dilligently
set up the initial conditions for all realms to have control over what
exactly happens with them. That makes this logic pretty fragile since if
some new (potentially unrelated) changes to Realm/RemoteRealm initial
state make it so some updates to other realms occur during the early
analytics upload calls in the test, the remote_audit_logs asserts here
will break in a very annoying to debug way.

If we want this query to be general, without limiting to the zephyr
realm, we could alter the test a bit to set up initial conditions
precisely.
2024-06-23 07:44:11 -07:00
Mateusz Mandera f8616fa013 analytics: Send ZULIP_MERGE_BASE to the bouncer. 2024-06-23 07:44:11 -07:00
Mateusz Mandera 4917e01ffb push_notifications: Migrate to FCM HTTP v1 API.
The legacy API we use via python-gcm is deprecated and about to be
disabled.

Fixes #29768.
2024-06-17 18:26:26 -07:00
Mateusz Mandera 496aa2a0f1 test_push_notifs: Remove gcm_client mock in test_connection_error.
This wasn't doing anything. The test simulates failure to connect to the
bouncer, so the codepaths inside the bouncer don't even get triggered.
2024-06-17 18:23:00 -07:00
Mateusz Mandera 2628af9ef4 test_push_notifs: Rename GCM->FCM in some var and func names. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 297141f8c4 push_notifs: Rename parse_gcm_options to parse_fcm_options. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 00b8cce50e push_notifs: Rename PushDeviceToken.GCM to FCM. 2024-06-17 18:22:59 -07:00
Mateusz Mandera 4591202032 push_notifs: Remove "canonical" handling in FCM API.
We believe this to already be obsolete and dead code and is about to be
removed with the migration to the FCM HTTP v1 API, where the concept
doesn't exist anymore.
2024-06-17 18:22:59 -07:00
Alex Vandiver 04450389e1 push_notifications: Provide a stable ordering.
Otherwise, this can cause test failures based on internals of the database.
2024-06-05 11:48:27 -07:00
Alex Vandiver 7792b12f05 test_push_notifications: Fix a typo. 2024-06-03 12:35:35 -07:00
Alex Vandiver 6c17cca208 zilencer: Drop unwanted data that old servers might still send. 2024-06-03 12:35:35 -07:00
Anders Kaseorg e53e3af0f6 codespell: Fix spelling mistakes caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-31 14:32:33 -07:00
Alex Vandiver e5a0b3b3c5 zilencer: Disambiguate no MX records from the domain not existing.
This switches to dnspython, since it offers the higher-level
`resolve_name` method to look up both A and AAAA records, and set
timeouts.
2024-05-24 09:34:31 -07:00
Alex Vandiver 9d66a8eb73 zilencer: Give a better error message for foo@example.com emails. 2024-05-24 09:34:31 -07:00
Mateusz Mandera 27c4e46b30 do_deactivate_realm: Add deactivation_reason kwarg.
It's going to be helpful in the future to record the reason for realm
deactivation.
- For information tracking
- For making a distinction between cases where we can allow realm owners
  to reactivate their realm via a self-serve flow (e.g.
  "owner_request") vs where we can't (ToS abuse).
2024-05-19 23:07:28 -07:00
Vector73 ac4dde24ae realm: Add an alias `realm_url` and deprecate `realm_uri` in the API.
The naming `uri` is deprecated while `url` should be used in order to
satisfy URL standards. For this reason, four endpoints are affected:

* The response content of three endpoints `/server_settings`,
`/register` and `/realm` that contain a field `realm_uri` is
changed to `realm_url`.

* In one of the common fields for all mobile push notifications payloads,
`realm_url` field is now added as an alias to `realm_uri`.

For backwards compatibility, we keep the field `realm_uri` and add
an alias `realm_url`.

Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
2024-05-08 17:39:15 -07:00
Vector73 8ab526a25a models: Replace realm.uri with realm.url.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.

This commit changes just the model property, which has no API
compatibility concerns.
2024-05-08 11:12:43 -07:00
Prakhar Pratyush c727b36e9c soft_reactivate: Soft reactivate if group mention has < 12 members.
Earlier, we didn't soft-reactivate users for group mentions
at all because it wasn't easy to calculate group size.

Now, we will soft reactivate if the user group mentions has
less than 12 members.

We don't reactivate all users because a user group can have a
very large size, which can lead to large backlogs in the
deferred-work queue.

Fixes part of #27586.
2024-04-24 17:40:37 -07:00
Kenneth Rodrigues 081119f461 push-notifications: Migrate to typed endpoint. 2024-04-21 11:09:32 -07:00
Mateusz Mandera 962ab13203 push_notifs: Make push_notifications_enabled more resistant to flapping.
Fixes #28403

Uses redis to remember the last time push notifications were experienced
working. This needs to work across processes, so can't be done just in
memory.
As this is transient data that's fairly harmless to lose and thus
doesn't require the persistence benefits of the database, and we're
keeping a single "row", so don't need an entire new db table, we settle
on using redis instead of postgres. This is also consistent with how we
store other kinds of such transient data.
2024-04-19 10:25:34 -07:00
Mahhheshh d82f33a3c8 zilencer: Migrate to `@typed_endpoint`.
migrated views:
- `zilencer.views.register_remote_server`
- `zilencer.views.register_remote_push_device`
- `zilencer.views.unregister_remote_push_device`
- `zilencer.views.unregister_all_remote_push_devices`
- `zilencer.views.remote_server_notify_push`

to make sure the previous checks for `remote_server_notify_push` matches
to old one, The `RemoteServerNotificationPayload` is defined.
2024-04-17 13:45:54 -07:00
Alex Vandiver 5654d051f7 worker: Split into separate files.
This makes each worker faster to start up.
2024-04-16 23:00:02 -07:00
Aditya Bajaj a20b454ebb
notifications: Shorten string for redacted content.
Replace the long string for organisations that have notification
body/content disabled (settings.PUSH_NOTIFICATION_REDACT_CONTENT
set to true) with "New message".

This allows more of the limited space on the mobile device screen to
be used for additional messages rather than this verbose content.

Fixes #29152
2024-03-22 09:37:51 -07:00
Lauryn Menard 373671080d remote-support: Include billing entity name in internal billing notice.
Adds a line to the top of the internal_billing_notice email with
the billing entity's display name.

Makes sure all internal_billng_notice email subjects also include
the billing entity's display name.

Makes small updates to the notice text for some cases.
2024-03-20 11:58:15 -07:00
Anders Kaseorg 0e71f52e86 ruff: Fix E203 Whitespace before ','
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 09:30:04 -08:00
Anders Kaseorg 570f3dd447 python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-29 17:07:16 -08:00
Prakhar Pratyush d66b7ad853 zilencer: Notify when paid plan attached to now-deleted remote realm.
When a server doesn't submit a remote realm info which was
previously submitted, we mark it as locally deleted.

If such a realm has paid plan attached to it, we should investigate.

This commit adds logic to send an email to sales@zulip.com for
investigation.
2024-02-29 12:50:23 -08:00
Lauryn Menard cf82d3316b push-bouncer: Exclude LoggingCountStats with partial data.
LoggingCountStats with a daily duration and that are directly stored
on the RealmCount table (not via aggregation in process_count_stat),
can be in a state, after the hourly cron job to update analytics
counts, where the logged value will be live-updated later, because
the end time for the stat is still in the future.

As these logging counts are designed to be used on the self-hosted
installation for either debugging or rate limiting, sending these
partial/incomplete counts to the bouncer has low value.
2024-02-26 17:53:12 -08:00
Mateusz Mandera d7cf4336e4 zilencer: Flip realm_locally_deleted when realm re-appears during sync. 2024-02-24 08:52:11 -08:00
Tim Abbott d0c276d863 corporate: Fix billing_session variable reuse confusion.
The previous logic incorrectly used the server-level number of users
even when a (presumably smaller) realm-level count was available.

Fixes a bug introduced in 2e1ed4431a.
2024-02-21 17:51:30 -08:00
Tim Abbott 2e1ed4431a corporate: Fix plan precedence issues with expired plans.
RemoteRealm customer takes precedence over RemoteServer
in general. But if an inactive plan is associated with
RemoteRealm and an active plan with RemoteServer, the
ACTIVE plan takes precendence.

Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
2024-02-19 17:58:49 -08:00