Commit Graph

858 Commits

Author SHA1 Message Date
Alex Vandiver b557297dd2 zilencer: Drop data which is no longer sent by remote servers. 2024-06-03 12:35:35 -07:00
Alex Vandiver 6c17cca208 zilencer: Drop unwanted data that old servers might still send. 2024-06-03 12:35:35 -07:00
Alex Vandiver 88be3246a0 management: Move commands to all use ZulipBaseCommand. 2024-05-24 10:30:16 -07:00
Alex Vandiver e5a0b3b3c5 zilencer: Disambiguate no MX records from the domain not existing.
This switches to dnspython, since it offers the higher-level
`resolve_name` method to look up both A and AAAA records, and set
timeouts.
2024-05-24 09:34:31 -07:00
Alex Vandiver 9388805471 zilencer: Fix typo in variable name. 2024-05-24 09:34:31 -07:00
Alex Vandiver 9d66a8eb73 zilencer: Give a better error message for foo@example.com emails. 2024-05-24 09:34:31 -07:00
Sahil Batra d1bcac0822 realm: Use enums for accessing setting values.
This commit updates code, majorly in tests, to use
setting values from enums instead of directly using
the constants defined in Realm.

We still have those constants defined Realm as they
are used in a couple of places where the same code
is used for different settings. These will be
handled later.
2024-05-22 17:20:37 -07:00
Anders Kaseorg b545abe1e2 typos: Fix typos caught by mwic.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-20 13:55:00 -07:00
tnmkr 51635c81fc streams: Set creator for some initial streams in populate_db.
This is a follow-up to #29425. It assigns creator to three streams
that are used in tests and another two that aren't. For tests,
we assign a guest, an admin and an owner as creator.
2024-05-15 11:21:46 -07:00
Prakhar Pratyush ac2152fdfd onboarding: Replace channel messages in new organizations.
As a part of improving onboarding experience, this
commit updates the channels and messages initially
available in new orgs.

Fixes #29297.
2024-05-09 14:05:35 -07:00
Prakhar Pratyush 61e606f656 onboarding: Remove 'core team' private channel.
We no longer create the 'core team' private channel when
a realm is created.

Earlier, "New user announcements" channel was set to the
"core team" channel. Now it is disabled by default.

populate_db still creates the 'core team' channel to
represent a private channel.
2024-05-09 14:05:35 -07:00
Vector73 8ab526a25a models: Replace realm.uri with realm.url.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.

This commit changes just the model property, which has no API
compatibility concerns.
2024-05-08 11:12:43 -07:00
Alex Vandiver 7607969f27 zilencer: Add a unique index on RemoteRealm counts with RemoteRealm objects. 2024-05-06 16:34:01 -07:00
Alex Vandiver 67fcaf266e invoice_plans: Add a --date flag, rather than a commented line. 2024-04-24 14:40:28 -07:00
Alex Vandiver e8e6abdbba management: Add lockfiles to all _once management jobs.
These would be bad if more than once ran at once.
2024-04-24 14:40:28 -07:00
Kenneth Rodrigues 326944542e views: Use ApnsAppId for validation. 2024-04-21 11:09:32 -07:00
Tim Abbott 951a474376 populate_db: Don't enqueue push notifications service updates.
Since we're just creating an initial database state, there's no reason
we need to submit increment analytics updates to the bouncer service;
such updates will happen during queue worker startup, if needed.
2024-04-18 11:51:42 -07:00
Mahhheshh d82f33a3c8 zilencer: Migrate to `@typed_endpoint`.
migrated views:
- `zilencer.views.register_remote_server`
- `zilencer.views.register_remote_push_device`
- `zilencer.views.unregister_remote_push_device`
- `zilencer.views.unregister_all_remote_push_devices`
- `zilencer.views.remote_server_notify_push`

to make sure the previous checks for `remote_server_notify_push` matches
to old one, The `RemoteServerNotificationPayload` is defined.
2024-04-17 13:45:54 -07:00
Alex Vandiver 5654d051f7 worker: Split into separate files.
This makes each worker faster to start up.
2024-04-16 23:00:02 -07:00
Alex Vandiver 936c2b54cb push_notifications: Use ignore_conflicts, over catching IntegrityError.
The IntegrityError shows up in the database logs, which looks
unnecessarily concerning.  Use `ON CONFLICT IGNORE` to mark this as
expected, especially since the return value is never used.
2024-04-12 11:36:40 -07:00
Alex Vandiver 9e28619727 zilencer: Fix typo in comment. 2024-04-12 11:36:40 -07:00
Aman Agrawal 17c88de75d invoice_plans: Add comment to help with testing.
This will help us run invoice plans on a specific date. Having
this in comment here would save me time adding this every time.
2024-04-11 21:57:33 -07:00
Mateusz Mandera 374178903e remote_billing: Move stripe_customer_id from server to realm if needed.
As explained in the comment, when we're moving the server plan to the
remote realm's Customer object, the realm Customer may not have
stripe_customer_id set and therefore that value needs to get moved from
the server Customer.
2024-03-29 16:27:03 -07:00
John Lu a5cf0ec526
refactor: Replace HUDDLE with DIRECT_MESSAGE_GROUP.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.

Fixes part of #28640.

Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
2024-03-21 16:39:33 -07:00
Anders Kaseorg 570f3dd447 python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-29 17:07:16 -08:00
Prakhar Pratyush d66b7ad853 zilencer: Notify when paid plan attached to now-deleted remote realm.
When a server doesn't submit a remote realm info which was
previously submitted, we mark it as locally deleted.

If such a realm has paid plan attached to it, we should investigate.

This commit adds logic to send an email to sales@zulip.com for
investigation.
2024-02-29 12:50:23 -08:00
Alex Vandiver 4b512b3409 email_validation: Rename to validate_is_not_disposable.
This clarifies what we are asserting -- namely, that the email is
_not_ disposable.
2024-02-27 10:19:23 -08:00
Prakhar Pratyush fe1a20ebb3 settings: Add realm-level setting 'zulip_update_announcements_stream'.
This commit adds a realm-level setting named
'zulip_update_announcements_stream' that configures the
stream to which zulip updates should be posted.

Fixes part of #28604.
2024-02-25 09:33:00 -08:00
Mateusz Mandera 32837a8ae4 update_remote_realm_data_for_server: Rename uuids to reported_uuids.
uuids is a bit too generic, the rename makes it clearer what exactly is
being referenced conceptually.
2024-02-24 08:52:11 -08:00
Mateusz Mandera d7cf4336e4 zilencer: Flip realm_locally_deleted when realm re-appears during sync. 2024-02-24 08:52:11 -08:00
Mateusz Mandera cb2d35cf27 zilencer: Fix incorrect registration_deactivated mentions in realm sync.
We no longer modify this flag upon encountering a locally deleted realm,
as of 880133295e
2024-02-24 08:52:11 -08:00
Lauryn Menard 1f72ab5133 remote-support: Show active billing user emails. 2024-02-23 13:21:54 -08:00
Lauryn Menard e5c92a1ee6 remote-billing: Add index to RemoteRealmAuditLog for billing events.
When profiling the database queries for the remote support view,
getting the user counts for remote realms was identifed as an
expensive query. Adds an index on RemoteRealmAuditLog to improve
this relatively common query for remote billing information.
2024-02-22 16:30:06 -08:00
Tim Abbott bd0b5d80ba zilencer: Use early return pattern for readability. 2024-02-21 11:10:45 -08:00
Mateusz Mandera ea863bab5b remote_billing: Remove code migrating Legacy plan from server to realms.
We no longer want to migrate Legacy plans from server to realms, since
Legacy plans are not really a thing in the original sense anymore, since
February 15th.

Now they're just a tool to give temporary extensions of access to the
push notification service for users, when needed. And as such, it makes
no sense to migrate like that.

The remaining code in this function is for migrating (any) plan from the
server object to the realm object, if the server has just a single
realm.
2024-02-21 11:10:45 -08:00
Prakhar Pratyush ab453fbe20 settings: Rename notifications_stream to new_stream_announce..._stream.
This commit renames the realm-level setting 'notifications_stream'
to 'new_stream_announcements_stream'.

The new name reflects better what the setting does.
2024-02-21 09:04:23 -08:00
Mateusz Mandera 834dbd552b remote_billing: Make handle_customer_migration_... more robust.
The logic in the case where there's only one realm and the function
tries to migrate the server's plan to it, had two main unhandled edge
cases that would throw exceptions:
1.
```
        remote_realm = RemoteRealm.objects.get(
            uuid=realm_uuids[0], plan_type=RemoteRealm.PLAN_TYPE_SELF_MANAGED
        )
```

This could throw an exception if the RemoteRealm exists, but has an
active e.g. Legacy plan. Then there'd be no object matching the
plan_type in the query, raising RemoteRealm.DoesNotExist.

2. If the RemoteRealm had e.g. a Legacy plan in the past, that's now
   expired, then it'd have a Customer object. Meaning that the attempt
   to move the server's customer to the realm:
   `server_plan.customer = remote_realm_customer`
   would trigger an IntegrityError since a RemoteRealm can't have two
   Customer objects.

In simple cases the situation in (2) can still be easily migrated, by
moving the plan from the server's customer to the realm's customer.
2024-02-20 16:02:03 -08:00
Mateusz Mandera 616527e73e zilencer: Exclude realm_locally_deleted in get_human_user_realm_uuids.
Just like deactivated realms should be excluded, so should locally
deleted realms.

In particular, failure to exclude locally deleted realms breaks
handle_customer_migration_from_server_to_realms.
2024-02-19 20:27:24 -08:00
Alex Vandiver 0f9b7f112b message: Move render_markdown into zerver.lib.markdown. 2024-02-14 12:27:03 -08:00
Mateusz Mandera 3bda31c48c zilencer: Improve json error when plan doesn't allow push notifs.
This allows the self-hosted server to explicitly test for that exception
and catch and log it nicely.
2024-02-07 10:36:33 -08:00
Lauryn Menard 47a5459637 zilencer: Add index on RemoteInstallationCount for remote activity.
When profiling the database query in `remote_activity.py`,
push_forwarded_count was identified as an expensive part of
the overall work. Adds an index on RemoteInstallationCount
so this is more efficient.
2024-02-01 12:01:16 -08:00
Prakhar Pratyush fce50bb4a0 populate_billing_realms: Add legacy remote realm. 2024-01-31 12:25:31 -08:00
Anders Kaseorg 93198a19ed requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-29 10:41:54 -08:00
Mateusz Mandera 4a245a3f62 register_remote_server: Add more validation of contact_email. 2024-01-26 09:09:31 -08:00
Mateusz Mandera 25f47bd749 register_remote_server: Don't allow duplicate hostnames.
This requires a bit of restructuring of the existing code to handle all
the cases correctly.
2024-01-26 09:09:31 -08:00
Alex Vandiver fd83c4ca1b populate_db: Only make valid email addresses.
The presence of the name "François" in the random first name list led
to email addresses of the form `françois1234@zulip.com` which are
invalid.

Strip invalid ASCII from the email addresses generated in populate_db,
and validate them before tossing them verbatim into the database.
2024-01-24 09:24:27 -08:00
Lauryn Menard c08e266956 zilencer-models: Create helper function for remote user counts.
Pulls out shared code in get_remote_realm_guest_and_non_guest_count
and get_remote_server_guest_and_non_guest_count for generating the
RemoteCustomerUserCount so that it can be used in logic for getting
these counts for all remote servers and remote realms on an
installation.
2024-01-19 11:46:13 -08:00
Prakhar Pratyush d21dc1d5cd zilencer: Rename *topic local variables to *topic_name.
This is preparatory work towards adding a Topic model.
We plan to use the local variable name as 'topic' for
the Topic model objects.

Currently, we use *topic as the local variable name for
topic names.

We rename local variables of the form *topic to *topic_name
so that we don't need to think about type collisions in
individual code paths where we might want to talk about both
Topic objects and strings for the topic name.
2024-01-15 09:40:43 -08:00
Mateusz Mandera 3a12e41c35 remote_billing: Fix handle_customer_migration_from_server_to_realms.
This was a bug from 4715a058b0 where this
was just incorrectly called. get_realms_info_for_push_bouncer() is a
function meant to be called on a self-hosted server - and this
handle_... call happens on the bouncer. Therefore this returns all
zulipchat realms in product.

With the way, handle_... is being called right now, there's no reason
for it to have an argument for passing a list of realms. It should just
fetch the relevant RemoteRealm entries  by itself, given the server arg.
2024-01-12 15:28:41 -08:00
Aman Agrawal f285de39e9 populate_realms: Allow billing page access without stripe_api_key. 2024-01-12 08:47:43 -08:00