Commit Graph

1194 Commits

Author SHA1 Message Date
Prakhar Pratyush cc793612f0 export: Create REALM_EXPORTED audit log for exports via shell.
Earlier, we were creating RealmAuditLog with REALM_EXPORTED
event_type when export of public data took place via organization
settings panel.

We were not creating the audit log when the export was executed
via shell i.e './manage.py export'.

This commit creates the audit log in that case too. It will
help during import to distinguish readily between imports
from another Zulip server vs imports from another product.
2024-05-08 16:16:37 -07:00
Vector73 8ab526a25a models: Replace realm.uri with realm.url.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.

This commit changes just the model property, which has no API
compatibility concerns.
2024-05-08 11:12:43 -07:00
Prakhar Pratyush 422f5c4e1a help: Rename unarchive_stream management command to unarchive_channel. 2024-05-06 11:18:45 -07:00
Alex Vandiver 572fbfe114 queue_processors: Pass the worker_num down into the class. 2024-05-02 14:25:10 -07:00
Prakhar Pratyush b17a006bf0 data_import: Remove gitter data import tool.
Gitter broke their older API as part of being integrated
into Matrix.

Their announcement blog says:
"Anything left using the Gitter APIs will need to be
updated to use the Matrix API"

This commit drops the legacy Gitter import tool and
we plan to build a new one for Matrix in future.
2024-05-02 13:25:25 -07:00
Anders Kaseorg d32d4434dd partial: Replace returns plugin with an annotation.
The returns plugin hasn’t been updated for mypy ≥ 1.6.  This
annotation is more limited in that it only supports a fixed number of
positional arguments and no keyword arguments, but is good enough for
our purposes.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-29 17:14:41 -07:00
Anders Kaseorg 96fbe060a6 python: Mark regexes as raw strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-26 12:30:31 -07:00
Alex Vandiver e8e6abdbba management: Add lockfiles to all _once management jobs.
These would be bad if more than once ran at once.
2024-04-24 14:40:28 -07:00
Alex Vandiver 11dd6791c4 management: Provide a common lockfile dir, and a decorator for it.
Factor out the repeated pattern of taking a lock, or immediately
aborting with a message if it cannot be acquired.  The exit code in
that situation is changed to be exit code 1, rather than the successful
0; we are likely missing new work since that process started.

We move the lockfiles to a common directory under `/srv/zulip-locks`
rather than muddy up `/home/zulip/deployments`.
2024-04-24 14:40:28 -07:00
Tim Abbott 342a9bd5cd zulip_updates: Prevent racing with self using a lock. 2024-04-18 15:14:02 -07:00
Anders Kaseorg f31579a220 python: Avoid relying on Collection supertype of QuerySet.
QuerySet doesn’t implement __contains__, so it can’t be a subtype of
Container or Collection (https://code.djangoproject.com/ticket/35154).
This incorrect subtyping annotation was removed in
https://github.com/typeddjango/django-stubs/pull/1925, so we need to
stop relying on it before upgrading to django-stubs 5.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-16 23:02:16 -07:00
Alex Vandiver d341e6ced1 runtornado: Honor the --skip-checks flag.
This is particularly important because they can be quite heavy-weight.
2024-04-15 14:29:27 -07:00
Prakhar Pratyush bc0325922e commands: Add 'skip-delay' option to send_zulip_update_announcements.
This commit adds a 'skip-delay' option to the
'send_zulip_update_announcements' management command.

It will be useful for self-hosted servers after 9.0 upgrade to
avoid the 24 hour delay to receive update messages after group
DM is sent to the admins.

One can run the management command with the --skip-delay flag
to immediately send the update messages.
2024-04-11 16:13:42 -07:00
rht c77ed52fa9
slack import: Refer the archive file as a file or dir. 2024-04-11 12:05:56 -07:00
Alex Vandiver d834f1fc01 unarchive_stream: Prevent unarchiving a stream in the wrong realm. 2024-04-11 09:41:41 -07:00
Mateusz Mandera 89a22fb076 register_server: Check write access to secrets file when rotate_key.
Using --rotate-key without write access to the secrets file is currently
quite painful, since you end up rotating your registration's secret with
no local record of it; so effectively you lose your registration and
need help from support. We should just prevent this failure mode.
2024-03-22 09:33:00 -07:00
Anders Kaseorg a1a341f0ae ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-21 08:59:17 -07:00
Prakhar Pratyush 118a7e8d9d zulip_updates: Send zulip updates based on zulip_update_*_level.
This commit adds a management command that will run regularly
as a cron job to send zulip updates to realms based on their
current and latest zulip_update_announcements_level.

For realms with:
* level = None: Send a group DM to admins notifying them about
this new feature & suggestion to set the stream accordingly.

* level = 0:
  * If stream is still not configured, wait for a week
    before setting their level to latest level. They will
    miss updates until their configure the stream.
  * If stream is configured, send updates.

* level > 0: Send one message/update per level & increase
  the level by 1 till the latest level.

Fixes #28604.
2024-03-20 11:48:06 -07:00
Alex Vandiver 64df8c7075 send_custom_email: is_realm_admin is a property, not a real column. 2024-03-19 13:19:45 -07:00
Alex Vandiver be59ef9c02 runtornado: Swap deferred reload events to the default.
This makes no immediate reloads the default for runtornado, matching
the production configuration, and changes the development incantation
to be the one to specify the departure from the norm, with
--immediate-reloads.
2024-02-26 22:29:19 -08:00
Alex Vandiver 8e7ab57bf2 export_search: Support exporting DMs or huddles. 2024-02-25 09:34:17 -08:00
Alex Vandiver b25d966352 tornado: Add a --no-immediate-reloads flag to leave web clients alone. 2024-02-15 15:42:50 -08:00
Anders Kaseorg 93198a19ed requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-29 10:41:54 -08:00
Alex Vandiver e1acd7b974 process_queue: For threaded workers, create them when they start.
Creating the QueueProcessingWorker objects when the ThreadedWorker is
created can lead to a race which caused confusing error messages:

1. A thread tries to call `self.worker = get_worker()`
2. This call raises an exception, which is caught by
   `log_and_exit_if_exception`
3. `log_and_exit_if_exception` sends our process a SIGUSR1, _but
    otherwise swallows the error_.
4. The thread's `.run()` is called, which tries to access
   `self.worker`, which was never set, and throws another exception.
5. The process handles the SIGUSR1, restarting.

Move the creation of the worker to when it is started, so the worker
object does not need to be stored, and possibly have a decoupled
failure.
2024-01-12 08:38:46 -08:00
Alex Vandiver 5b1387c9ba send_password_reset_email: Add a flag to only email users who need it.
Emailing the password reset email to users who have already logged in
is not as useful.
2024-01-07 16:43:31 -08:00
Alex Vandiver 6635f3ea12 send_password_reset_email: Narrow type of users.
5a32ea52ae changed the return type of `get_users`, enabling this.
2024-01-07 16:43:31 -08:00
Alex Vandiver 511cc4d8f7 management: Provide a way to deactivate the push registration. 2024-01-04 15:34:28 -08:00
Alex Vandiver bd38e6cb69 send_email: Distinct emails means distinct, case-insensitively. 2024-01-04 10:46:53 -08:00
Alex Vandiver 8d9ead0f6d send_custom_email: Order by delivery_email if necessary.
If we `.distinct("delivery_email")` then we must also
`.order_by("delivery_email")`; adc987dc43 added the `.order_by`
call, which broke the newsletter codepath, since it did not contain
the `delivery_email` in the ordering fields.

Add a flag to distinct on emails in `send_custom_email`.
2024-01-04 10:46:53 -08:00
Alex Vandiver 0697247d59 send_custom_email: Send to recently-active plus owners and admins.
The set of `enable_marketing_emails=True` are those that have opted
into getting marketing newsletter emails -- but we previously limited
further to only those users active in the last month.

Broaden that to "opted in, and either recently active or an owner or
an admin," with the goal of providing information to folks who may
have tried out Zulip in the past.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2024-01-04 10:46:53 -08:00
Anders Kaseorg cff0b78771 models: Move some functions to zerver.lib.attachments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 4aa2d76bea models: Extract zerver.models.streams.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 21ab3858a7 models: Extract zerver.models.linkifiers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg cd96193768 models: Extract zerver.models.realms.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg 45bb8d2580 models: Extract zerver.models.users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-16 22:08:44 -08:00
Anders Kaseorg b472d769d1 change_user_role: List role choices in --help output.
Fixes #28227.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-14 15:06:18 -08:00
Alex Vandiver 612f2c73d6 send_custom_email: Support JSON extra context with remote servers. 2023-12-13 22:28:20 -08:00
Alex Vandiver 2863b5954e send_custom_email: Make --json an additional filter, not a target. 2023-12-13 22:28:20 -08:00
Alex Vandiver 791d66fe28 send_custom_email: Split out the sending to remote servers. 2023-12-13 22:28:20 -08:00
Alex Vandiver 62e6b10ecd send_email: Pull admins_only back into send_custom_email filtering.
There is no reason for this to be an explicit argument to
send_custom_email, when the rest of the filtering already happens
upstream.
2023-12-13 22:28:20 -08:00
Alex Vandiver 1229e38a0d send_custom_email: Stop lying about the type of options.
Saying `**options: str` is a lie, since it contains bools.  We pluck
out the two bools that we need properly typed because we will be
pushing them into function calls, and type them explicitly as bools.
2023-12-13 22:28:20 -08:00
Alex Vandiver 44e9171a46 register_server: POST requests take data, not params.
This currently _works_, but only because we are very forgiving about
how we parse requests.
2023-12-12 19:58:19 -08:00
Prakhar Pratyush 886a86866d register_server: Send server data just after registering server.
We call 'send_server_data_to_push_bouncer' just after registering
server for push notification.

This helps to have a current state of the user counts when first
logging in after the RemoteRealm flow.
2023-12-11 14:07:39 -08:00
Mateusz Mandera a45438619d register_server: Update all the text. 2023-12-10 15:39:10 -08:00
Anders Kaseorg cf16ddd4d6 tornado: Fix autoreload for Tornado 6.4 changes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-09 14:24:45 -08:00
Anders Kaseorg 396e995502 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-07 14:41:01 -08:00
Tim Abbott 9d3ef7f589 settings: Print about custom settings from runtornado. 2023-12-05 12:33:34 -08:00
Anders Kaseorg 8a7916f21a python: Consistently use from…import for datetime.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 12:01:18 -08:00
Alex Vandiver 3eb5b20c41 register_server: Redact the key when printing it. 2023-11-29 09:56:14 -08:00
Mateusz Mandera 1819b85b85 management: Allow changing is_billing_admin using change_user_role. 2023-11-15 18:36:07 -08:00