Commit Graph

1214 Commits

Author SHA1 Message Date
Mateusz Mandera bf7fc5701f management: Add change_auth_backends command.
A utility command to enable or disable certain authentication backends
for a realm from the command line. Can be helpful e.g. if the
administrator accidentally disables some auth methods in the UI leaving
themselves with none remaining that they could actually use to log back
into the organization.

Example usage:
```
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --show
Current authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitHub
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --disable GitHub
Disabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
Disabled backends:
  GitHub
Done!
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --enable GitHub
Enabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitHub
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
Done!
```
2024-08-14 14:04:46 -07:00
Alex Vandiver 4351cc5914 thumbnail: Move get_image_thumbnail_path and split_thumbnail_path. 2024-07-18 13:50:28 -07:00
Mateusz Mandera 4a93149435 settings: Rework how push notifications service is configured.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
2024-07-17 17:14:06 -07:00
Anders Kaseorg ad9837911e runtornado: Remove explicit AsyncIOMainLoop installation.
Tornado ≥ 5 uses AsyncIOLoop and AsyncIOMainLoop automatically when
appropriate.  Fixes “DeprecationWarning: make_current is deprecated;
start the event loop first” with warnings enabled.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-16 14:01:32 -07:00
Alex Vandiver 2e38f426f4 upload: Generate thumbnails when images are uploaded.
A new table is created to track which path_id attachments are images,
and for those their metadata, and which thumbnails have been created.
Using path_id as the effective primary key lets us ignore if the
attachment is archived or not, saving some foreign key messes.

A new worker is added to observe events when rows are added to this
table, and to generate and store thumbnails for those images in
differing sizes and formats.
2024-07-16 13:22:15 -07:00
Mateusz Mandera d89ab2694f register_server: Call crudini with --inplace when rotating secret key.
This is needed for the command to work in docker-zulip, where the zulip
user may not have write permissions to the parent directory of the
secrets file. By default, crudini creates a temporary file, which makes
the command fail under those conditions.
With --inplace, the secret file gets written to directly without
creation of temporary files.
2024-07-15 13:24:14 -07:00
Anders Kaseorg 48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 3f29bc42b1 ruff: Fix B905 `zip()` without an explicit `strict=` parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
roanster007 02d0566dc5 refactor: Rename `Huddle` Django model class to `DirectMessageGroup`.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".

Fixes part of #28640.
2024-07-07 21:31:30 -07:00
roanster007 52692a6448 refactor: Rename `huddle` to `direct_message_group` in non API.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.

It also renames variables and methods of type -
"huddle_message" to "group_direct_message".

This is a part of #28640
2024-07-04 07:56:31 -07:00
Aman Agrawal 4b091aafc2 makemessages: Allow excluding whitespace before `*inline` custom tag.
This otherwise throws capitalization error that handlebars are included
in the translated string when trying to use {{~#*inline}}.
2024-07-03 10:38:35 -07:00
Anders Kaseorg 6a73006723 ruff: Fix FURB105 Unnecessary empty string passed to `print`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-30 22:37:15 -07:00
Lauryn Menard 673a01ea0c realm-deactivation: Send email to owners as part of deactivation.
Creates a new "realm_deactivated" email that can be sent to realm
owners as part of `do_deactivate_realm`, via a boolean flag,
`email_owners`.

This flag is set to `False` when `do_deactivate_realm` is used for
realm exports or changing a realm's subdomain, so that the active
organization owners are not emailed in those cases.

This flag is optional for the `deactivate_realm` management command,
but as there is no active user passed in that case, then the email
is sent without referencing who deactivated the realm.

It is passed as `True` for the support analytics view, but the email
that is generated does not include information about the support
admin user who completed the request for organization deactivation.

When an active organization owner deactivates the organization, then
the flag is `True` and an email is sent to them as well as any other
active organization owners, with a slight variation in the email text
for those two cases.

Adds specific tests for when `email_owners` is passed as `True`. All
existing tests for other functionality of `do_deactivate_user` pass
the flag as `False`.

Adds `localize` from django.util.formats as a jinja env filter so
that the dates in these emails are internationlized for the owner's
default language setting in the "realm_deactivated" email templates.

Fixes #24685.
2024-06-26 16:48:18 -07:00
Vector73 93262e03ad emails: Replace occurrences of uri with url in email templates.
This commit replaces occurrences of realm_uri with realm_url in email templates
and other related backend files.

Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
2024-05-24 11:18:35 -07:00
Alex Vandiver e73e44402a export_search: Support --dm with one user, for all their DMs. 2024-05-24 10:49:52 -07:00
Alex Vandiver 88be3246a0 management: Move commands to all use ZulipBaseCommand. 2024-05-24 10:30:16 -07:00
Mateusz Mandera 27c4e46b30 do_deactivate_realm: Add deactivation_reason kwarg.
It's going to be helpful in the future to record the reason for realm
deactivation.
- For information tracking
- For making a distinction between cases where we can allow realm owners
  to reactivate their realm via a self-serve flow (e.g.
  "owner_request") vs where we can't (ToS abuse).
2024-05-19 23:07:28 -07:00
Prakhar Pratyush cc793612f0 export: Create REALM_EXPORTED audit log for exports via shell.
Earlier, we were creating RealmAuditLog with REALM_EXPORTED
event_type when export of public data took place via organization
settings panel.

We were not creating the audit log when the export was executed
via shell i.e './manage.py export'.

This commit creates the audit log in that case too. It will
help during import to distinguish readily between imports
from another Zulip server vs imports from another product.
2024-05-08 16:16:37 -07:00
Vector73 8ab526a25a models: Replace realm.uri with realm.url.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.

This commit changes just the model property, which has no API
compatibility concerns.
2024-05-08 11:12:43 -07:00
Prakhar Pratyush 422f5c4e1a help: Rename unarchive_stream management command to unarchive_channel. 2024-05-06 11:18:45 -07:00
Alex Vandiver 572fbfe114 queue_processors: Pass the worker_num down into the class. 2024-05-02 14:25:10 -07:00
Prakhar Pratyush b17a006bf0 data_import: Remove gitter data import tool.
Gitter broke their older API as part of being integrated
into Matrix.

Their announcement blog says:
"Anything left using the Gitter APIs will need to be
updated to use the Matrix API"

This commit drops the legacy Gitter import tool and
we plan to build a new one for Matrix in future.
2024-05-02 13:25:25 -07:00
Anders Kaseorg d32d4434dd partial: Replace returns plugin with an annotation.
The returns plugin hasn’t been updated for mypy ≥ 1.6.  This
annotation is more limited in that it only supports a fixed number of
positional arguments and no keyword arguments, but is good enough for
our purposes.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-29 17:14:41 -07:00
Anders Kaseorg 96fbe060a6 python: Mark regexes as raw strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-26 12:30:31 -07:00
Alex Vandiver e8e6abdbba management: Add lockfiles to all _once management jobs.
These would be bad if more than once ran at once.
2024-04-24 14:40:28 -07:00
Alex Vandiver 11dd6791c4 management: Provide a common lockfile dir, and a decorator for it.
Factor out the repeated pattern of taking a lock, or immediately
aborting with a message if it cannot be acquired.  The exit code in
that situation is changed to be exit code 1, rather than the successful
0; we are likely missing new work since that process started.

We move the lockfiles to a common directory under `/srv/zulip-locks`
rather than muddy up `/home/zulip/deployments`.
2024-04-24 14:40:28 -07:00
Tim Abbott 342a9bd5cd zulip_updates: Prevent racing with self using a lock. 2024-04-18 15:14:02 -07:00
Anders Kaseorg f31579a220 python: Avoid relying on Collection supertype of QuerySet.
QuerySet doesn’t implement __contains__, so it can’t be a subtype of
Container or Collection (https://code.djangoproject.com/ticket/35154).
This incorrect subtyping annotation was removed in
https://github.com/typeddjango/django-stubs/pull/1925, so we need to
stop relying on it before upgrading to django-stubs 5.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-16 23:02:16 -07:00
Alex Vandiver d341e6ced1 runtornado: Honor the --skip-checks flag.
This is particularly important because they can be quite heavy-weight.
2024-04-15 14:29:27 -07:00
Prakhar Pratyush bc0325922e commands: Add 'skip-delay' option to send_zulip_update_announcements.
This commit adds a 'skip-delay' option to the
'send_zulip_update_announcements' management command.

It will be useful for self-hosted servers after 9.0 upgrade to
avoid the 24 hour delay to receive update messages after group
DM is sent to the admins.

One can run the management command with the --skip-delay flag
to immediately send the update messages.
2024-04-11 16:13:42 -07:00
rht c77ed52fa9
slack import: Refer the archive file as a file or dir. 2024-04-11 12:05:56 -07:00
Alex Vandiver d834f1fc01 unarchive_stream: Prevent unarchiving a stream in the wrong realm. 2024-04-11 09:41:41 -07:00
Mateusz Mandera 89a22fb076 register_server: Check write access to secrets file when rotate_key.
Using --rotate-key without write access to the secrets file is currently
quite painful, since you end up rotating your registration's secret with
no local record of it; so effectively you lose your registration and
need help from support. We should just prevent this failure mode.
2024-03-22 09:33:00 -07:00
Anders Kaseorg a1a341f0ae ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-21 08:59:17 -07:00
Prakhar Pratyush 118a7e8d9d zulip_updates: Send zulip updates based on zulip_update_*_level.
This commit adds a management command that will run regularly
as a cron job to send zulip updates to realms based on their
current and latest zulip_update_announcements_level.

For realms with:
* level = None: Send a group DM to admins notifying them about
this new feature & suggestion to set the stream accordingly.

* level = 0:
  * If stream is still not configured, wait for a week
    before setting their level to latest level. They will
    miss updates until their configure the stream.
  * If stream is configured, send updates.

* level > 0: Send one message/update per level & increase
  the level by 1 till the latest level.

Fixes #28604.
2024-03-20 11:48:06 -07:00
Alex Vandiver 64df8c7075 send_custom_email: is_realm_admin is a property, not a real column. 2024-03-19 13:19:45 -07:00
Alex Vandiver be59ef9c02 runtornado: Swap deferred reload events to the default.
This makes no immediate reloads the default for runtornado, matching
the production configuration, and changes the development incantation
to be the one to specify the departure from the norm, with
--immediate-reloads.
2024-02-26 22:29:19 -08:00
Alex Vandiver 8e7ab57bf2 export_search: Support exporting DMs or huddles. 2024-02-25 09:34:17 -08:00
Alex Vandiver b25d966352 tornado: Add a --no-immediate-reloads flag to leave web clients alone. 2024-02-15 15:42:50 -08:00
Anders Kaseorg 93198a19ed requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-29 10:41:54 -08:00
Alex Vandiver e1acd7b974 process_queue: For threaded workers, create them when they start.
Creating the QueueProcessingWorker objects when the ThreadedWorker is
created can lead to a race which caused confusing error messages:

1. A thread tries to call `self.worker = get_worker()`
2. This call raises an exception, which is caught by
   `log_and_exit_if_exception`
3. `log_and_exit_if_exception` sends our process a SIGUSR1, _but
    otherwise swallows the error_.
4. The thread's `.run()` is called, which tries to access
   `self.worker`, which was never set, and throws another exception.
5. The process handles the SIGUSR1, restarting.

Move the creation of the worker to when it is started, so the worker
object does not need to be stored, and possibly have a decoupled
failure.
2024-01-12 08:38:46 -08:00
Alex Vandiver 5b1387c9ba send_password_reset_email: Add a flag to only email users who need it.
Emailing the password reset email to users who have already logged in
is not as useful.
2024-01-07 16:43:31 -08:00
Alex Vandiver 6635f3ea12 send_password_reset_email: Narrow type of users.
5a32ea52ae changed the return type of `get_users`, enabling this.
2024-01-07 16:43:31 -08:00
Alex Vandiver 511cc4d8f7 management: Provide a way to deactivate the push registration. 2024-01-04 15:34:28 -08:00
Alex Vandiver bd38e6cb69 send_email: Distinct emails means distinct, case-insensitively. 2024-01-04 10:46:53 -08:00
Alex Vandiver 8d9ead0f6d send_custom_email: Order by delivery_email if necessary.
If we `.distinct("delivery_email")` then we must also
`.order_by("delivery_email")`; adc987dc43 added the `.order_by`
call, which broke the newsletter codepath, since it did not contain
the `delivery_email` in the ordering fields.

Add a flag to distinct on emails in `send_custom_email`.
2024-01-04 10:46:53 -08:00
Alex Vandiver 0697247d59 send_custom_email: Send to recently-active plus owners and admins.
The set of `enable_marketing_emails=True` are those that have opted
into getting marketing newsletter emails -- but we previously limited
further to only those users active in the last month.

Broaden that to "opted in, and either recently active or an owner or
an admin," with the goal of providing information to folks who may
have tried out Zulip in the past.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2024-01-04 10:46:53 -08:00