Earlier, we used to store the key data related to realm exports
in RealmAuditLog. This commit adds a separate table to store
those data.
It includes the code to migrate the concerned existing data in
RealmAuditLog to RealmExport.
Fixes part of #31201.
In docker-zulip installs, /etc/zulip/zulip.conf,
/etc/zulip/zulip-secrets.conf, and /home/zulip/uploads are all
symlinks into the `/data` directory which is mounted as a Docker
Volume. By default, `tar` does not dereference symlinks, leading to
backups that are missing these critical pieces.
Add `-h` to the `tar` invocation, to follow symlinks, so backups in
Docker have all of their pieces. Since none of the contents of the
backup intentionally use symlinks, this is safe.
Co-authored-by: Alex Vandiver <alexmv@zulip.com>
For exporting full with consent:
* Earlier, a message advertising users to react with thumbs up
was sent and later used to determine the users who consented.
* Now, we no longer need to send such a message. This commit
updates the logic to use `allow_private_data_export` user-setting
to determine users who consented.
Fixes part of #31201.
Adds a check for changing an existing guest user's role before
calling do_update_user in the case that a realm has a current
paid plan with manual license management.
Currently, it handles two hook types: 'pre-create' (to verify that the
user is authenticated and the file size is within the limit) and
'pre-finish' (which creates an attachment row).
No secret is shared between Django and tusd for authentication of the
hooks endpoints, because none is necessary -- tusd forwards the
end-user's credentials, and the hook checks them like it would any
end-user request. An end-user gaining access to the endpoint would be
able to do no more harm than via tusd or the normal file upload API.
Regardless, the previous commit has restricted access to the endpoint
at the nginx layer.
Co-authored-by: Brijmohan Siyag <brijsiyag@gmail.com>
Renamed event types below in the enum class to use channel instead of
stream.
Event types moved: STREAM_CREATED, STREAM_DEACTIVATED, STREAM_NAME_CHANGED
STREAM_REACTIVATED, STREAM_MESSAGE_RETENTION_DAYS_CHANGED
STREAM_PROPERTY_CHANGED, STREAM_GROUP_BASED_SETTING_CHANGED
A utility command to enable or disable certain authentication backends
for a realm from the command line. Can be helpful e.g. if the
administrator accidentally disables some auth methods in the UI leaving
themselves with none remaining that they could actually use to log back
into the organization.
Example usage:
```
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --show
Current authentication backends for the realm:
Enabled backends:
Dev
Email
GitHub
GitLab
Google
Apple
SAML
OpenID Connect
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --disable GitHub
Disabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
Dev
Email
GitLab
Google
Apple
SAML
OpenID Connect
Disabled backends:
GitHub
Done!
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --enable GitHub
Enabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
Dev
Email
GitHub
GitLab
Google
Apple
SAML
OpenID Connect
Done!
```
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
Tornado ≥ 5 uses AsyncIOLoop and AsyncIOMainLoop automatically when
appropriate. Fixes “DeprecationWarning: make_current is deprecated;
start the event loop first” with warnings enabled.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
A new table is created to track which path_id attachments are images,
and for those their metadata, and which thumbnails have been created.
Using path_id as the effective primary key lets us ignore if the
attachment is archived or not, saving some foreign key messes.
A new worker is added to observe events when rows are added to this
table, and to generate and store thumbnails for those images in
differing sizes and formats.
This is needed for the command to work in docker-zulip, where the zulip
user may not have write permissions to the parent directory of the
secrets file. By default, crudini creates a temporary file, which makes
the command fail under those conditions.
With --inplace, the secret file gets written to directly without
creation of temporary files.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".
Fixes part of #28640.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.
It also renames variables and methods of type -
"huddle_message" to "group_direct_message".
This is a part of #28640
Creates a new "realm_deactivated" email that can be sent to realm
owners as part of `do_deactivate_realm`, via a boolean flag,
`email_owners`.
This flag is set to `False` when `do_deactivate_realm` is used for
realm exports or changing a realm's subdomain, so that the active
organization owners are not emailed in those cases.
This flag is optional for the `deactivate_realm` management command,
but as there is no active user passed in that case, then the email
is sent without referencing who deactivated the realm.
It is passed as `True` for the support analytics view, but the email
that is generated does not include information about the support
admin user who completed the request for organization deactivation.
When an active organization owner deactivates the organization, then
the flag is `True` and an email is sent to them as well as any other
active organization owners, with a slight variation in the email text
for those two cases.
Adds specific tests for when `email_owners` is passed as `True`. All
existing tests for other functionality of `do_deactivate_user` pass
the flag as `False`.
Adds `localize` from django.util.formats as a jinja env filter so
that the dates in these emails are internationlized for the owner's
default language setting in the "realm_deactivated" email templates.
Fixes#24685.
This commit replaces occurrences of realm_uri with realm_url in email templates
and other related backend files.
Co-authored-by: Junyao Chen <junyao.chen@socitydao.org>
It's going to be helpful in the future to record the reason for realm
deactivation.
- For information tracking
- For making a distinction between cases where we can allow realm owners
to reactivate their realm via a self-serve flow (e.g.
"owner_request") vs where we can't (ToS abuse).
Earlier, we were creating RealmAuditLog with REALM_EXPORTED
event_type when export of public data took place via organization
settings panel.
We were not creating the audit log when the export was executed
via shell i.e './manage.py export'.
This commit creates the audit log in that case too. It will
help during import to distinguish readily between imports
from another Zulip server vs imports from another product.
In #23380, we are changing all occurrences of uri with url in order to
follow the latest URL standard. Previous PRs #25038 and #25045 has
replaced the occurences of uri that has no direct relation with realm.
This commit changes just the model property, which has no API
compatibility concerns.
Gitter broke their older API as part of being integrated
into Matrix.
Their announcement blog says:
"Anything left using the Gitter APIs will need to be
updated to use the Matrix API"
This commit drops the legacy Gitter import tool and
we plan to build a new one for Matrix in future.
The returns plugin hasn’t been updated for mypy ≥ 1.6. This
annotation is more limited in that it only supports a fixed number of
positional arguments and no keyword arguments, but is good enough for
our purposes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Factor out the repeated pattern of taking a lock, or immediately
aborting with a message if it cannot be acquired. The exit code in
that situation is changed to be exit code 1, rather than the successful
0; we are likely missing new work since that process started.
We move the lockfiles to a common directory under `/srv/zulip-locks`
rather than muddy up `/home/zulip/deployments`.
This commit adds a 'skip-delay' option to the
'send_zulip_update_announcements' management command.
It will be useful for self-hosted servers after 9.0 upgrade to
avoid the 24 hour delay to receive update messages after group
DM is sent to the admins.
One can run the management command with the --skip-delay flag
to immediately send the update messages.
Using --rotate-key without write access to the secrets file is currently
quite painful, since you end up rotating your registration's secret with
no local record of it; so effectively you lose your registration and
need help from support. We should just prevent this failure mode.
Alex Vandiver
Prakhar Pratyush
bao-qian
Mateusz Mandera
Tim Abbott
Lauryn Menard
Anders Kaseorg
roanster007
Aman Agrawal
Vector73
Prakhar Pratyush
rht