Commit Graph

1114 Commits

Author SHA1 Message Date
Alex Vandiver 9f231322c9 workers: Pass down if they are running multi-threaded.
This allows them to decide for themselves if they should enable
timeouts.
2023-05-16 14:05:01 -07:00
Sahil Batra 7f01b3fb63 users: Set tos_version to -1 for users who have not logged-in yet.
We now set tos_version to "-1" for imported users and the ones
created using API or using other methods like LDAP, SCIM and
management commands. This value will help us to allow users to
change email address visibility setting during first login.
2023-05-16 13:52:56 -07:00
Tim Abbott fe9656237e scheduled_message: Move remaining logic out of management command.
While this doesn't include tests yet, it will make it much easier to
write tests for the core server logic here.
2023-05-09 13:48:28 -07:00
Tim Abbott 7051d3416b scheduled_messages: Add reasonable failure handling.
Previously, it seemed possible for the scheduled messages API to try
to send infinite copies of a message if we had the very poor luck of a
persistent failure happening after a message was sent.

The failure_message field supports being able to display what happened
in the scheduled messages modal, though that's not exposed to the API
yet.
2023-05-09 13:48:28 -07:00
Tim Abbott 24bb03c89f scheduled_messages: Add logging on successful send. 2023-05-09 13:48:28 -07:00
Tim Abbott 1fc6ed224a scheduled_messages: Remove batching logic.
The previous logic would attempt to send a large number of unrelated
messages in a single transaction, which is just asking for trouble in
the event that one of the attempts fails.
2023-05-09 13:48:28 -07:00
Tim Abbott c8f3443fb6 schedules_messages: Move logic out of management command. 2023-05-09 13:48:28 -07:00
Tim Abbott 611cee177d deliver_scheduled_messages: Rename misleading message variable. 2023-05-09 13:48:28 -07:00
Lauryn Menard a5b527f321 onboarding: Specialize Welcome Bot message for education organizations.
Because education organizations and users have slightly specialized
use cases, we update the Welcome Bot message content sent to new
users and new organization owners for these types of organizations
to link to help center articles/guides geared toward these users
and organizations.

Also, updates the demo organization warning to only go to the new
demo organization owner because the 30 day deletion text is only
definitely accurate when the organization is created.

Fixes #21694.
2023-05-01 16:48:48 -07:00
Anders Kaseorg 03b3c8522d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 21:20:33 -07:00
Zixuan James Li 032e6f8cb4 linkifier: Update edit_linkifier command to use url_template.
This is mainly updating the variable names and relevant docstrings
without actual change to the behavior of the command.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
Mateusz Mandera ffa3aa8487 auth: Rewrite data model for tracking enabled auth backends.
So far, we've used the BitField .authentication_methods on Realm
for tracking which backends are enabled for an organization. This
however made it a pain to add new backends (requiring altering the
column and a migration - particularly troublesome if someone wanted to
create their own custom auth backend for their server).

Instead this will be tracked through the existence of the appropriate
rows in the RealmAuthenticationMethods table.
2023-04-18 09:22:56 -07:00
Alex Vandiver e3f6e810dc export-search: Support writing out attachments as well. 2023-04-07 09:13:48 -07:00
Alex Vandiver 885334a3ad export-search: Support CSV output, trim down output fields. 2023-04-07 09:13:48 -07:00
Alex Vandiver bde978d5b1 export-search: Move topic into recipient field. 2023-04-07 09:13:48 -07:00
Alex Vandiver 26bc112adc export-search: Add a --recipient filter. 2023-04-07 09:13:48 -07:00
Alex Vandiver e44ba1bef1 export-search: Write output size before beginning export. 2023-04-07 09:13:48 -07:00
Alex Vandiver 21db267202 export-search: Non-time-limited is fine with a sender. 2023-04-07 09:13:48 -07:00
Alex Vandiver ae20af84a9 export-search: Realm is required. 2023-04-07 09:13:48 -07:00
Daniil Fadeev 2f203f4de1 emails: Inline CSS in emails in build_email.
Previously, we had an architecture where CSS inlining for emails was
done at provision time in inline_email_css.py. This was necessary
because the library we were using for this, Premailer, was extremely
slow, and doing the inlining for every outgoing email would have been
prohibitively expensive.

Now that we've migrated to a more modern library that inlines the
small amount of CSS we have into emails nearly instantly, we are able
to remove the complex architecture built to work around Premailer
being slow and just do the CSS inlining as the final step in sending
each individual email.

This has several significant benefits:

* Removes a fiddly provisioning step that made the edit/refresh cycle
  for modifying email templates confusing; there's no longer a CSS
  inlining step that, if you forget to do it, results in your testing a
  stale variant of the email templates.
* Fixes internationalization problems related to translators working
  with pre-CSS-inlined emails, and then Django trying to apply the
  translators to the post-CSS-inlined version.
* Makes the send_custom_email pipeline simpler and easier to improve.

Signed-off-by: Daniil Fadeev <fadeevd@zulip.com>
2023-04-05 12:22:29 -07:00
Alex Vandiver 113a8c4782 export: Make --deactivate-realm exports be imported as active. 2023-04-03 16:08:43 -07:00
Alex Vandiver 822683249d web: Save a 301 redirect from /terms/ to /policies/terms. 2023-03-24 14:51:01 -07:00
Anders Kaseorg b719227b04 ruff: Fix C416 Unnecessary `dict` comprehension.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-24 11:57:07 -07:00
Alex Vandiver 8ad4ecb3a2 management: Add a command to send initial welcome bot messages. 2023-03-21 11:40:37 -07:00
Anders Kaseorg 8cd78d356f templates: Allow the same line-wrapping in {{t}} as in {{#tr}}.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-16 11:26:45 -07:00
Anders Kaseorg d3efd4c095 python: Import F, Q, QuerySet from their canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg 43b4f10578 run-dev: Drop .py from script name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:37 -08:00
Alex Vandiver 3bf3f47b49 delete_old_unclaimed_attachments: Add flag to clean up storage.
Actions like deleting realms may leave unreferenced uploads in the
attachment storage backend.

Fix these by walking the complete contents of the attachment storage
backend, and removing files which are no longer present in the
database.  This may take quite some time, as it is necessarily O(n) in
the number of files uploaded to the system.
2023-03-02 16:36:19 -08:00
Alex Vandiver cdda4bc089 delete_realm: Clean up attachments when deleting or scrubbing realms. 2023-03-02 16:36:19 -08:00
Anders Kaseorg c1675913a2 web: Move web app to ‘web’ directory.
Ever since we started bundling the app with webpack, there’s been less
and less overlap between our ‘static’ directory (files belonging to
the frontend app) and Django’s interpretation of the ‘static’
directory (files served directly to the web).

Split the app out to its own ‘web’ directory outside of ‘static’, and
remove all the custom collectstatic --ignore rules.  This makes it
much clearer what’s actually being served to the web, and what’s being
bundled by webpack.  It also shrinks the release tarball by 3%.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Anders Kaseorg 6992d3297a ruff: Fix PIE810 Call `startswith` once with a `tuple`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-08 16:40:35 -08:00
Anders Kaseorg b08c81b312 ruff: Fix G201 Logging `.error(..., exc_info=True)` → `.exception(...)`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-04 16:36:20 -08:00
Anders Kaseorg df001db1a9 black: Reformat with Black 23.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.

(This does not actually upgrade our Python environment to Black 23
yet.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-02 10:40:13 -08:00
Anders Kaseorg 6303ebfc2f ruff: Fix SIM115 Use context handler for opening files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg b0e569f07c ruff: Fix SIM102 nested `if` statements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg 2c5e114f8b ruff: Fix ISC001 Implicitly concatenated string literals on one line.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg 705460ff58 ruff: Fix DTZ004 `datetime.datetime.utcfromtimestamp()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg e1ed44907b ruff: Fix SIM118 Use `key in dict` instead of `key in dict.keys()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg 5f2e895c5e scrub_realm: Exit unsuccessfully when asked to scrub an active realm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-05 11:07:48 -08:00
Anders Kaseorg 9a89a52086 ruff: Fix UP015 Unnecessary open mode parameters.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Anders Kaseorg f3f5dfb5aa ruff: Fix RUF004 exit() is only available in the interpreter.
‘exit’ is pulled in for the interactive interpreter as a side effect
of the site module; this can be disabled with python -S and shouldn’t
be relied on.

Also, use the NoReturn type where appropriate.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Anders Kaseorg 73c4da7974 ruff: Fix N818 exception name should be named with an Error suffix.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-17 16:52:00 -08:00
Alex Vandiver e41ee10966 export_search: Add a sender limit. 2022-11-17 11:19:59 -08:00
Anders Kaseorg e5c26eeb86 tornado: Support sharding by user ID.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-15 17:27:01 -08:00
Alex Vandiver f3135eb414 export_search: Add ability to limit by date sent. 2022-11-10 16:49:19 -08:00
Alex Vandiver 2d5366f455 export_search: Add a compliance tool to export a subset of messages.
This is primarily for administrators needing to provide message
history for compliance or auditing purposes.  Search terms can be
pulled from a file, one per line, or from arguments provided on the
command line.
2022-11-10 16:49:19 -08:00
Mateusz Mandera ebc144896f delete_realm: Use Message.realm column for message count. 2022-10-31 09:57:42 -07:00
Anders Kaseorg 1385a827c2 python: Clean up getattr, setattr, delattr calls with literal names.
These were useful as a transitional workaround to ignore type errors
that only show up with django-stubs, while avoiding errors about
unused type: ignore comments without django-stubs.  Now that the
django-stubs transition is complete, switch to type: ignore comments
so that mypy will tell us if they become unnecessary.  Many already
have.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:40:28 -07:00
Anders Kaseorg 5e1ebf2861 python: Fix __dict__ mutation abuse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:33:08 -07:00
Mateusz Mandera 00b3546c9f models: Add denormalized .realm column to Message.
This commit adds the OPTIONAL .realm attribute to Message
(and ArchivedMessage), with the server changes for making new Messages
have this set. Old Messages still have to be migrated to backfill this,
before it can be non-nullable.

Appropriate test changes to correctly set .realm for Messages the tests
manually create are included here as well.
2022-10-07 10:09:38 -07:00