Commit Graph

1082 Commits

Author SHA1 Message Date
Anders Kaseorg df001db1a9 black: Reformat with Black 23.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.

(This does not actually upgrade our Python environment to Black 23
yet.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-02 10:40:13 -08:00
Anders Kaseorg 6303ebfc2f ruff: Fix SIM115 Use context handler for opening files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg b0e569f07c ruff: Fix SIM102 nested `if` statements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg 2c5e114f8b ruff: Fix ISC001 Implicitly concatenated string literals on one line.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg 705460ff58 ruff: Fix DTZ004 `datetime.datetime.utcfromtimestamp()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg e1ed44907b ruff: Fix SIM118 Use `key in dict` instead of `key in dict.keys()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg 5f2e895c5e scrub_realm: Exit unsuccessfully when asked to scrub an active realm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-05 11:07:48 -08:00
Anders Kaseorg 9a89a52086 ruff: Fix UP015 Unnecessary open mode parameters.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Anders Kaseorg f3f5dfb5aa ruff: Fix RUF004 exit() is only available in the interpreter.
‘exit’ is pulled in for the interactive interpreter as a side effect
of the site module; this can be disabled with python -S and shouldn’t
be relied on.

Also, use the NoReturn type where appropriate.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Anders Kaseorg 73c4da7974 ruff: Fix N818 exception name should be named with an Error suffix.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-17 16:52:00 -08:00
Alex Vandiver e41ee10966 export_search: Add a sender limit. 2022-11-17 11:19:59 -08:00
Anders Kaseorg e5c26eeb86 tornado: Support sharding by user ID.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-15 17:27:01 -08:00
Alex Vandiver f3135eb414 export_search: Add ability to limit by date sent. 2022-11-10 16:49:19 -08:00
Alex Vandiver 2d5366f455 export_search: Add a compliance tool to export a subset of messages.
This is primarily for administrators needing to provide message
history for compliance or auditing purposes.  Search terms can be
pulled from a file, one per line, or from arguments provided on the
command line.
2022-11-10 16:49:19 -08:00
Mateusz Mandera ebc144896f delete_realm: Use Message.realm column for message count. 2022-10-31 09:57:42 -07:00
Anders Kaseorg 1385a827c2 python: Clean up getattr, setattr, delattr calls with literal names.
These were useful as a transitional workaround to ignore type errors
that only show up with django-stubs, while avoiding errors about
unused type: ignore comments without django-stubs.  Now that the
django-stubs transition is complete, switch to type: ignore comments
so that mypy will tell us if they become unnecessary.  Many already
have.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:40:28 -07:00
Anders Kaseorg 5e1ebf2861 python: Fix __dict__ mutation abuse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:33:08 -07:00
Mateusz Mandera 00b3546c9f models: Add denormalized .realm column to Message.
This commit adds the OPTIONAL .realm attribute to Message
(and ArchivedMessage), with the server changes for making new Messages
have this set. Old Messages still have to be migrated to backfill this,
before it can be non-nullable.

Appropriate test changes to correctly set .realm for Messages the tests
manually create are included here as well.
2022-10-07 10:09:38 -07:00
Anders Kaseorg 9198fe4fac scim: Downgrade SCIMClient from a model to an ephemeral dataclass.
SCIMClient is a type-unsafe workaround for django-scim2’s conflation
of SCIM users with Django users.  Given that a SCIMClient is not a
UserProfile, it might as well not be a model at all, since it’s only
used to satisfy django-scim2’s request.user.is_authenticated queries.

This doesn’t solve the type safety issue with assigning a SCIMClient
to request.user, nor the performance issue with running the SCIM
middleware on non-SCIM requests.  But it reduces the risk of potential
consequences worse than crashing, since there’s no longer a
request.user.id for Django to confuse with the ID of an actual
UserProfile.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 11:36:48 -07:00
Anders Kaseorg ce9ceb7f9f tornado: Fix Tornado CSRF check with X-Forwarded-Proto.
Since Django factors request.is_secure() into its CSRF check, we need
this to tell it to consider requests forwarded from nginx to Tornado
as secure.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-23 16:01:12 -07:00
Mateusz Mandera d48075de58 logout_all_users: Add --rotate-api-keys option to the command.
Fixes #19397.
2022-08-15 17:58:05 -07:00
Mateusz Mandera 43928ca323 import: Remove dead do_import_system_bots code.
This code is actually a noop (and would be a bug if it wasn't a noop),
because when this runs the server is already initialized, meaning the
internal realm exists and the system bots have been created, so
UserProfile.objects.filter(email=email) is always truthy. Also, system
bots are supposed to live in the internal realm, not in the realm being
imported so this code doesn't make sense currently.
2022-08-11 13:59:50 -07:00
Sahil Batra ef48047dfc realm: Create RealmAuditLog entry when removing realm linkifiers.
This commit also adds 'acting_user' parameter to do_remove_linkifier
function.

Fixes a part of #21268.
2022-07-31 18:32:28 -07:00
Sahil Batra 4f1dd0710c realm: Create RealmAuditLog entry when adding realm linkifiers.
This commit also adds 'acting_user' parameter to do_add_linkifier
function.

Fixes a part of #21268.
2022-07-31 18:32:28 -07:00
Zixuan James Li aa733f0bbd webhook: Fix f-string str-bytes-safe error.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 7c62af0d73 typing: Add assertions before accessing settings.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Adam Sah cb6382369e merge_streams: Move logic to zerver/actions/streams.py. 2022-07-13 16:12:36 -07:00
Zixuan James Li 27af5865b0 soft_deactivation: Tighten function signatures with generic QuerySet.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:28:13 -07:00
Zixuan James Li ab1bbdda65 typing: Broaden type annotations for QuerySet compatibility.
To explain the rationale of this change, for example, there is
`get_user_activity_summary` which accepts either a `Collection[UserActivity]`,
where `QuerySet[T]` is not strictly `Sequence[T]` because its slicing behavior
is different from the `Protocol`, making `Collection` necessary.

Similarily, we should have `Iterable[T]` instead of `List[T]` so that
`QuerySet[T]` will also be an acceptable subtype, or `Sequence[T]` when we
also expect it to be indexed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-07 11:27:42 -07:00
Alex Vandiver 5529639b84 change_realm_subdomain: Support skipping the redirect.
Sometimes (e.g. when moving an old realm out of the way of an import
into that name) we do *not* wish to add a redirect realm.  Add a flag
to support that.
2022-06-25 08:44:24 -07:00
Zixuan James Li 10dc6ea39e typing: Narrow connection type before accessing pg_version.
This change is solely for removing the attribute error that mypy raises when we
 access `pg_version` on `connection`. django-stubs annotate `connection` as
`BaseDatabaseWrapper` while it is actually a proxy object, so we cannot
use an regular assertion with isinstance to narrow the type.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Zixuan James Li 3dcb4ca20a commands: Use getattr to access patched attribute during runtime.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 19:25:48 -07:00
Alex Vandiver 160a8b9945 compilemessages: Use a consistent ordering for the languages list.
This makes the generated data more consistent.
2022-06-23 16:25:33 -07:00
Zixuan James Li aebed0e57f management: Remove rename_stream.
Now that it is trivial to rename a stream in the UI, And due
to the fact that the command has been broken for 3 years unnoticed,
it is unnecessary to maintain it anymore.

Fixes #22244.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-21 12:56:54 -07:00
Anders Kaseorg a7f9c4f958 logging: Pass more format arguments to logging.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-03 12:27:23 -07:00
Mateusz Mandera 09dc166b45 do_delete_old_unclaimed_attachments: Consider ArchivedAttachment rows.
This function is oblivious to the existence of ArchivedAttachment, which
is incorrect. A file can be removed if and only if it is not referenced
by any Messages or ArchivedMessages.
2022-06-02 17:32:23 -07:00
Alex Vandiver 3d66dd9eeb fill_memcached_caches: Document possible arguments to --cache. 2022-05-22 14:52:38 -07:00
Anders Kaseorg 7acb642fa5 requirements: Upgrade to Tornado 6.
Fixes #8913.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 6fd1a558b7 runtornado: Switch to asyncio event loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg c263bfdb41 queue: Use a thread-local Pika connection.
According to the documentation: “Pika does not have any notion of
threading in the code. If you want to use Pika with threading, make
sure you have a Pika connection per thread, created in that thread. It
is not safe to share one Pika connection across threads, with one
exception: you may call the connection method add_callback_threadsafe
from another thread to schedule a callback within an active pika
connection.”

https://pika.readthedocs.io/en/stable/faq.html

This also means that synchronous Django code running in Tornado will
use its own synchronous SimpleQueueClient rather than sharing the
asynchronous TornadoQueueClient, which is unfortunate but necessary as
they’re about to be on different threads.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 5d69dafddb runtornado: Avoid deprecated IOLoop debugging methods.
IOLoop.set_blocking_log_threshold and IOLoop.handle_callback_exception
are removed in Tornado 6.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg e4bf7066f3 tornado: Remove instrument_tornado_ioloop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg bded7180f7 tornado: Unfork tornado.autoreload.
We previously forked tornado.autoreload to work around a problem where
it would crash if you introduce a syntax error and not recover if you
fix it (https://github.com/tornadoweb/tornado/issues/2398).

A much more maintainable workaround for that issue, at least in
current Tornado, is to use tornado.autoreload as the main module.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Mateusz Mandera c8fee06521 sync_ldap_user_data: Show a proper error if no relevant users exist.
It doesn't make sense to run sync_ldap_user_data if user_profiles list
is empty. Otherwise this misleading exception gets raised:

```
raise Exception(
    "LDAP sync would have deactivated all users. This is most likely due "
    "to a misconfiguration of LDAP settings. Rolling back...\n"
    "Use the --force option if the mass deactivation is intended."
)
```

With some work by tabbott to manage the type of user_profiles and
provide a special error message for the empty server case.
2022-04-18 20:32:30 +02:00
Tim Abbott f446560607 do_delete_user: Clean up acting_user logic.
This now uses our standard acting_user convention for functions called
only from management commands.
2022-04-15 15:55:21 -07:00
Anders Kaseorg e01faebd7e actions: Split out zerver.actions.create_realm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:37 -07:00
Anders Kaseorg 59f6b090c7 actions: Split out zerver.actions.realm_settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:37 -07:00
Anders Kaseorg cbad5739ab actions: Split out zerver.actions.create_user.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:35 -07:00
Anders Kaseorg 5fcbc412cf actions: Split out zerver.actions.streams.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:35 -07:00
Anders Kaseorg 975066e3f0 actions: Split out zerver.actions.message_send.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:34 -07:00