Commit Graph

15913 Commits

Author SHA1 Message Date
Mateusz Mandera 470c0458e6 do_deactivate_user: Use .on_commit around send_event calls.
The previous commit did this for revoking sessions. send_events should
be handled similarly too, to correctly handle calling do_deactivate_user
inside a transaction.
2022-10-11 10:49:07 -07:00
Mateusz Mandera a94b2572be do_deactivate_user: Revoke sessions in transaction.on_commit().
Fixes #21709.
2022-10-11 10:49:07 -07:00
Anders Kaseorg 6d19256877 notes: Separate __notes_map per-subclass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:42:13 -07:00
Anders Kaseorg 1385a827c2 python: Clean up getattr, setattr, delattr calls with literal names.
These were useful as a transitional workaround to ignore type errors
that only show up with django-stubs, while avoiding errors about
unused type: ignore comments without django-stubs.  Now that the
django-stubs transition is complete, switch to type: ignore comments
so that mypy will tell us if they become unnecessary.  Many already
have.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:40:28 -07:00
Anders Kaseorg 5e1ebf2861 python: Fix __dict__ mutation abuse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:33:08 -07:00
Anders Kaseorg b45484573e python: Use format string for logging str(obj).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00
Anders Kaseorg fcd81a8473 python: Replace avoidable uses of __special__ attributes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00
Mateusz Mandera cefed552f6 test_slack_importer: Add assertion about message count.
This will help catch any future regression that might lead the import
tool to fail to import messages into the correct realm.
2022-10-07 10:10:01 -07:00
Mateusz Mandera 42b3575183 models: Make Message.realm and ArchivedMessage.realm non-null.
In the previous commits, these have been backfilled, so now we can
enforce non-nullable.
2022-10-07 10:10:01 -07:00
Mateusz Mandera c1e701ec22 migrations: Add migration to backfill Message.realm. 2022-10-07 10:10:01 -07:00
Mateusz Mandera 00b3546c9f models: Add denormalized .realm column to Message.
This commit adds the OPTIONAL .realm attribute to Message
(and ArchivedMessage), with the server changes for making new Messages
have this set. Old Messages still have to be migrated to backfill this,
before it can be non-nullable.

Appropriate test changes to correctly set .realm for Messages the tests
manually create are included here as well.
2022-10-07 10:09:38 -07:00
Mateusz Mandera a4008d938a test_soft_deactivation: Add docstring explaining send_fake_message. 2022-10-07 10:09:38 -07:00
Anders Kaseorg 133c8b16ed requirements: Upgrade openapi-core.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-07 10:03:51 -07:00
Anders Kaseorg ba3aefc3d1 openapi: Set x-model: dict for objects used as associative arrays.
This works around an openapi-core bug.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-07 10:03:51 -07:00
Sahil Batra 2bf70fe4db custom_profile_field: Add "Pronouns" custom field type.
This commit adds "Pronouns" custom profile field type. We also
add "Pronouns" type field in the development environment
2022-10-06 17:56:26 -07:00
Anders Kaseorg 92ad4455ed requirements: Upgrade Django to 4.1.
zerver/migrations/0240_usermessage_migrate_bigint_id_into_id.py needs
to be updated to account for Django 4.1 creating AutoField as an
identity column rather than a serial column.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:59:07 -07:00
Anders Kaseorg 67cbb21267 migrations: Fix UserMessage.id sequence calculation in 0240.
The sequence value should reflect the last id, not the next id, to
avoid leaving a gap of 1.  Also, it should take ArchivedUserMessage.id
into account to avoid collisions during future archiving.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:59:07 -07:00
Anders Kaseorg 8230324068 markdown: Store ZulipMarkdown in members with the right type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg 2bd81dd5c9 fenced_code: Avoid sloppy AttributeError handler.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg 3cf91e9e45 markdown: Rename our Markdown subclass to ZulipMarkdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg 97be895cf0 markdown: Remove Optional from zulip_rendering_result type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg d01c99d2ee markdown: Add missing None check in InlineInterestingLinkProcessor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg eb9957aa35 subdomains: Fix realm=None case for is_static_or_current_realm_url.
Fixes #22636.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg ad2795698b models: Remove explicit id fields.
With django-stubs, these explicit copies of Django’s implicit id
fields are no longer needed for type checking.  An exception is the
BigAutoField AbstractUserMessage.id, which is left alone.

This reverts commit c08ee904d8 (#15641).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 13:49:11 -07:00
Anders Kaseorg 47c5deeccd python: Mark dict parameters with defaults as read-only.
Found by semgrep 0.115 more accurately applying the rule added in
commit 0d6c771baf (#15349).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 13:48:28 -07:00
Zixuan James Li 7fd8d77ce0 typing: Import ValuesQuerySet alias from django_stubs_ext.
This saves us from using a conditional import.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
Zixuan James Li a4eaa770f0 typing: Import StrPromise alias from django_stubs_ext.
This saves us from using a conditional import.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
PIG208 df18bbbd48 requirements: Add django-stubs and configure plugin.
Note that django_stubs_ext is required to be placed within common.in
because we need the monkeypatched types in runtime; django-stubs
itself is for type checking only.

In the future, we would like to pin to a release instead of a git
revision, but several patches we've contributed upstream have not
appeared in a release yet.

We also remove the type annotation for RealmAuditLog.event_last_message_id
here instead of earlier because type checking fails otherwise.

Fixes #11560.
2022-10-05 16:15:56 -07:00
Zixuan James Li 4c3c976174 models: Implicitly type model fields with django-stubs.
Previously, we type the model fields with explicit type annotations
manually with the approximate types. This was because the lack of types
for Django.

django-stubs provides more specific types for all these fields that
incompatible with our previous approximate annotations. So now we can
remove the inline type annotations and rely on the types defined in the
stubs. This allows mypy to infer the types of the model fields for us.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
Zixuan James Li b28949b9e7 typing: Use django-stubs' type annotations for QuerySet.
We no longer need to annotate the type of objects returned
from queries since django-stubs plugin infers that already.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
evykassirer 6a90a4b27c emoji: Switch users with blob emoji to use Google style. 2022-10-04 12:29:35 -07:00
evykassirer c321f57785 settings: Explain that Google blob emojis are deprecated.
Part of fixing #19371.
2022-10-04 12:29:35 -07:00
evykassirer 8a9e68e026 emoji: Finish script to generate emoji_names.py with CLDR data.
This script pulls from our previously custom-written emoji strings
and fills in the rest from CLDR. It also removes 4 custom emoji which
collide with some of the new CLDR names (they will now just be called
by their CLDR name).
2022-10-04 12:29:35 -07:00
Anders Kaseorg 7dcffca50e tornado: Construct Django BaseHandler once, not per-request.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-30 11:20:45 -07:00
Anders Kaseorg 4c808026b2 tests: Disable lazy loading of moto.
This works around some regression in moto 1.3.15 that I bisected to
b8820009e8
where ‘tools/test-backend test_transfer’ fails when run by itself.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-30 11:19:39 -07:00
Zixuan James Li e16de8d9e7 scim: Further slim down SCIMClient removing unused attributes.
This removes everything from SCIMClient except the "is_authenticated`
method. Previously, "realm" and "name" were only needed for logging
purposes. It is the best to keep SCIMClient as minimal as possible, as
it is only intended to be used for authenticating requests to SCIM
views.

This change also gurantees that the "LogRequests" middleware will not
rely on the type unsafe access of the format_requestor_for_logs method
on SCIMClient.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-30 15:34:50 +02:00
Zixuan James Li a9273c9aac scim: Remove unused test case attribute.
The scim_client attribute on SCIMTestCase is currently unused since
9198fe4fac.

The creation of the SCIMClient instance was previously needed because
zerver.middleware.validate_scim_bearer_token returns SCIMClient from the db.
The attribute itself on the was never really used in the test case.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-30 15:34:50 +02:00
Anders Kaseorg e00f883263 test_runner: Remove empty ParallelTestSuite.__init__ override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-29 18:43:24 -07:00
khanhdq ee2141b0f1 email_notifications: Change missed message for wildcard mention.
Add #stream_name to wildcard mention because it is important
information for interpreting the wildcard mention (larger streams may
mean something very different to you than small ones).

Fixes #22885.
2022-09-29 15:55:14 -07:00
khanhdq bb8365ab36 email_notifications: Add ``realm_name`` to email notifications.
Add {{ realm_name }} to the "Reply to this email directly ..." line.

This ensures the realm name is always present in the email
notification footer area, in a consistent location.
2022-09-29 15:54:35 -07:00
khanhdq 59d0f9b301 email_notifications: Highlight personal mentions in explanations.
Change the explanations for why notifications were received, with
different text for personal and wildcard mentions.
2022-09-29 15:54:21 -07:00
Sahil Batra 04693b6ac1 message_edit: Send only changed settings in event data and api response.
Previously, we included all three message edit related settings
("allow_message_editing", "message_content_edit_limit_seconds" and
"edit_topic_policy") in the event data and api response irrespective
of which of these settings were changed. Now, we only include changed
settings and separate events are sent for each setting if more than
one of them is changed.

Note that the previous typed in event_schema.py for
`message_content_edit_limit_seconds` incorrectly did not allow `None`
as a value, which is used to encode no limit.
2022-09-28 11:47:40 -07:00
Mateusz Mandera 522c159441 test_push_notifs: Change type_id arg of get_message to no default.
This isn't used anywhere, nor would type_id 100 make much sense.
2022-09-28 16:45:34 +02:00
Mateusz Mandera b35ad99035 test_push_notifications: Use proper user.id for Recipient type_id.
Recipient with type PERSONAL type_id 1 is a Recipient for a system bot,
since those get created first. Even if it doesn't break tests, it's
still bad, because it is not the intention of those tests to simulate a
cross-realm private message to a system bot.
2022-09-28 16:45:34 +02:00
Mateusz Mandera 5850c38f4e test_digest: Use proper stream.id in test_get_hot_topics.
Just using values 1 and 2 as stream ids is not good, because there's no
idea in which realm these streams are (or hypothetically if they exist).
This can create weird Messages with sender being a user of "zulip" realm
and the stream being in another realm - which would be a corrupted
state.
2022-09-28 16:45:25 +02:00
Kartik Srivastava 409ef040bb stream_topic: Refactor user_ids_muting_topic.
This refactors and renames user_ids_muting_topic to accept a parameter
'visibility_policy' and fetch user IDs that have a specific
visibility_policy(provided as the parameter) set for a topic.
2022-09-27 17:18:48 -07:00
Anders Kaseorg 8eec4bf171 message_fetch: Move narrowing query builder to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-27 17:02:10 -07:00
Aman Agrawal b84722de01 events: Clean up logic for spectator events_register parameters.
Unfortunately, doing so requires forking common API documentation
text, since we're not making any changes to other endpoints that don't
allow unauthenticated requests at all.

Follow-up on #21995.
2022-09-27 16:12:59 -07:00
Mateusz Mandera 2811a1228f import_util: Make build_message only take kwargs.
build_message has a lot of arguments, so it's hard to verify correctness
of callers that just try to get the order right. It's much clearer to be
explicit via kwargs. mattermost.py and rocketchat.py already do this, so
let's bring slack.py and gitter.py up to par.
2022-09-27 15:04:48 -07:00
Mateusz Mandera e360b5e29b export: Remove unnecessary if in export with consent code.
This might be a bit cleaner.
2022-09-27 11:56:27 -07:00