Commit Graph

7319 Commits

Author SHA1 Message Date
Anders Kaseorg 69e94b5991 ruff: Fix C413 Unnecessary `list` call around `sorted()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Mateusz Mandera 6e336ef6f6 delete_topic: Use the same timeout pattern as /mark_all_as_read.
We don't want to use the nginx 502 timeout as an API pattern. In
/mark_all_as_read we've already figured out how we want to handle this
instead.
2022-11-02 16:50:06 -04:00
Zixuan James Li da9ad7c3a7 markdown: Pass realm down into render_stream_description.
This guarantees that the Realm is always non-None when we hit the
codepath is_static_or_current_realm_url via
do_change_stream_description, so that we can properly skip rewritting
some images.

Fixes #19405

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-11-01 08:20:47 -07:00
Lauryn Menard 0c947985bd tests: Update tests to use example profile picture.
So that we can stop using Tim's photo for tests, adds an open
license profile picture to use instead.

Updates tests that used `tim.png` to use the new example profile
picture, which is located in `static/images/test-images/avatars/`.
2022-10-31 14:36:54 -07:00
Mateusz Mandera 760587450e retention: Remove two redundant comments.
These two identical comments don't contribute anything useful and seem
just out of place at this point.
2022-10-31 10:23:57 -07:00
Mateusz Mandera 7b13204e8f retention: Use Message.realm to simplify private message query.
We no longer need to do the inner joins to figure out the message's
realm and split up the cross-realm and regular case - now we just look
at zerver_message.realm directly.
2022-10-31 10:23:57 -07:00
Mateusz Mandera 601f0677f3 update_first_visible_message_id: Use Message.realm column.
With the Message.realm column added, this query can now be improved.
2022-10-31 09:57:42 -07:00
Sahil Batra 2a8b7412ff url_encoding: Use user name instead of email in personal_narrow_url.
This commit updates the urls for personal narrow sent in email
notifications to be of form "{user_id}-{encoded_full_name}" to
make it consistent with the urls that we use for such narrows
in webapp which were recently updated in b4eddad for improving
performance. We encode the full name in the same way that we do in
webapp by replacing the url characters encoded by browser with "-".
2022-10-28 09:47:43 -07:00
Mateusz Mandera a410f6b241 do_mark_all_as_read: Split up the work into batches.
Fixes #15403.
2022-10-27 16:59:54 -07:00
Alya Abbott 8f6cf65f28 help center: Reorganize and extend the "Logging in" help page.
Also move some instructions from related pages into shared /include files,
with minor tweaks.
2022-10-26 12:28:29 -07:00
Lauryn Menard e66f125ee6 recent-topics: Update hash for view to be "#recent".
Updates the hash used for the recent conversations view to be
"#recent" instead of "#recent_topics".

We will need to keep the logic for handling "#recent_topics"
permanently because users potentially have messages from
Welcome Bot with links to that hash.

Including "recent_topics" as a web_public_allowed_hash in
hash_util.js can be changed once self-hosted servers cannot
upgrade directly to Zulip 5.x from the current version.

Fixes #23132.
2022-10-25 11:03:44 -07:00
Prakhar Pratyush 38aa1ee1ff test_classes: Improve assert_database_query_count output message.
Output message should talk about both the cases:
actual_count > expected_count and actual_count < expected_count.

The message now includes information for the case where
actual_query_count < expected_query_count.

Fixes: #23325
2022-10-24 10:18:58 -07:00
Lauryn Menard c9044f7acd recent-topics: Rename to "Recent conversations" in web-app and docs.
Replaces instances of "recent topics" in the web-app and documentation
to be "recent conversations".

Renames both `recent-topics.md` files in the help center to be
`recent-conversations.md` and updates/redirects links to new URL.

Does not update instances of "recent topics" in frontend code comments
and does not update the main overview changelog, for now.

Does not change case study text where "recent topics" was referenced
in a quote, but does change generic text references to be "recent
conversations".
2022-10-21 10:28:19 -07:00
Anders Kaseorg c5c180fda3 email_validation: Restore case-insensitive domain validation.
This was broken by commit b945aa3443
(#22604), because email_to_domain implicitly lowercased the result.

No adjustment is needed for is_disposable_domain, which already
lowercases its argument.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-20 08:19:07 -07:00
Anders Kaseorg 89e4233b68 timeout: Correct thread id type passed to PyThreadState_SetAsyncExc.
This type changed in Python 3.7:
https://github.com/python/cpython/pull/781

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-19 14:49:24 -07:00
David Rosa 64c31b7cb9 help: Rename "Add custom profile fields" -> "Custom profile fields".
Renames the help article on custom profile fields to reflect that
its content is not just about adding fields.

Adds a redirect from the old URL to the new URL and updates internal
links, linking to #add-a-custom-profile-field where appropriate.

Fixes #23170.
2022-10-19 11:58:00 -07:00
David Rosa d9dd272a78 help: Update documentation on bots.
Adds new tab to `zerver/lib/markdown/tabbed_sections.py` to document
managing bots from both personal settings and organization settings.

Documents adding bots from the organization settings Bots panel.

Separates instructions for deactivating and reactivating a bot from
both personal settings and organization settings.

Fixes a few formatting issues such as missing bold formatting and
heading level.

Fixes: #23066.
2022-10-18 10:16:42 -07:00
Florian Pritz f37ac80384 import_realm: Speed up first_message_id calculation.
On my data (about 10 million messages in 1600 streams) this used to take
about 40 hours, while the improved statement completes in roughly 30
seconds.

The old solution had postgres go through the entire table until the
first match for each stream. Thus, the time spent scanning the table
got longer and longer for each stream because postgres always started at
the beginning (and somehow it did not use any indices) and had to skip
over all rows until it found the first message from the stream that is
was looking for each time.

This new statement just performans a bulk operation, scanning the table
only once and then inserts the results directly into the destination
table.

Slightly more verbose inforation about this change can be found in:
https://chat.zulip.org/#narrow/stream/31-production-help/topic/Import.20Rocketchat.20data/near/1408867

Signed-off-by: Florian Pritz <bluewind@xinu.at>
2022-10-17 11:43:21 -07:00
Zixuan James Li 46329a2710 test_classes: Create a dedicate helper for query count check.
This adds a helper based on testing patterns of using the "queries_captured"
context manager with "assert_length" to check the number of queries
executed for preventing performance regression.

It explains the rationale of checking the query count through an
"AssertionError" and prints the queries captured as assert_length does,
but with a format optimized for displaying the queries in a more
readable manner.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-17 11:32:52 -07:00
Zixuan James Li 9117db8887 test_helpers: Tighten type annotation for queries_captured.
This adds CapturedQueryDict to provide a more accurate type annotation
for the return value of queries_captured. We also replace "Generator"
with "Iterator" because the latter two type parameters were unused.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-17 11:32:52 -07:00
Lauryn Menard 171da34546 events: Send empty list for custom_profile_fields in spectator view.
Send an empty list of `custom_profile_fields` in `page_params` for
spectators, rather than not sending the field at all.

Also, updates the user info popover to not show the manage user
three-dot menu when in a spectator view.
2022-10-14 13:05:35 -07:00
Alex Vandiver c328de3372 cache: Log a warning when attempting to store a whole QuerySet.
As noted in the previous commit, this causes bloat in memcached, for
no purpose.  Log a warning when `cache_with_key` sees a QuerySet
returned from the function it is decorating.
2022-10-12 22:25:48 -07:00
Alex Vandiver 204f1b58e8 cache: Drop realm_id from `realm_user_dict_fields`.
Storing this key is superfluous, as it will be the same for all users,
and definitionally already known to fetch the cache for the realm.  It
is also not currently used by the callsites that read rows from the
cache.
2022-10-12 22:25:48 -07:00
Alex Vandiver 5d42a0cb00 linkifiers: Support %20 in URLs for topic links.
9381a3bd45 added support for linkifier pattern URLs containing
`%20`-style escapes, but only did so for the codepath which is used in
the message body -- topic links did not understand them.

Expand the support to include when they are substituted into topics.
2022-10-11 14:31:13 -07:00
Anders Kaseorg 6d19256877 notes: Separate __notes_map per-subclass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:42:13 -07:00
Anders Kaseorg 1385a827c2 python: Clean up getattr, setattr, delattr calls with literal names.
These were useful as a transitional workaround to ignore type errors
that only show up with django-stubs, while avoiding errors about
unused type: ignore comments without django-stubs.  Now that the
django-stubs transition is complete, switch to type: ignore comments
so that mypy will tell us if they become unnecessary.  Many already
have.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:40:28 -07:00
Anders Kaseorg b45484573e python: Use format string for logging str(obj).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00
Anders Kaseorg fcd81a8473 python: Replace avoidable uses of __special__ attributes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00
Mateusz Mandera 00b3546c9f models: Add denormalized .realm column to Message.
This commit adds the OPTIONAL .realm attribute to Message
(and ArchivedMessage), with the server changes for making new Messages
have this set. Old Messages still have to be migrated to backfill this,
before it can be non-nullable.

Appropriate test changes to correctly set .realm for Messages the tests
manually create are included here as well.
2022-10-07 10:09:38 -07:00
Anders Kaseorg 92ad4455ed requirements: Upgrade Django to 4.1.
zerver/migrations/0240_usermessage_migrate_bigint_id_into_id.py needs
to be updated to account for Django 4.1 creating AutoField as an
identity column rather than a serial column.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:59:07 -07:00
Anders Kaseorg 8230324068 markdown: Store ZulipMarkdown in members with the right type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg 2bd81dd5c9 fenced_code: Avoid sloppy AttributeError handler.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg 3cf91e9e45 markdown: Rename our Markdown subclass to ZulipMarkdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg 97be895cf0 markdown: Remove Optional from zulip_rendering_result type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg d01c99d2ee markdown: Add missing None check in InlineInterestingLinkProcessor.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg eb9957aa35 subdomains: Fix realm=None case for is_static_or_current_realm_url.
Fixes #22636.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:15:10 -07:00
Anders Kaseorg 47c5deeccd python: Mark dict parameters with defaults as read-only.
Found by semgrep 0.115 more accurately applying the rule added in
commit 0d6c771baf (#15349).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 13:48:28 -07:00
Zixuan James Li 7fd8d77ce0 typing: Import ValuesQuerySet alias from django_stubs_ext.
This saves us from using a conditional import.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
Zixuan James Li a4eaa770f0 typing: Import StrPromise alias from django_stubs_ext.
This saves us from using a conditional import.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
PIG208 df18bbbd48 requirements: Add django-stubs and configure plugin.
Note that django_stubs_ext is required to be placed within common.in
because we need the monkeypatched types in runtime; django-stubs
itself is for type checking only.

In the future, we would like to pin to a release instead of a git
revision, but several patches we've contributed upstream have not
appeared in a release yet.

We also remove the type annotation for RealmAuditLog.event_last_message_id
here instead of earlier because type checking fails otherwise.

Fixes #11560.
2022-10-05 16:15:56 -07:00
Zixuan James Li b28949b9e7 typing: Use django-stubs' type annotations for QuerySet.
We no longer need to annotate the type of objects returned
from queries since django-stubs plugin infers that already.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
Anders Kaseorg 4c808026b2 tests: Disable lazy loading of moto.
This works around some regression in moto 1.3.15 that I bisected to
b8820009e8
where ‘tools/test-backend test_transfer’ fails when run by itself.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-30 11:19:39 -07:00
Anders Kaseorg e00f883263 test_runner: Remove empty ParallelTestSuite.__init__ override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-29 18:43:24 -07:00
khanhdq 59d0f9b301 email_notifications: Highlight personal mentions in explanations.
Change the explanations for why notifications were received, with
different text for personal and wildcard mentions.
2022-09-29 15:54:21 -07:00
Sahil Batra 04693b6ac1 message_edit: Send only changed settings in event data and api response.
Previously, we included all three message edit related settings
("allow_message_editing", "message_content_edit_limit_seconds" and
"edit_topic_policy") in the event data and api response irrespective
of which of these settings were changed. Now, we only include changed
settings and separate events are sent for each setting if more than
one of them is changed.

Note that the previous typed in event_schema.py for
`message_content_edit_limit_seconds` incorrectly did not allow `None`
as a value, which is used to encode no limit.
2022-09-28 11:47:40 -07:00
Kartik Srivastava 409ef040bb stream_topic: Refactor user_ids_muting_topic.
This refactors and renames user_ids_muting_topic to accept a parameter
'visibility_policy' and fetch user IDs that have a specific
visibility_policy(provided as the parameter) set for a topic.
2022-09-27 17:18:48 -07:00
Anders Kaseorg 8eec4bf171 message_fetch: Move narrowing query builder to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-27 17:02:10 -07:00
Aman Agrawal b84722de01 events: Clean up logic for spectator events_register parameters.
Unfortunately, doing so requires forking common API documentation
text, since we're not making any changes to other endpoints that don't
allow unauthenticated requests at all.

Follow-up on #21995.
2022-09-27 16:12:59 -07:00
Mateusz Mandera e360b5e29b export: Remove unnecessary if in export with consent code.
This might be a bit cleaner.
2022-09-27 11:56:27 -07:00
Mateusz Mandera 318d7fd4cd export: Only export messages that a consenting user can access.
As mentioned in the TODO this commit deletes, the export with member
consent system was failing to account for the fact that if consenting
users only have access to a subset of messages of a stream with
protected history, only that subset should be exported - rather than all
the stream's messages.
2022-09-27 11:56:27 -07:00