Commit Graph

47286 Commits

Author SHA1 Message Date
Greg Price ce2014bfef typeahead: Normalize spaces for sorting emojis, just like for filtering.
And add a pair of tests.  The first one (with an underscore) passed
already; the second (with a space) passes only with this change.
2022-04-21 17:59:28 -07:00
Greg Price 0f6483449f typeahead: Lower-case query in sort_emojis more consistently.
This makes this function easier to reason about, by having only one
version of the query floating around.

The change is nearly NFC: the one other place this `query` parameter
is used is the `triage` function, and that already lower-cases the
query too.

But `triage` has some additional case-related behavior: among prefix
matches (but not among exact matches), it moves any that match
case-sensitively ahead of any that don't.

As long as all emoji names are lowercase -- as all our built-in
emoji are, and as all custom emoji probably are in most realms --
that still has no effect: either the query is lowercase too and all
matches are case-sensitive matches, or it isn't and none of them are.

But it can show up if someone adds a custom emoji like `:GitHub:`
or `:LaTeX:` (like we have a `:github:` in chat.zulip.org), and then
someone does the natural thing of searching for them in lowercase.

When the behavior does show up, it seems like it can only come
across to the user as a glitch: the emoji that have capital letters
get weirdly taken out of order and moved to the end, or just don't
show up if there are more than 8 results.

In general I'm not convinced there are any situations at all where
this behavior of `triage` makes sense: basically every other
search UI in the computing universe is case-insensitive except for
some aimed at programmers searching through code, and none of our
typeahead searches are aimed at doing that.  But for the moment,
just simplify the emoji case in particular.
2022-04-21 17:59:28 -07:00
Greg Price 866a523e55 typeahead tests: Test sort_emojis with capitalized query.
We have logic that handles this.  Best to test it.
2022-04-21 17:59:28 -07:00
Greg Price 5e109aaa4d typeahead tests [nfc]: Tighten sort_emojis test data. 2022-04-21 17:59:28 -07:00
Greg Price 66f47791e4 typeahead: Test spaces equivalent to underscores in get_emoji_matcher.
We already correctly treat spaces equivalent to underscores here.
But we don't do so when we then go on to sort the same results.
(We'll be fixing that shortly.)  So it seems worth testing for it
explicitly.
2022-04-21 17:59:28 -07:00
Greg Price dc6e9df43a typeahead: Don't stop midphrase matching when a second word is typed.
For example, if a user's name is "Simon Peyton Jones", we'll already
match that name on the queries "Pey" or "Peyton", as well as on
"Simon P".  We should do so on "Peyton J" or "Peyton Jones", too.

Similarly, if the user is looking for an emoji of a face in the moon
and they start by typing ":moon", we'll show them both 🌝 "moon face"
and 🌚 "new moon face", along with some other moon-related results.
If they go on to make it ":moon " or ":moon f", though -- as one very
naturally would in order to eliminate things like "waxing moon" and
"moon ceremony" -- then we mysteriously eliminate 🌚 "new moon face".
Instead, the query "moon f" should match both 🌚 and 🌝.

Found this while comparing the web/shared implementation with the
mobile implementation of emoji search.  The new behavior here
reflects what we already do for emoji search in mobile, both in the
compose box's typeahead and in the add-a-reaction screen.  The
existing behavior here seems pretty annoying, so fixing it will be
part of switching on mobile to the shared code (zulip/zulip-mobile#4636)
without regressing the user experience.

The current behavior was introduced, more or less, in 245d65eb9; then
revised in 5edbcb87f to make the logic more clear, and a fix made in
542f4766d, all 2018.  The PR thread was #8286, following issue #8279.
The old behavior before those changes was pure substring matching,
plus a trailing space was ignored (which is the part the issue was
about.)  None of the discussion touches on this question; as far as I
can tell, the fact that "Peyton J" doesn't match "Simon Peyton Jones",
nor "moon " match "new moon face", was entirely an unintentional
side effect of those changes.
2022-04-21 17:59:28 -07:00
Greg Price 9c206ca8e5 typeahead [nfc]: Simplify query_matches_string logic.
This comment was widely separated from the code it's describing, which
is the `return source_str.includes(query)`.
2022-04-21 17:59:28 -07:00
Greg Price 6690f79c80 typeahead tests [nfc]: Split up get_emoji_matcher test function.
The better to add more cases here, and to keep them organized.
2022-04-21 17:59:28 -07:00
Greg Price 38eb3f5764 typeahead tests: Add more Unicode emoji to the test data.
We'll use these in upcoming tests.
2022-04-21 17:59:28 -07:00
Greg Price efc26eacec typeahead tests [nfc]: Simplify writing down test Unicode emoji.
This will make it more comfortable to add more.
2022-04-21 17:59:28 -07:00
Greg Price ba1b55cd59 typeahead tests: Fill in emoji_code consistently on test Unicode emoji.
Also delete a comment about what properties are used.  The comment
isn't true: `sort_emojis` looks at `emoji_code` as well as
`emoji_name`, when identifying popular emoji.

And in any case, over here in a test isn't the right place for an
interface description like that to live, if it were true -- rather,
it should be next to the code itself.  That'd make the information
more discoverable when trying to use the code, and would also
increase the chances of getting updated when things change that
would make it untrue.

(For this specific kind of interface information, of course, better
yet would be to be in machine-checkable form right on the code -- in
other words, to be a type annotation.)
2022-04-21 17:59:28 -07:00
Greg Price e91e05d187 typeahead tests [nfc]: Tighten up list of test emoji.
This will make it more comfortable to add a good bit more of these.
2022-04-21 17:59:28 -07:00
Greg Price e7858e094d typeahead tests [nfc]: Simplify expected data to just names.
This will let us stop making individual variables for the different
test emoji, which will make it more comfortable to add more of them.
2022-04-21 17:59:28 -07:00
Alex Vandiver e5548ecba0 puppet: Upgrade external dependencies. 2022-04-21 13:54:14 -07:00
yogesh sirsat c2148dc4d9 settings_bots: Clarify title of "Edit bot" button.
A bot is technically a special case of a user, in terms of how they're
stored in the database at least, but for end users, we avoid referring
to them that way.
2022-04-21 08:21:14 -07:00
Sahil Batra 77fd03e426 invite: Add custom option for invite expiration time in frontend. 2022-04-20 13:31:37 -07:00
Sahil Batra 93eb5799ef invite: Pass time value to valid_to.
This is a prep commit for adding
custom expiration option.
2022-04-20 13:31:37 -07:00
Sahil Batra 61365fbe21 invites: Use expiration time in minutes instead of days.
This commit changes the invite API to accept invitation
expiration time in minutes since we are going to add a
custom option in further commits which would allow a user
to set expiration time in minutes, hours and weeks as well.
2022-04-20 13:31:37 -07:00
Sahil Batra 6bb7f57ec2 settings: Decrease left-margin for the dependent blocks in settings UI. 2022-04-20 13:31:37 -07:00
Tim Abbott 1cf25d783c openapi: Fix Zulip version for feature level 125.
This fixes a bug introduced by
22a5d008c1 having been rebased past the
Zulip 5.0 release.
2022-04-20 12:41:45 -07:00
Lauryn Menard 2e9adad123 update_message: Use `rendering_only` flag for notifications logic.
When `update_message` events were updated to have a consistent
format for both normal message updates/edits and special
rendering preview updates, the logic used in the tornado event
queue processor to identify the special events for sending
notifications no longer applied.

Updates that logic to use the `rendering_only` flag (if present)
that was added to the `update_message` event format to identify
if the event processor should potentially send notifications to
users.

For upgrade compatibility, if `rendering_only` flag is not present,
uses previous event structure and checks for the absence of the
`user_id` property, which indicated the special rendering preview
updates.

Fixes #16022.
2022-04-20 08:24:12 -07:00
Anders Kaseorg 1db591de54 node_tests: Add test covering display_emoji_reaction_users setting.
Commit 22a5d008c1 incorrectly left this
out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-19 19:46:36 -07:00
Raghav Luthra 22a5d008c1 user_settings: Add a setting to display names of users who reacted.
Added a setting to the bottom of Settings > Display settings > Theme section
to display the reacting users on a message when numnber of reactions are
small.

This is a preparatory commit for #20980.
2022-04-19 17:30:16 -07:00
George A. Michel 6c6986e2d3 popovers: Introduce case sensitivity to move topic button behavior.
Previously, the confirm button would be disabled when the user
attempted to change the capitalization of a topic, but still keep
it in the same stream. This commit fixes this discrepancy.
2022-04-19 16:10:35 -07:00
George A. Michel ca60660dc3 popovers: Replace move topic error with conditionally disabled button.
This commit changes the behavior of the move topic modal in
'stream_popover.js'. Instead of relying on an error banner, it
will now disable the submit button whenever an identically named
topic exists in the currently selected stream. To accomplish this,
it introduces a callback, update_submit_button_disabled_state(),
and calls it in three cases:

1. When the modal is initially loaded.
2. When the dropdown is changed.
3. When the topic name is changed.

The case insensitivity of topic comparison has been preserved.

Fixes #21711.
2022-04-19 16:03:38 -07:00
Anders Kaseorg 63a1ef0e91 configure-rabbitmq: Remove use of sudo.
It already runs as root everywhere except in provision_inner, so move
the sudo there.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-19 12:36:31 -07:00
Tim Abbott f63ad171f7 i18n: Update translations from Transifex. 2022-04-18 13:50:19 -07:00
Lauryn Menard fd114cc0bf help_docs: Update link in `restrict-private-messages`.
Updates a link to point to the correct help center documentation,
and makes some small style adjustments to the page in general.
2022-04-18 12:45:27 -07:00
Mateusz Mandera c8fee06521 sync_ldap_user_data: Show a proper error if no relevant users exist.
It doesn't make sense to run sync_ldap_user_data if user_profiles list
is empty. Otherwise this misleading exception gets raised:

```
raise Exception(
    "LDAP sync would have deactivated all users. This is most likely due "
    "to a misconfiguration of LDAP settings. Rolling back...\n"
    "Use the --force option if the mass deactivation is intended."
)
```

With some work by tabbott to manage the type of user_profiles and
provide a special error message for the empty server case.
2022-04-18 20:32:30 +02:00
Sahil Batra 69a7cf9b7b streams: Check/uncheck the announce checkbox as per the previous state.
On changing the stream announce from being disabled to enabled, we
check/uncheck it based on its previous state when it was enabled.
2022-04-17 17:27:22 -07:00
Sahil Batra 2b6d2df6ff streams: Fix behavior of announce-stream checkbox in stream creation form.
This commit solves the bug which keeps the announce stream checkbox checked
for non-admin users when users are only allowed to create private streams
and not public streams.

The desired behavior is to not allow users to announce private streams, so
we keep the checkbox unchecked and disabled.

This commit fixes the above mentioned bug by removing the if-else block which
was executed after update_announce_stream_state (the function which handles
updating the checkbox considering if the realm has notifications stream or
not and whether the stream being created is public or private) and only checks
whether the realm has notification stream or not to show or hide the announce
stream checkbox irrespective of privacy of the stream being created.

This commit also fixes the handler to update the checkbox state on changing
privacy to update the checkbox state only on changing privacy value and not
on toggling the checkbox itself or changing post policy.

Fixes #21705.
2022-04-17 17:27:12 -07:00
Hannah Salameh 02305e5927 tests: Add test coverage for followup_email_delay. 2022-04-17 16:19:28 -07:00
Mateusz Mandera 6a6fd38d03 models: Reserve a RealmAuditLog.USER_DELETED_PRESERVING_MESSAGES number.
The relevant function is waiting to be merged in #21299 - but we have
already used it on Zulip Cloud, creating RealmAuditLog entries with the
number 107 and thus should reserve it before another PR takes
it for another purpose, creating confusion in the logs.
2022-04-17 16:17:58 -07:00
Tim Abbott b63757621a realm_audit_log: Set acting_user=None explicitly, with comments.
Based on an audit, this closes out the last core instances in which
acting_user was not being passed explicitly when creating
RealmAuditLog instances.

There are some outstanding issues in the billing system, which we plan
to extract as a separate issue.

Fixes #14808.
2022-04-15 15:55:21 -07:00
Tim Abbott f446560607 do_delete_user: Clean up acting_user logic.
This now uses our standard acting_user convention for functions called
only from management commands.
2022-04-15 15:55:21 -07:00
Alex Vandiver 7cc9b93b91 settings: Remove misleading and irrelevant comment.
This comment was _originally_ for the `default` memcached cache, back
when it was added all of the way back in 0a84d7ac62.  9e64750083
made it a lie, and edc718951c made it even more confusing when it
removed the `default` cache configuration block, leaving the wrong
comment next to the wrong cache configuration block.

Banish the comment.
2022-04-15 14:48:12 -07:00
Alex Vandiver 9030d53acb test_link_embed: Remove unnecessary TEST_CACHES.
The only purpose of this seems to be to not have to reset the cache;
fae59502ab added it without any explanation for why it is necessary.

Remove it, and explicitly flush the cache in the one place where it is
necessary.
2022-04-15 14:48:12 -07:00
Alex Vandiver 56058f3316 caches: Remove unnecessary "in-memory" cache.
This cache was added in da33b72848 to serve as a replacement for the
durable database cache, in development; the previous commit has
switched that to be the non-durable memcached backend.

The special-case for "in-memory" in development is mostly-unnecessary
in contrast to memcached -- `./tools/run-dev.py` flushes memcached on
every startup.  This differs in behaviour slightly, in that if the
codepath is changed and `run-dev` restarts Django, the cache is not
cleared.  This seems an unlikely occurrence, however, and the code
cleanup from its removal is worth it.
2022-04-15 14:48:12 -07:00
Alex Vandiver 04ca2e92f7 caches: Cache link preview data in memcached, not in PostgreSQL.
The choice to cache these in the database dates back to c93f1d4eda,
with the comment added in da33b72848 while working around the
durability of the "database" cache in local development.

The values were stored in a durable cache, as they needed to be
ensured to persist between when they were inserted in
`get_link_embed_data` and when they were used in
`render_incoming_message` via `link_embed_data_from_cache`.

However, database accesses are not fast compared to memcached, and we
wish to avoid the overhead of the database connection from the
`embed_links` worker.  Specifically, making the connection may not be
thread-safe -- and in low-memory (and Docker) configurations, all
workers run as separate threads in a single process.  This can lead to
stalled database connections in `embed_links` workers, and failed
previews.

Since the previous commit made the durability of the cache no longer
necessary, this will have minimal effect; at worst, posting the same
URL twice, on either side of an upgrade, will result in two preview
fetches of it.
2022-04-15 14:48:12 -07:00
Alex Vandiver 351bdfaf78 preview: Use cache only as a non-durable cache, not an IPC.
The `get_link_embed_data` / `link_embed_data_from_cache` pair as
introduced in c93f1d4eda uses the cache
as a temporary store inside of the `embed_links` worker; this means
that it must be durable storage, or the worker will stall and re-fetch
the same links to preview them.

Switch to plumbing through the fetched URL embed data as an parameter
to the Markdown evaluation which uses them, rather than using the
cache as an intermediary.  This frees up the cache to be merely a
non-durable cache.

As a side-effect, this removes get_cache_with_key, and
link_embed_data_from_cache which was its only callsite.
2022-04-15 14:48:12 -07:00
Alex Vandiver 327ff9ea0f preview: Use a dataclass for the embed data.
This is significantly cleaner than passing around `Dict[str, Any]` all
of the time.
2022-04-15 14:48:12 -07:00
Alex Vandiver ede4a88b49 populate_db: Remove unnecessary pre-population of URL cache.
76deb30312 changed this to not just be the URL, but rather a
prefixed hash of the URL, but failed to update this location which
wrote to it.  This meant that this pre-population step was writing to
the wrong keys in the durable cache, and thus ineffective.

Then, da33b72848 switched the cache to be in-memory, making this
write to the wrong keys in an in-process memory store.  There is no
way to pre-fill this sort of cache, except at server start-up.

Finally, and most fundamentally, 8c0c9ca7a4 then disabled
`inline_url_embed_preview` by default, making the code entirely moot.

Remove the triply-unnecessary code.
2022-04-15 14:48:12 -07:00
Alex Vandiver aaa58a49db cache: Make the cache_name=None behaviour clearer.
`django.core.cache.cache` is equal to
`django.core.cache.caches["default"]`; the latter is more
understandable in context.
2022-04-15 14:48:12 -07:00
Alex Vandiver 661c333377 markdown: Use named parameters to add_a helper.
This has enough parameters that it benefits from making which is which
explicit.
2022-04-15 14:48:12 -07:00
Alex Vandiver 452a30305d markdown: Clarify url parameter of "add_a" helper. 2022-04-15 14:48:12 -07:00
Mateusz Mandera fcf82bf047 digest: Don't send emails to deactivated users, even if queued. 2022-04-15 14:32:55 -07:00
Mateusz Mandera 7a13836d26 test_digest: Fix typo in a comment. 2022-04-15 14:32:55 -07:00
Anders Kaseorg cc30ed8ec7 actions: Delete zerver.lib.actions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:38 -07:00
Anders Kaseorg 729019acdd actions: Move part into zerver.lib.test_classes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:37 -07:00
Anders Kaseorg e01faebd7e actions: Split out zerver.actions.create_realm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-14 17:14:37 -07:00