Fixes#23517.
While this feature was added to Zulip very early, it has been troubled
for most of that time; it never looked great visually, had a lot of
implementation complexity around resize.js, and has a weird model (a
setting that changes the UI only in certain window sizes).
This option is not commonly used; while a significant portion of users
have it enabled, many of them just don't use window sizes where it
actually has an effect. So it's not clear that it will be missed if
removed; we got very few bug reports when it was completely broken for
a few days after we first integrated the new left sidebar private
messages design.
Even with it no longer being broken, it does not work very well with
the addition of the new PMs section in the left sidebar. (Having two
scrollbars in the sidebar looks quite awkward.) The new private
messages section in the left sidebar also addresses some of the use
cases for always keeping the Users list always visible, even in narrow
windows.
This option is only removed from frontend for now. To make this
decision easily reversible, the backend code of this feature
is still kept.
This doesn't make sense if the realm is active and will fail as soon as
do_reactivate_realm is fixed in the next commit to be a noop and not
create confused RealmAuditLog entries when the realm is active.
There was the following bug here:
1. Send an email invite to a user.
2. Have the user sign up via social auth without going through that
invite, meaning either going via a multiuse invite link or just
straight-up Sign up if the org permissions allow.
That resulted in the PreregistrationUser that got generated in step (1)
having 2 Confirmations tied to it - because maybe_send_to_registration
grabbed the object and created a new confirmation link for it. That is a
corrupted state, Confirmation is supposed to be unique.
One could try to do fancy things with checking whether a
PreregistrationUser already have a Confirmation link, but to avoid races
between ConfirmationEmailWorker and maybe_send_to_registration, this
would require taking locks and so on - which gets needlessly
complicated. It's simpler to not have them compete for the same object.
The point of the PreregistrationUser re-use in
maybe_send_to_registration is that if an admin invites a user, setting
their initial streams and role, it'd be an annoying experience if the
user ends up signing up not via the invite and those initial streams
streams etc. don't get set up. But to handle this, we can just copy the
relevant values from the pre-existing prereg_user, rather than re-using
the object itself.
This line was added in 94e099eaab,
presumably because of the
del request.session["multiuse_object_key"]
line that was just above it.
Looks like it should have been removed in
868a763cec, which eliminated that `del`
operation.
This is primarily for administrators needing to provide message
history for compliance or auditing purposes. Search terms can be
pulled from a file, one per line, or from arguments provided on the
command line.
In 1fce1c3c73, we added logic to parse
the User-Agent in /register requests; this logic crashed if an HTTP request
was missing that header.
Includes a test for `/register` with no user agent passed; this should catch
similar regressions in the future.
Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
For PATCH requests to `/realm/profile_fields`, the field_type is
determined via the field.id, which is in the URL. Therefore,
`field_type` does not need to be passed as part of the request
data in tests.
We do not create historical UserMessage rows, for messages that didn't
have one, while marking messages as read and simply ignore those messages.
We do so because there is no user of creating UserMessage rows and it just
wastes storage.
Note that we still allow to mark messages from unsubscribed streams as
read but only those which have UserMessage rows for them to handle the
case when the unread messages were not marked as read while unsubscribing
from the stream due to some race condition. In such cases, messages
will not be included in the unread count shown in "All messages" menu
(and stream is anyways not present in the left sidebar), but the message
border on the left is green if viewing the stream after unsusbcribing it.
So, to avoid the confusion for users, the messages will be marked as read
when user scrolls down.
This unifies the length of the shortened SHA our integrations generate,
and ensures that they are long enough for projects of various sizes with
a chosen value defined in get_short_sha.
Fixes#23475
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
7 characters are not enough for large projects, so we change
it to reasonably longer. As an example, The Linux kernel needs
at least 11 characters of sha in its shortened form to identify
a revision. We pick 11 so it should work for most of the projects.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We use `templates/zerver/api/api-doc-template.md` as a base template
for the documented API endpoints in `zerver/openapi/zulip.yaml`.
Previously, if this template was called as an endpoint, then it
would fail an assertion check and send server error. Now we check
for specifically for that potential path and return a 404 error
response for no existing article.
Fixes#21876.
Prior to 53231aa, the `ignore_unhashable_lru_cache` decorator had
a check for the development environment so that changes could be
seen on refresh.
Puts that check back in IgnoreUnhashableLruCacheWrapper class.
In the outgoing webhook handler, there is potentially several seconds
of trying between when a message triggering an outgoing webhook
arrives, and when it fails. In the meantime, the stream the
triggering message was on may have been deleted, causing the
"Failure!" message to have no valid stream to be sent to.
Rather than raise an exception in the outgoing webhook worker, ignore
the exception and move on.
The lambda passed to `queue_json_publish` is used if
`settings.USING_RABBITMQ` is unset -- which is only true in tests. As
such, this pattern causes failures to never actually retry within
tests.
This behaviour has existed ever since the outgoing webhook code was
introduced in 53a8b2ac87, with no explanation. Not passing that
argument allows tests to verify the retry behaviour when webhooks
fail.
Previously, test cases or clients accessing /json/ views using HTTP
Basic Auth would be accepted, while we intended to only allow clients
authenticated with a session cookie to access these views.
This adds a check on the accessed path to avoid this possibility.
It seems unlikely that any API clients clients were taking advantage
of this unintended quirk; so we're not going to bother documenting
this bug fix as an API change. In any case, it should be trivial for
anyone affected to consult the documentation and then switch their
/json/foo URL to a correct /api/v1/foo URL.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Mobile clients older than v27.192 do not support PRONOUNS type
custom profile fields, so we instead change the type of it to
SHORT_TEXT in the data sent with register response and also in
the events sent to those clients.
On multi-realm systems this results in traversal of all messages in
all realms and returns a massive payload of 1 row per stream on
the server, not the intended one row per realm.
Zulip's unread messages design has an invariant that all unread stream
messages must be in streams the user is subscribed to. For example, We
do not include the unread messages from unsubscribed streams in the
"unread_msgs" data structure in "/register" response and we mark all
unread messages as read when unsubscribing a user from a stream.
Previously, the mark as unread endpoint allowed violating that
invariant, allowing you to mark messages in any stream as unread.
Doing so caused the "message_details" data structures sent with
"update_message_flags" events to not contain messages from
unsubscribed streams, even though those messages were present in the
set of message IDs. These malformed events, in turn, caused exceptions
in the frontend's processing of such an event.
This change is paired with a separate UI change to not offer the "Mark
as unread" feature in such streams; with just this commit, that will
silently fail.
With some additions to the tests by tabbott.
This guarantees that the Realm is always non-None when we hit the
codepath is_static_or_current_realm_url via
do_change_stream_description, so that we can properly skip rewritting
some images.
Fixes#19405
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The previous error page was inadequate for serving the two different
scenarios where we show errors in realm_creations, in particular
containing a misleading sentence about realm creation being disabled
(even in the case where it was actually enabled and the user simply had
an expired link).
So that we can stop using Tim's photo for tests, adds an open
license profile picture to use instead.
Updates tests that used `tim.png` to use the new example profile
picture, which is located in `static/images/test-images/avatars/`.
Creates `static/images/authentication_backends` directory for icons
of backend authentication methods, which are used on the log-in page.
And updates the example documentation in the API `/server_settings`
endpoint.
Updates the markdown test case that used `zulip-octopus.png` to
instead use an zulip logo that's also referenced in a frontend
puppeteer test, `static/images/logo/zulip-icon-128x128.png`.
The query in display_in_profile_summary_limit_reached should check
realm also since there is per-realm limit of 2 fields, otherwise
this will cause issues where multiple realms are hosted on the
same server.
Fixes#23368.
We no longer need to do the inner joins to figure out the message's
realm and split up the cross-realm and regular case - now we just look
at zerver_message.realm directly.
I don't think this is used anywhere outside of tests, but we should have
this logic correct. If this function is used to send a message from a
user to a cross-realm bot, the message.realm should be the realm of the
user.
In the normal case, where a user send a message to a cross-realm bot
through the API is already handled correctly, this bug is unrelated.
Clarifies most of the narrow parameter descriptions by adding
information about what a user's message history includes, about
new bot users not generally being subscribed to streams, and
about the specific `streams:public` narrow.
Updates the main descriptions for the `/get-messages` and
`/check-messages-match-narrow` endpoints.
Fixes#19477.
This commit updates the urls for personal narrow sent in email
notifications to be of form "{user_id}-{encoded_full_name}" to
make it consistent with the urls that we use for such narrows
in webapp which were recently updated in b4eddad for improving
performance. We encode the full name in the same way that we do in
webapp by replacing the url characters encoded by browser with "-".
It doesn't seem to make sense to append _{number} to the status code in
that arg, because the resulting string stops looking like a status code
and actually makes this test fail in the follow-up commit with the
confusing error message of
Unknown response http status: 2000
So this just seems like a bug.
Updates the hash used for the recent conversations view to be
"#recent" instead of "#recent_topics".
We will need to keep the logic for handling "#recent_topics"
permanently because users potentially have messages from
Welcome Bot with links to that hash.
Including "recent_topics" as a web_public_allowed_hash in
hash_util.js can be changed once self-hosted servers cannot
upgrade directly to Zulip 5.x from the current version.
Fixes#23132.
Output message should talk about both the cases:
actual_count > expected_count and actual_count < expected_count.
The message now includes information for the case where
actual_query_count < expected_query_count.
Fixes: #23325
Replaces instances of "recent topics" in the web-app and documentation
to be "recent conversations".
Renames both `recent-topics.md` files in the help center to be
`recent-conversations.md` and updates/redirects links to new URL.
Does not update instances of "recent topics" in frontend code comments
and does not update the main overview changelog, for now.
Does not change case study text where "recent topics" was referenced
in a quote, but does change generic text references to be "recent
conversations".
This was broken by commit b945aa3443
(#22604), because email_to_domain implicitly lowercased the result.
No adjustment is needed for is_disposable_domain, which already
lowercases its argument.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Renames the help article on custom profile fields to reflect that
its content is not just about adding fields.
Adds a redirect from the old URL to the new URL and updates internal
links, linking to #add-a-custom-profile-field where appropriate.
Fixes#23170.
Adds new tab to `zerver/lib/markdown/tabbed_sections.py` to document
managing bots from both personal settings and organization settings.
Documents adding bots from the organization settings Bots panel.
Separates instructions for deactivating and reactivating a bot from
both personal settings and organization settings.
Fixes a few formatting issues such as missing bold formatting and
heading level.
Fixes: #23066.
On my data (about 10 million messages in 1600 streams) this used to take
about 40 hours, while the improved statement completes in roughly 30
seconds.
The old solution had postgres go through the entire table until the
first match for each stream. Thus, the time spent scanning the table
got longer and longer for each stream because postgres always started at
the beginning (and somehow it did not use any indices) and had to skip
over all rows until it found the first message from the stream that is
was looking for each time.
This new statement just performans a bulk operation, scanning the table
only once and then inserts the results directly into the destination
table.
Slightly more verbose inforation about this change can be found in:
https://chat.zulip.org/#narrow/stream/31-production-help/topic/Import.20Rocketchat.20data/near/1408867
Signed-off-by: Florian Pritz <bluewind@xinu.at>
This adds a helper based on testing patterns of using the "queries_captured"
context manager with "assert_length" to check the number of queries
executed for preventing performance regression.
It explains the rationale of checking the query count through an
"AssertionError" and prints the queries captured as assert_length does,
but with a format optimized for displaying the queries in a more
readable manner.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This adds CapturedQueryDict to provide a more accurate type annotation
for the return value of queries_captured. We also replace "Generator"
with "Iterator" because the latter two type parameters were unused.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We followed a same approach as in #22611 to mark migrations as noops. We
might eventually squash them.
Migration operations that only change the validators should be removed
as they are essentially noops that do not affect the database schema.
However, ./manage.py makemigration still generates a new migration for
validators change regardless. So we still have to keep one migration
that updates the validators to the latest state. We prefer to keep the
earliest one for this purpose.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
These used to only be shown conditional on the
{% if password_auth_enabled %} in the template. Meaning that if you had
an org with email auth disabled and a deactivated user tried to log in,
they wouldn't see the error shown and get confused.
This switches the position of where these error will be shown (above the
login+password form instead of below it), but it looks fine.
Send an empty list of `custom_profile_fields` in `page_params` for
spectators, rather than not sending the field at all.
Also, updates the user info popover to not show the manage user
three-dot menu when in a spectator view.
As noted in the previous commit, this causes bloat in memcached, for
no purpose. Log a warning when `cache_with_key` sees a QuerySet
returned from the function it is decorating.
Storing a QuerySet rather than the list version of the result in it
has a large overhead -- and, as noted by the type annotations, the
result is only ever used as a list. This difference is particularly
important because the cached `get_realm_user_dicts` can get extremely
large for realms with large numbers of users, potentially overflowing
the 1MB default object limit in memcached.
Switch all cases of `cache_with_key` which return QuerySets to
returning the list values of them.
Storing this key is superfluous, as it will be the same for all users,
and definitionally already known to fetch the cache for the realm. It
is also not currently used by the callsites that read rows from the
cache.
9381a3bd45 added support for linkifier pattern URLs containing
`%20`-style escapes, but only did so for the codepath which is used in
the message body -- topic links did not understand them.
Expand the support to include when they are substituted into topics.
Hari Prashant Bhimaraju
Matt Keller
Aman Agrawal
Lauryn Menard
Mateusz Mandera
Alex Vandiver
Anders Kaseorg
Sahil Batra
Zixuan James Li
Alya Abbott
Tim Abbott
Prakhar Pratyush
David Rosa
Florian Pritz
PIG208