This is a best-effort rendering of the "fields" of Slack incoming
hooks, which Slack renders in two columns. We approximate them in a
Markdown table, with some minor in-place replacements.
Fixes#22228.
`check_text_block` transformed its input, making the object it
returned not the same object it was passed; this invalidated it for
use in `check_list`. It is also, in general, unlike all other
validators.
Make it return a TypedDict cast of its input.
If `invite_as` is passed as a number outside the range of a PostgreSQL
`SMALLINT` field, the database throws an exception. Move this exception
to the glass as a validation error to allow better client-side error
handling and reduce database round-trips.
Updates the help center article to match the style and formatting
of "Import from Slack" and replaces existing content with its
corresponding Markdown macro.
When this code was moved from being in zerver in 21a2fd482e, it kept
the `if ZILENCER_ENABLED` blocks. Since ZILENCER and CORPORATE are
generally either both on or both off, the if statement became
mostly-unnecessary.
However, because tests cannot easily remove elements from
INSTALLED_APPS and re-determine URL resolution, we switch to checking
`if CORPORATE_ENABLED` as a guard, and leave these in-place.
The other side effect of this is that with e54ded49c4, most Zulip
deployments started to 404 requests for `/apps` instead of redirecting
them to `https://zulip.com/apps/` since they no longer had any path
configured for `/apps`. Unfortunately, this URL is in widespread use
in the app (e.g. in links from the Welcome Bot), so we should ensure
that it does successfully redirect.
Add the `/apps` path to `zerver`, but only if not CORPORATE_ENABLED,
so the URLs do not overlap.
‘logging.warning("Naive datetime:", item)’ is an invalid call that
crashes with “TypeError: not all arguments converted during string
formatting”. I take that to mean this check has not been tripped in
the six years it’s been there, and can safely be replaced with an
error.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Some email clients (notably, Gmail Web) support automatically threading
emails together if recipients and subjects match[1]. Manual testing
indicated that prefixing a subject with "[bracketed content]" does not
break this threading behavior, but the added checkmark in a resolved
topic's title does. Before sending an email notification, determine
whether the topic is resolved, and pass this information to the Jinja
template to properly format a threadable email subject.
Fixes: #22538
[1]: https://support.google.com/mail/answer/5900
Previously, stream names and topics (without consideration for their
resolution status) were concatenated in Python-land and passed through
to the template. To more cleanly separate concerns, and to prepare for
accounting for topic resolution status being a third, independent,
component of a subject line, instead pass stream and topic strings
independently to the Jinja template, which can format them as it sees
fit.
Additionally, migrate existing EditMessageTest to use this helper
method, with the side effect of migrating the tested flow from a
/json/messages URL to a /api/v1/messages URL.
This commit renames "can_edit_topic_of_any_message" function
in models.py to "can_move_messages_to_another_topic" and
"user_can_edit_topic_of_any_message" function in settings_data.js
to "user_can_move_messages_to_another_topic".
This change is done since topic editing permission does not
depend on message sender now and messages are considered same
irrespective of whether the user who is editing the topic had sent
the message or not. This also makes the naming consistent with
what we use for the label of this setting in webapp and how we
describe this action in help documentation.
This commit changes the topic edit permssions to not depend whether the user
editing the message had sent the message or it was sent by someone else.
We only do backend changes in this commit and frontend changes will be done
in further commits.
Previously, we always allowed topic edits when the user themseleves had
sent the message not considering the edit_topic_policy and the 3-day time
limit. But now we consider all messages as same and editing is allowed only
according to edit_topic_policy setting and the time limit of 3 days in
addition for users who are not admins or moderators.
We change the topic and stream edit permssions to not depend on
allow_message_editing setting in the API and are allowed even
if allow_message_editing is set to False based on other settings
like edit_topic_policy and can_move_message_between_streams.
Fixes a part of #21739.
This uses the linkifier index among the list of linkifiers in the
replacement as the priority to order the replacement order for
patterns in the topic. This avoids having multiple overlapping matches
that each produce a link.
The linkifier with the lowest id will be prioritized when its pattern
overlaps with another. Linkifiers are prioritized over raw URLs.
Note that the same algorithm is used for local echoing and the
backend markdown processor.
Fixes#23715.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This explicitly enforces ordering on the linkifiers. This is useful when
there are overlapping linkifier patterns that matches the same text. In
our current linkifier implementation, this order affects how the
patterns are handled in the markdown processor, with the earlier ones
being prioritized.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
The same pattern being matched multiple times in a topic cannot be
properly ordered using topic_name.find(match_text) and etc. when there
are multiple matches of the same pattern in the topic.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Moves files in `templates/zerver/help/include` that are used
specifically for API documentation pages to be in a new directory:
`templates/zerver/api/include`.
Adds a boolean parameter to `render_markdown_path` to be used
for help center documentation articles.
Also moves the test file `empty.md` to the new directory since
this is the default directory for these special include macros
that are used in documentation pages.
Moves files in `templates/zerver/help/include` that are used
specifically for integrations documentation to be in a new
directory: `templates/zerver/integrations/include`.
Adds a boolean parameter to `render_markdown_path` to be used
for integrations documentation pages.
Track `create_realm` and `new_realm_send_confirm` using
google analytics.
This will help us track number of users who want to
create a new Zulip organization.
Since the removal of `CurlHttpRequest` in Rabbix 6.2, the old script for
setting up the Zabbix integration no longer works.
https://www.zabbix.com/documentation/6.2/en/manual/installation/upgrade_notes_620?hl=CurlHttpRequest#curlhttprequest-removed
This updates the documentation to use `HttpRequest` instead and keep it
up-to-date with the latest Zabbix server. We raise the minimum supported
version from 5.2 to 5.4 because `HttpRequest` was introduced in 5.4.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
‘exit’ is pulled in for the interactive interpreter as a side effect
of the site module; this can be disabled with python -S and shouldn’t
be relied on.
Also, use the NoReturn type where appropriate.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
As we have seen no further cases of this in production since #23215,
increase the severity to an error, and switch from returning a
list (which is not type-safe if the function declares a QuerySet
return) to returning the QuerySet without caching.
Failing to store the result in the cache, with an error, seems
superior to raising an exception; in both cases the next request will
redo the work, but we are guaranteed a worse user experience if we 500
the request.
Ref https://github.com/zulip/zulip/pull/23215#discussion_r994186493
The previous regular expression required a `[^\w]` at the start and
end of the match. This had two unintended effects -- it meant that it
could never match at the start or end of a string, and it meant
that *adjacent* words required *two* non-word characters between them,
as the pattern matches cannot overlap.
Switch to allowing string start/end to anchor the matches, and make
the trailing `[^\w]` be a zero-width look-ahead, to allow the patterns
to overlap. Also remove the spurious `^` within the inner character
classes, which prevented `*foo^bar*` from matching. Finally, add
tests to cover the functionality, which was previously untested.
Updates the Hello World integration documentation and the section
of the related tutorial on documenting the example integration
for the currently used shared macro `create-bot-construct-url.md`.
Also, updates them to use the numbered style currently used in
the majority of the integrations documentation pages.
remove_user_from_user_group's only caller has been removed in 271333301d.
Its usage has been superseded by remove_members_from_user_group.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Doing rapid pace mark-as-unread in the Zulip web application, one
observed assertion failures showing that the server would send an
event containing multiple message IDs but only one of the messages
present in the message_details side data structure.
The cause of this was the "virtual events" compression system; two
flags/remove/read events were being combined by simply concatenating
the lists of events, without any attempt to merge the
`message_details` field on those objects.
The immediate fix is to disable virtual events compression for this
event class, but it's not unlikely we'll need to just eliminate the
virtual_events system entirely, because it seems difficult to make it
soundly handle a message whose state for a given flag changes back and
forth while the client is offline.
But we'll leave that for later, since removing that optimization
deserves more discussion than fixing this event corruption bug.
The naive solution #23465 creates situations where the same user can have
multiple reactions as the base emojis are not unique, e.g. +1::skin2
and +1::skin4 would both reduce to +1 but the userlists are separate.
This solution handles the reduction, merges the same-base reactions,
and deduplicates the userlist.
Co-authored-by: Alex Vandiver <alexmv@zulip.com>
Co-authored-by: rht <rhtbot@protonmail.com>
return inside finally blocks causes exceptions to be silenced.
Although these blocks follow blanket ‘except Exception’ handlers, they
do not seem to have a goal of silencing BaseException and exceptions
thrown by the exception handler, so rewrite them to avoid it.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Both of these compatibility blocks can be deleted, since you can't
upgrade directly to any supported release from the versions where the
old event formats would be used.
This solves the problem that resolving a topic with a long name (>60
characters) will cause the topic name to be truncated, and thus the edit
message code path thinks that the topic is being moved in addition to
being resolved.
We store the pre-truncation topic and use it to check against the
original topic when determining whether a topic is being moved while
getting (un)resovled or not.
Fixes#23482
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We intended to send both the "topic was resolved" and the "topic was
moved here" notification when resolving and moving a topic at the same
time in #22312.
The previous implementation did not work as expected and it was only
sending the "topic was moved here" notification.
This removes the check for old_topic and new_topic that have
RESOLVED_TOPIC_PREFIX stripped in maybe_send_resolve_notifications, so
that the notification will be sent regardless if the topic name without
the prefix stays the same or not.
Note that weird topic handling ("✔ ✔✔ some topic") in the comments
was added in e231a03eff is unaffected. In case of confusion, the lstrip
check is not essential to detecting topic being unresolved/resolved.
As we mainly have that handled in the latter part of the helper.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
CircleCI has updated its webhook format[1] for CircleCI Cloud,
Server version 3.x and 4.x. This commit rewrites the CircleCI
integration to parse the new webhook structure. The tests have also
been rewritten for the new format.
With this commit, we support webhooks from projects that use GitHub,
BitBucket and GitLab as VCS providers. The CircleCI integration doc
has been updated to mention the same. The doc has also been updated
with the latest instructions for configuring a webhook on the CircleCI
interface, and the new output screenshots.
References:
[1]: https://circleci.com/docs/webhooks
Previously, emoji.json was read from
"$ZULIP_PATH/node_modules/emoji-datasource-google/emoji.json".
This path doesn't exist in production when installing from scratch from
a release tarball. And so, we ensure emoji.json exists by copying it to
`static/generated/emoji`.
With tweaks to comments by tabbott.
Fixes: #23469
This is a follow-up to d201229df8.
do_get_invites_controlled_by_user queries for Confirmations when finding
multiuse invites controlled by a user. This means that a revoked
multiuse invite cannot really be fetched here, because
do_revoke_multi_use_invite deletes the Confirmation object when revoking
the invitations. However, having a defensive assert here should be
useful to make this doesn't secretly break in the future if the query
used changes or if there are unexpected revoked multiuse invites with an
existing Confirmations for any (buggy) reason.
This allows us to revoke MultiUseInvites by changing their .status
instead of deleting them (which has been deleting the helpful tracking
information on PreregistrationUsers about which MultiUseInvite they came
from).
This commit adds support for Grafana's new alerting system, Grafana
Alerting. The existing Grafana integration has been modified to
detect the version of the notification through the structure of the
payload body, since the the structure varies by version. Support for
legacy alerting is been continued. Example fixtures have been added
for Grafana Alerting's webhooks.
Tests updated.
Fixes#23517.
While this feature was added to Zulip very early, it has been troubled
for most of that time; it never looked great visually, had a lot of
implementation complexity around resize.js, and has a weird model (a
setting that changes the UI only in certain window sizes).
This option is not commonly used; while a significant portion of users
have it enabled, many of them just don't use window sizes where it
actually has an effect. So it's not clear that it will be missed if
removed; we got very few bug reports when it was completely broken for
a few days after we first integrated the new left sidebar private
messages design.
Even with it no longer being broken, it does not work very well with
the addition of the new PMs section in the left sidebar. (Having two
scrollbars in the sidebar looks quite awkward.) The new private
messages section in the left sidebar also addresses some of the use
cases for always keeping the Users list always visible, even in narrow
windows.
This option is only removed from frontend for now. To make this
decision easily reversible, the backend code of this feature
is still kept.
This doesn't make sense if the realm is active and will fail as soon as
do_reactivate_realm is fixed in the next commit to be a noop and not
create confused RealmAuditLog entries when the realm is active.
There was the following bug here:
1. Send an email invite to a user.
2. Have the user sign up via social auth without going through that
invite, meaning either going via a multiuse invite link or just
straight-up Sign up if the org permissions allow.
That resulted in the PreregistrationUser that got generated in step (1)
having 2 Confirmations tied to it - because maybe_send_to_registration
grabbed the object and created a new confirmation link for it. That is a
corrupted state, Confirmation is supposed to be unique.
One could try to do fancy things with checking whether a
PreregistrationUser already have a Confirmation link, but to avoid races
between ConfirmationEmailWorker and maybe_send_to_registration, this
would require taking locks and so on - which gets needlessly
complicated. It's simpler to not have them compete for the same object.
The point of the PreregistrationUser re-use in
maybe_send_to_registration is that if an admin invites a user, setting
their initial streams and role, it'd be an annoying experience if the
user ends up signing up not via the invite and those initial streams
streams etc. don't get set up. But to handle this, we can just copy the
relevant values from the pre-existing prereg_user, rather than re-using
the object itself.
This line was added in 94e099eaab,
presumably because of the
del request.session["multiuse_object_key"]
line that was just above it.
Looks like it should have been removed in
868a763cec, which eliminated that `del`
operation.
This is primarily for administrators needing to provide message
history for compliance or auditing purposes. Search terms can be
pulled from a file, one per line, or from arguments provided on the
command line.
In 1fce1c3c73, we added logic to parse
the User-Agent in /register requests; this logic crashed if an HTTP request
was missing that header.
Includes a test for `/register` with no user agent passed; this should catch
similar regressions in the future.
Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
For PATCH requests to `/realm/profile_fields`, the field_type is
determined via the field.id, which is in the URL. Therefore,
`field_type` does not need to be passed as part of the request
data in tests.
We do not create historical UserMessage rows, for messages that didn't
have one, while marking messages as read and simply ignore those messages.
We do so because there is no user of creating UserMessage rows and it just
wastes storage.
Note that we still allow to mark messages from unsubscribed streams as
read but only those which have UserMessage rows for them to handle the
case when the unread messages were not marked as read while unsubscribing
from the stream due to some race condition. In such cases, messages
will not be included in the unread count shown in "All messages" menu
(and stream is anyways not present in the left sidebar), but the message
border on the left is green if viewing the stream after unsusbcribing it.
So, to avoid the confusion for users, the messages will be marked as read
when user scrolls down.
This unifies the length of the shortened SHA our integrations generate,
and ensures that they are long enough for projects of various sizes with
a chosen value defined in get_short_sha.
Fixes#23475
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
7 characters are not enough for large projects, so we change
it to reasonably longer. As an example, The Linux kernel needs
at least 11 characters of sha in its shortened form to identify
a revision. We pick 11 so it should work for most of the projects.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
We use `templates/zerver/api/api-doc-template.md` as a base template
for the documented API endpoints in `zerver/openapi/zulip.yaml`.
Previously, if this template was called as an endpoint, then it
would fail an assertion check and send server error. Now we check
for specifically for that potential path and return a 404 error
response for no existing article.
Fixes#21876.
Prior to 53231aa, the `ignore_unhashable_lru_cache` decorator had
a check for the development environment so that changes could be
seen on refresh.
Puts that check back in IgnoreUnhashableLruCacheWrapper class.
In the outgoing webhook handler, there is potentially several seconds
of trying between when a message triggering an outgoing webhook
arrives, and when it fails. In the meantime, the stream the
triggering message was on may have been deleted, causing the
"Failure!" message to have no valid stream to be sent to.
Rather than raise an exception in the outgoing webhook worker, ignore
the exception and move on.
The lambda passed to `queue_json_publish` is used if
`settings.USING_RABBITMQ` is unset -- which is only true in tests. As
such, this pattern causes failures to never actually retry within
tests.
This behaviour has existed ever since the outgoing webhook code was
introduced in 53a8b2ac87, with no explanation. Not passing that
argument allows tests to verify the retry behaviour when webhooks
fail.
Anders Kaseorg
Alex Vandiver
Josh Klar
David Rosa
Josh Klar
Sahil Batra
Ujjawal Modi
Zixuan James Li
Lauryn Menard
Aman Agrawal
Tim Abbott
M@
Hari Prashant Bhimaraju
rht
Mateusz Mandera
Alya Abbott