Commit Graph

17122 Commits

Author SHA1 Message Date
Lauryn Menard 29bb346480 typing: Update main typing notifications protocol documentation.
Updates the main description of the `api/set-typing-status` endpoint
for the new fields in the register response for the typing start,
stop, expired time intervals. Previously these were hardcoded by
the client side code and not the server side code.

Also updates the developer documentation for typing indicators in
the subsystems docs. This refreshes a few parts of that doc that
were already out of date, as well as adds the information about
the new register response fields noted above.
2023-08-23 16:38:59 -07:00
Samuel 3ce7b77092 typing: Add typing constants to the post register api response.
Adds typing notification constants to the response given by
`POST /register`. Until now, these were hardcoded by clients
based on the documentation for implementing typing notifications
in the main endpoint description for `api/set-typing-status`.

This change also reflects updating the web-app frontend code
to use the new constants from the register response.

Co-authored-by: Samuel Kabuya <samuel.mwangikabuya@kibo.school>
Co-authored-by: Wilhelmina Asante <wilhelmina.asante@kibo.school>
2023-08-23 16:36:44 -07:00
Anders Kaseorg d26a94a0db management: Validate string_id when creating or renaming a realm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-23 16:33:13 -07:00
evykassirer b9303a6506 push notifications: Use hex_codepoint_to_emoji.
Now that we have this util function, we can
use it here. No functional changes.
2023-08-23 16:18:15 -07:00
evykassirer 0289beb784 emoji: Match emoji sequences in markdown.
Fixes #11767.

Previously multi-character emoji sequences weren't matched in the
emoji regex, so we'd convert the characters to separate images,
breaking the intended display.

This change allows us to match the full emoji sequence, and
therefore show the correct image.
2023-08-23 16:18:15 -07:00
Sahil Batra 5a8416ff6a message: Do not pass "sender__realm" to select_related.
We have modified the code to directly fetch realm from Message
object instead of "sender" field and thus we no longer need to
fetch "sender__realm" using select_related.
2023-08-23 11:38:32 -07:00
Sahil Batra e9a6f69849 tests: Access realm directly from ScheduledMessage object.
There is no need to get realm for sender as ScheduledMessage
object also has realm field.

There is no direct benefit of this change but it is nice to
maintain the pattern which we want to follow in the code
in tests as well.
2023-08-23 11:38:32 -07:00
Sahil Batra 58aecbe443 message: Pass realm as argument to wildcard_mention_allowed.
We do not want to access realm from "sender" field so that
we do not need to pass "sender__realm" argument to
select_related call when querying messages. We can instead
pass realm as argument to wildcard_mention_allowed.
2023-08-23 11:38:32 -07:00
Sahil Batra df2407f97a message: Access realm from SendMessageRequest object directly.
We store realm object in SendMessageRequest object, so we can
access it directly instead of getting it from "sender" field.
2023-08-23 11:38:32 -07:00
Sahil Batra 7295028194 message: Access realm object directly from message.
We can directly get the realm object from Message object now
and there is no need to get the realm object from "sender"
field of Message object.

After this change, we would not need to fetch "sender__realm"
field using "select_related" and instead only passing "realm"
to select_related when querying Message objects would be enough.

This commit also updates a couple of cases to directly access
realm ID from message object and not message.sender. Although
we have fetched sender object already, so accessing realm_id
from message directly or from message.sender should not matter,
but we can be consistent to directly get realm from Message
object whenever possible.
2023-08-23 11:38:32 -07:00
Sahil Batra 4f30447b95 test_markdown: Set realm for Message objects.
We do not set realm to Message objects defined for markdown tests
and this works because we currently access realm from sender object.

This commit changes the code to set realm in Message objects as
we would be accessing realm from Message object directly in further
commits.
2023-08-23 11:38:32 -07:00
Alex Vandiver adc987dc43 send_email: Use a consistent order when sending custom emails to users. 2023-08-23 10:49:34 -07:00
David Rosa 0349152f0f help: Rename edit-or-delete-a-message.md and update links. 2023-08-22 14:50:23 -07:00
Satyam Bansal eb8714c9dc integrations: Add support for "Test plugin" in Sentry integration.
Previously, if a user tried to create a webhook using the Webhooks
plugin in Sentry and used the "Test plugin" to test the webhook,
the server would send a 500 error, even though the integration
worked perfectly. This led users to believe that the integration
was not working.

Fixes #26173.
2023-08-22 12:09:02 -07:00
Satyam Bansal 6898667fa4 integrations: Add Raven SDK test to Sentry Integration. 2023-08-22 12:09:02 -07:00
Satyam Bansal fdc14ee3f0 integrations: Rename fixture in Sentry integration.
This is done to clarify from where this fixture is coming from; as there
are two documented ways to test the integration.
2023-08-22 12:09:02 -07:00
Anders Kaseorg 328cdde243 documentation: Remove duplicate heading IDs on server side.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-21 17:25:14 -07:00
Sahil Batra 7137eba222 streams: Don't compute traffic data for sub objects in zephyr realm.
We set stream_weekly_traffic field to "null" for Subscription
objects in zephyr mirror realm as we do not need stream traffic
data in zephyr mirror realm. This makes the subscription data
consistent with steams data.

This commit also udpates test to check never_subscribed data for
zephyr mirror realm.
2023-08-21 15:21:58 -07:00
Sahil Batra 6776e380b2 stream_traffic: Update get_streams_traffic to return None for zephyr realm.
Instead of having a "realm.is_zephyr_mirror_realm" check for every
get_streams_traffic call, this commit udpates get_streams_traffic to
accept realm as parameter and return "None" for zephyr mirror realm.
2023-08-21 15:21:58 -07:00
Anders Kaseorg 1244c066ad help: Highlight current article on the server side.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-21 14:32:19 -07:00
bjorn3 9c409a332a openapi: Extract an Anchor schema.
It was previously duplicated in multiple places.
2023-08-21 11:26:19 -07:00
bjorn3 ab464e6728 openapi: Use enum for the anchor parameter.
This helps catch incorrect anchor parameters and makes bindings
generated directly from the openapi document a bit nicer.
2023-08-21 11:26:19 -07:00
bjorn3 78176d8982 openapi: Move description from SuccessDescription to JsonSuccessBase.
Almost all users of JsonSuccessBase seem to also include
SuccessDescription. /server_settings used a different description from
the rest of the JsonSuccessBase users, but the difference is small
enough that using the generic description of the former
SuccessDescription is fine.
2023-08-21 11:26:19 -07:00
Lauryn Menard 5d5a578d6c scheduled-mails: Migrate existing scheduled emails to new templates.
Migrates existing ScheduledEmails for onboarding emails that have
either "zerver/emails/followup_day1" or "zerver/emails/followup_day2"
as the email template prefix to instead use the new template
prefixes "zerver/emails/account_registered" and
"zerver/emails/zulip_onboarding_topics".
2023-08-18 16:51:46 -07:00
Lauryn Menard 438bcc1585 email-templates: Remove followup_day from EMAIL_TYPES.
Now that we're using the new templates for the onboarding emails,
remove "followup_day1" and "followup_day2" from the EMAIL_TYPES
that are used for scheduled emails.
2023-08-18 16:25:48 -07:00
Lauryn Menard 5e29e025c5 email-templates: Add zulip_onboarding_topics email templates.
The "followup_day2" email template name is not clear or descriptive
about the purpose of the email. Creates a duplicate of those email
template files with the template name "zulip_onboarding_topics".

Because any existing scheduled emails that use the "followup_day2"
templates will need to be updated before the current templates can
be removed, we don't do a simple file rename here.
2023-08-18 16:25:48 -07:00
Lauryn Menard c491bef07b email-templates: Add account_registered email templates.
The "followup_day1" email template name is not clear or descriptive
about the purpose of the email. Creates a duplicate of those email
template files with the template name "account_registered".

Because any existing scheduled emails that use the "followup_day1"
templates will need to be updated before the current templates can
be removed, we don't do a simple file rename here.
2023-08-18 16:25:48 -07:00
Anders Kaseorg c43629a222 ruff: Fix PLW1510 `subprocess.run` without explicit `check` argument.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Anders Kaseorg 1df3db286b test_import_export: Fix incorrect type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Anders Kaseorg 36dde99308 ruff: Appease SIM118 `"class" not in uncle.keys()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Anders Kaseorg ec00c2970f ruff: Fix PYI032 Prefer `object` for the second parameter to `__eq__`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Anders Kaseorg 53e8c0c497 ruff: Fix E721 Do not compare types, use `isinstance()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Alex Vandiver 5368d1bd4c middleware: Allow HTTP from localhost, not through a reverse proxy.
In servers with `application_server.http_only = true` and
`loadbalancer.ips` set, the DetectProxyMisconfiguration middleware
prevents access over HTTP from IP addresses other than the
loadbalancer.

However, this misses the case of access from localhost over HTTP,
which is safe and expected -- for instance, the `email-mirror-postfix`
script used in the email gateway[^1] will post to `http://localhost/`
by default in such configurations.  With the
DetectProxyMisconfiguration installed, this will result in a 403
response.

Make an exception for requests from `127.0.0.1` and `::1` from
proxy-misconfiguration rejections.

[^1]: https://zulip.readthedocs.io/en/latest/production/email-gateway.html
2023-08-17 12:07:37 -07:00
Lauryn Menard ab897fa3b9 api-docs: Remove incorrect error response from `/api/subscribe`.
Removes a response example in the `POST users/me/subscriptions`
documentation that was listed as a 400 error response. It is
actually a variation on the success response for this endpoint.

The current rendering of our API documentation is not set up to
support `"anyOf"` which would allow for validating examples that
match multiple response schemas.
2023-08-17 11:26:36 -07:00
Zixuan James Li 30495cec58 migration: Rename extra_data_json to extra_data in audit log models.
This migration applies under the assumption that extra_data_json has
been populated for all existing and coming audit log entries.

- This removes the manual conversions back and forth for extra_data
throughout the codebase including the orjson.loads(), orjson.dumps(),
and str() calls.

- The custom handler used for converting Decimal is removed since
DjangoJSONEncoder handles that for extra_data.

- We remove None-checks for extra_data because it is now no longer
nullable.

- Meanwhile, we want the bouncer to support processing RealmAuditLog entries for
remote servers before and after the JSONField migration on extra_data.

- Since now extra_data should always be a dict for the newer remote
server, which is now migrated, the test cases are updated to create
RealmAuditLog objects by passing a dict for extra_data before
sending over the analytics data. Note that while JSONField allows for
non-dict values, a proper remote server always passes a dict for
extra_data.

- We still test out the legacy extra_data format because not all
remote servers have migrated to use JSONField extra_data.
This verifies that support for extra_data being a string or None has not
been dropped.

Co-authored-by: Siddharth Asthana <siddharthasthana31@gmail.com>
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-08-16 17:18:14 -07:00
Sahil Batra 98b0fa8ae2 bots: Remove subscription from inaccessible streams on reactivating bot.
This commit removes the private stream suscriptions of the bot if the
original owner is deactivated and we change the owner to the user who
is reactivating the bot. We unsusbcribe the bot from private streams
that the new owner is not subscribed to.

Fixes part of #21700.
2023-08-16 15:37:37 -07:00
Sahil Batra 35d5609996 bots: Remove private stream subscriptions on changing bot owner.
We remove bot's subscriptions for private streams to which the
new owner is not subscribed and keep the ones to which the new
owner is subscribed on changing owner.

This commit also changes the code for sending subscription
remove events to use transaction.on_commit since we call
the function inside a transactopn in do_change_bot_owner and
this also requires some changes in tests in test_events.
2023-08-16 15:37:37 -07:00
Sahil Batra ba00907946 bots: Re-parent bot while reactivating if original owner is deactivated.
Since the original owner is deactivated, we change the owner to the
admin who is reactivating the bot.
2023-08-16 15:37:37 -07:00
Sahil Batra 133bd1e4cb bots: Extract events sent on chaning bot owner in a separate function.
This is a prep commit to re-parent the bots on reactivating them if
original owner is deactivated such that we can avoid duplicating code.
2023-08-16 15:37:37 -07:00
Lauryn Menard 1713449639 api-docs: Fix documentation of `realm_emoji` in register response.
Fixes the `/api/register-queue` endpoint documentation so that the
`realm_emoji` has the correct type, object that contains objects.

By correcting the API documentation, we also fix an error in the
test for the events system, which had been relying on the API
documentation having a list as a possible type for `realm_emoji`
in the register response.
2023-08-16 14:32:20 -07:00
Prakhar Pratyush 665f491f22 message_edit: Fix wildcard_mentioned flag set for all user-messages.
Earlier, for topic wildcard mentions, the 'wildcard_mentioned'
flag was set for all the user-messages. (similar to stream wildcard
mention).

The flag should be set for the topic participants only.

The bug was introduced in 4c9d26c.
2023-08-16 11:31:56 -07:00
Prakhar Pratyush 379a08eb1e message_send: Fix wildcard_mentioned flag unset for few participants.
For topic wildcard mentions, the 'wildcard_mentioned' flag is set
for those user messages having 'user_profile_id' in
'topic_participant_user_ids', i.e. all topic participants.

Earlier, the flag was set if the 'user_profile_id' exists in
'all_topic_wildcard_mention_user_ids'.
'all_topic_wildcard_mention_user_ids' contains the ids of those
users who are topic participants and have enabled notifications
for '@topic' mentions.

The earlier approach was incorrect, as it would set the
'wildcard_mentioned' flag only for those topic participants
who have enabled the notifications for '@topic' mention instead
of setting the flag for all the topic participants.

The bug was introduced in 4c9d26c.
2023-08-16 11:31:56 -07:00
Tim Abbott ea83c911e9 Revert "narrow: Fix topic highlighting issue with apostrophes in search results."
This reverts commit baede93f69.

This failed tests after rebasing it on top of
5151dd7ff8.
2023-08-15 17:51:03 -07:00
Akshat baede93f69 narrow: Fix topic highlighting issue with apostrophes in search results.
This commit addresses the issue where the topic highlighting
in search results was offset by one character when an
apostrophe was present. The problem stemmed from the disparity
in HTML escaping generated by the function `func.escape_html` which
is used to obtain `topic_matches` differs from the escaping performed
by the function `django.utils.html.escape` for apostrophes (').

func.escape_html | django.utils.html.escape
-----------------+--------------------------
      &#39;      |           &#x27;

To fix this SQL query is changed to return the HTML-escaped
topic name generated by the function `func.escape_html`.

Fixes: #25633.
2023-08-15 17:29:20 -07:00
Akshat 5151dd7ff8 messages_in_narrow: Use `add_narrow_conditions`.
The `messages_in_narrow_backend` function was directly
calling `NarrowBuilder` instead of utilizing the
`add_narrow_conditions` method like `fetch_messages` does.

This behaviour was not combing any search operands together
as it happens inside the `add_narrow_conditions`.

Fixes: https://chat.zulip.org/#narrow/stream/3-backend/topic/messages_in_narrow_backend.20calling.20NarrowBuilder.20directly/near/1611193.

Test added by tabbott.

Signed-off-by: Akshat <akshat25iiit@gmail.com>
2023-08-15 13:18:10 -07:00
Joelute eb78264162 navbar_alerts: Delay showing "Complete the organization profile" banner.
Currently, we are displaying the "Complete the organization profile"
banner immediately after the organization was created. It's important to
strongly encourage orgs to configure their profile, so we should delay
showing the banner if the profile has not been configured after 15 days.
Thus also allows the users to check out Zulip and see how it works before
configuring the organization settings.

Fixes: #24122.
2023-08-15 10:46:33 -07:00
David Rosa 6aee4bb768 help: Document how to view all direct messages in the desktop/web app. 2023-08-15 10:10:31 -07:00
bjorn3 762c9667a7 uploads: Rewrite comment in do_claim_attachments to improve clarity.
Co-Authored-By: Tim Abbott <tabbott@zulip.com>
2023-08-15 10:08:11 -07:00
bjorn3 8279ad603d uploads: Fix double negation in do_claim_attachments comment. 2023-08-15 10:08:11 -07:00
Lauryn Menard 571bd16ba4 test-openapi: Add coverage for documented fetch API key endpoints.
Previoulsy, test_openapi_arguments had assumed that an endpoint
not using rest_dispatch used the GET method for the request. This
was not the case for the "/fetch_api_key" and "/dev_fetch_api_key"
endpoints, which is why those endpoints were marked as pending
even though they were documented in `zerver/openapi/zulip.yaml`.

Updates test_openapi_arguments to check a set of endpoints that
are documented and don't use the GET method so that these endpoints
can be tested and removed from the pending_endpoints set.
2023-08-15 09:52:55 -07:00