Commit Graph

16576 Commits

Author SHA1 Message Date
Lauryn Menard 06dd7a3a68 emails: Add corporate_enabled to context for emails.
In commit fc58c35c0, we added a check in various emails for the
settings.CORPORATE_ENABLED value, but that context is only always
included for views/templates with a request.

Here we add that to common_context, which is often used when there
is not a request (like with emails). And we manually add it to the
email context in various cases when there is not a user account to
call with common_context: new user invitations, registration emails,
and realm reactivation emails.
2023-05-01 10:32:43 -07:00
Lauryn Menard d41f12495d api-docs: Document create/edit scheduled message endpoint and events.
Adds API changelog update for the new endpoint added for creating
or editing scheduled messages, for the events sent for scheduled
messages (add, update, delete), and for the `scheduled_messages`
field in the register response.

Adds a specific API markdown documentation file for the create
or update scheduled messages endpoint, so that the parameters in
the cURL examples are accurate for the type of message (direct or
stream) and action (create or edit).

Bumps the API feature level, adds Changes notes to API documentation
and adds the scheduled message endpoints to the API documentation
sidebar.
2023-04-28 17:25:00 -07:00
Lauryn Menard 15c6d67e9c populate-db: Add scheduled message to test database.
Prep commit for adding the scheduled-message endpoints to the API
documentation.

Adds a scheduled message for Iago in the test database so that it
can be deleted in the delete cURL example in the api-test suite.
2023-04-28 17:25:00 -07:00
Aman Agrawal bd2545b0d7 scheduled_message: Send CRUD events to clients. 2023-04-28 17:25:00 -07:00
Aman Agrawal f40855bad2 reminder: Remove feature from zulip.
This is being removed to make the code simpler. We have plans
to add it as a feature in the future , but it will most likely
not use the same code.
2023-04-28 17:25:00 -07:00
Aman Agrawal d60d6e9115 urls: Add new endpoint to create scheduled messages.
This will help us remove scheduled message and reminder logic
from `/messages` code path.

Removes `deliver_at`/`defer_until` and `tz_guess` parameters. And
adds the `scheduled_delivery_timestamp` instead. Also updates the
scheduled message dicts to return `scheduled_delivery_timestamp`.

Also, revises some text in `/delete-scheduled-message` endpoint
and in the `ScheduledMessage` schema in the API documentation.
2023-04-28 17:25:00 -07:00
Lauryn Menard 7739703111 scheduled-messages: Update scheduled message objects in the API for type.
Updates the objects in the API for scheduled messages so that those
for stream messages return the `to` property as an integer since it
is always the unique stream ID and so that those for direct messages
do not have a `topic` property since direct messages never have a
topic.

Also makes small update so that web app scheduled messages overlay
has the correct stream ID.
2023-04-28 17:25:00 -07:00
Aman Agrawal 4718eaa213 scheduled_messages: Extract method to get undelivered scheduled messages.
This also changes key for `ID` of scheduled message from `message_id`
to `scheduled_message_id`.
2023-04-28 17:25:00 -07:00
Aman Agrawal 4cb238fb6e models: Add method to convert ScheduleMessage objects into dicts. 2023-04-28 17:25:00 -07:00
Aman Agrawal 963fe566d7 scheduled_messages: Use scheduled_message_id instead of message_id.
Using `message_id` can be confusing for API users since it can be
mistaken for the ID of the message that will be sent.
2023-04-28 17:25:00 -07:00
Aman Agrawal 7bf0793c94 scheduled_messages: Move database related function to /actions.
This would match the pattern we use for other modules and also
shrink the giant message_send.py.
2023-04-28 17:25:00 -07:00
Aman Agrawal cddf25656f test_message_send: Move out scheduled message tests. 2023-04-28 17:25:00 -07:00
Alex Vandiver 4f2417cfc4 soft_reactivation: Add a partial index to speed up event lookups.
The full auditlog table is moderately large, and the previously-chosen
index (on `modified_user_id`) is not terribly specific.
2023-04-28 12:43:34 -07:00
Alex Vandiver a56da4be76 soft_deactivation: Only fetch necessary columns.
Existing tests verify that this does not add more queries.
2023-04-28 12:43:34 -07:00
Alex Vandiver ae7485a96e soft_deactivation: Do not bother to fetch stream data as well.
This prefetch is unnecessary and makes this query load more data than
needed.

Existing tests verify that this does not add more queries.
2023-04-28 12:43:34 -07:00
Tim Abbott a03dca93ca message_edit: Clarify losing_access_user_ids calculation.
This was previously called delete_event_notify_user_ids, which seemed
to narrow its purpose in a way that was confusing given that it's also
used for other calculations.

Further, calculate it as soon as we know it, not when we're first
going to use it.
2023-04-27 18:35:32 -07:00
Tim Abbott 71e57d2022 message_edit: Clarify name for gaining_usermessage_user_ids.
This list contains user IDs, not subscription objects.
2023-04-27 18:35:32 -07:00
Tim Abbott 3ff361a1f0 message_edit: Use sets for old/new stream IDs.
This fixes a quadratic performance issue iterating through these
lists. Give these variables slightly better names while we're at it.
2023-04-27 18:35:32 -07:00
Prakhar Pratyush c8a9c0ee04 realm_redirect: Redirect always to the login page with the next parameter.
Previously, entering an organization via 'accounts/go' with the
web-public stream enabled took the user to the web-public view
even if the user was not logged in.

Now, a user is always redirected to the 'login_page' with
the next parameter, if present.

The 'login_page' view is updated to redirect an authenticated
user based on the 'next' parameter instead of always redirecting
to 'realm.uri'.

Fixes #23344.
2023-04-27 16:50:10 -07:00
Sahil Batra afc5066e36 registration: Fix "Resend" link not working for realm creation.
The "Resend" link for realm creation was not working correctly
because it is implemented by basically submiting the registration
form again which results in resending the email but all the
required parameters were not passed to the form after recent
changes in the realm creation flow.

This commit fixes it by passing all the required parameters -
email, realm name, realm type and realm subdomain, when submitting
form again by clicking on the "resend" link.

Fixes #25249.
2023-04-27 12:28:37 -07:00
Sahil Batra f8f4fa4c5e tests: Extract realm name and string_id values in variables.
This is a prep commit so that we can use these variables to
verify the urls in next commit.
2023-04-27 12:28:37 -07:00
Sahil Batra 850e0046eb models: Add ORG_TYPE_IDS constant field to Realm.
This commit adds ORG_TYPE_IDS constant field to Realm class
such that it can be used when we want to validate the org_type
passed in request. This was previously defined in realm.py, but
we move it inside Realm class such that we can use it at other
places as well.
2023-04-27 12:28:37 -07:00
Daniil Fadeev db37880d08 emails: Fix some css not being applied to emails.
This commit places the email CSS into the `style` tag located in the
`head` section. This resolves the issue of being unable to apply
certain CSS styles that cannot be inlined, such as media queries and
pseudo-classes.
2023-04-27 08:55:24 -07:00
AcKindle3 521487f444 tornado_url: Replace occurrences of `uri` with `url`.
In #23380 we want to replace all occurrences of `uri` with `url`.
This commit replaces the occurrences appeared in a variable name
`tornado_uri` and a function name `get_tornado_uri`.
2023-04-26 16:37:16 -07:00
AcKindle3 0a1ccb3d89 api_url_context: Replace `uri` with `url`.
In #23380 we want to change all occurrences of `uri` with `url`.
This commit changes the occurrences in a context key `api_uri_context`
and a function name `add_api_uri_context`.
2023-04-26 16:37:16 -07:00
AcKindle3 98c9a0366a settings: Replace occurrences of `uri` with `url`.
In #23380 we want to change all occurrences of `uri` with `url`.
This commit changes the names of two variables `external_uri_scheme`
and `main_site_uri`, who are constructed using `settings` constants.
2023-04-26 16:37:16 -07:00
AcKindle3 4544eb4576 email: Replace `uri` with `url` in templates and backend.
In #23380 we want to change all ocurrences of `uri` to `url`. This
commit changes the ocurrences of `uri` appeared in files related to
email, including templates (`.html`, `.txt`) and backend (`.py`)
codes.

In `email.md`, `base_images_uri` is changed to `images_base_url` -
the words `base` and `images` are swapped and plural form is added
for `image`.  This is becasue the former is not found anywhere in
the codebase while the later appears a lot. To reduce confusion,
this doccumentation changed accordingly.
2023-04-26 16:37:16 -07:00
Lauryn Menard a22168d8b3 templates: Add CSS formatting to invalid email redirect page.
Adds CSS formatting for `invalid_email.html`.

Uses the `white-box` style because this page is a redirect when
there is an error with the email the user provided during
registration.

Also, updates the text of this page for some grammar errors and
to clarify the language between an invalid email and an email that
is not allowed by the Zulip organization in question.

Finally, makes any references to the `realm_name` also link to
the Zulip organization with the `realm_uri`.
2023-04-26 15:41:20 -07:00
Lauryn Menard 601d8101f5 templates: Add CSS formatting to no available licenses page.
Adds CSS formatting for `no_spare_licenses.html`.

Uses the `white-box` style because this page is a redirect when
a user tries to register for a Zulip Cloud organization that does
not have any available licenses for new users.

Updates reference to `realm_name` to be a link to the `realm_uri`.
2023-04-26 15:41:20 -07:00
Alya Abbott 977bec25ba portico: Add Atolio case study. 2023-04-26 14:33:30 -07:00
Tim Abbott 10b893e1e3 api_docs: Update API docs for presence changes. 2023-04-26 14:26:47 -07:00
Mateusz Mandera a9f40a64fd presence: Support null values in UserPresence. 2023-04-26 14:26:47 -07:00
Mateusz Mandera 0d79f6dd27 presence: Deduplicate code formatting legacy presence info.
This also removes the error in one of these functions that was using a
different constant instead of
PRESENCE_LEGACY_EVENT_OFFSET_FOR_ACTIVITY_SECONDS.
2023-04-26 14:26:47 -07:00
Mateusz Mandera ed91e72c5c presence: Rewrite the backend data model - final migration. 2023-04-26 14:26:47 -07:00
Tim Abbott 027b67be80 presence: Rewrite the backend data model.
This implements the core of the rewrite described in:

For the backend data model for UserPresence to one that supports much
more efficient queries and is more correct around handling of multiple
clients.  The main loss of functionality is that we no longer track
which Client sent presence data (so we will no longer be able to say
using UserPresence "the user was last online on their desktop 15
minutes ago, but was online with their phone 3 minutes ago").  If we
consider that information important for the occasional investigation
query, we have can construct that answer data via UserActivity
already.  It's not worth making Presence much more expensive/complex
to support it.

For slim_presence clients, this sends the same data format we sent
before, albeit with less complexity involved in constructing it.  Note
that we at present will always send both last_active_time and
last_connected_time; we may revisit that in the future.

This commit doesn't include the finalizing migration, which drops the
UserPresenceOld table.
The way to deploy is to start the backfill migration with the server
down and then start the server *without* the user_presence queue worker,
to let the migration finish without having new data interfering with it.
Once the migration is done, the queue worker can be started, leading to
the presence data catching up to the current state as the queue worker
goes over the queued up events and updating the UserPresence table.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-04-26 14:26:47 -07:00
Tanmay Munjal 80c01e9fc3
auth: Refactor JWT code to push REQ to top level functions.
Fixes #25015.
2023-04-26 10:57:22 -07:00
Anders Kaseorg 03b3c8522d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 21:20:33 -07:00
Anders Kaseorg f74cfe0ed5 test_console_output: Fix WrappedIO.writelines with iterator.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 21:20:33 -07:00
Anders Kaseorg 5ecff88c1b ruff: Fix C418 Unnecessary `dict` literal passed to `dict()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 21:20:33 -07:00
Anders Kaseorg 9db3451333 Remove statsd support.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 19:58:16 -07:00
Anders Kaseorg ae7a21834e utils: Fix make_safe_digest type safety.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 19:01:01 -07:00
Anders Kaseorg 408ef63404 utils: Fix process_list_in_batches type safety.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 19:01:01 -07:00
Daniil Fadeev d79f4d4f46 emails: Fix style regression for dynamically generated content.
The migration to css-inline "fixed" the fact that styles from this
file previously were never applied to the internals of missed-message
emails.

Rewrite much of the CSS to more correctly scope to apply to the
appropriate elements, and document with comments the purpose of most
blocks.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2023-04-25 17:53:14 -07:00
Alya Abbott dd4b391229 help: Cover unmuting topics in /help/mute-a-topic. 2023-04-25 15:04:24 -07:00
Lauryn Menard 9880fcb936 narrow: Document new filters `is:dm`, `dm` and `dm-including`.
Documents narrows now have support for new filters for direct messages:
`is:dm`, `dm`, and `dm-including`. Also documents that `is:private`,
`pm-with` and `group-pm-with` are now legacy aliases for these three
new filters respectively.

Note that API documentation references the help center documentation
for search/narrow filters.

Fixes #24806.
2023-04-25 09:07:08 -07:00
Lauryn Menard 243216cb60 url-encoding: Update URLs for direct messages.
In a previous commit, the frontend of the web app was updated for
URLs with `#narrow/dm/...` for direct messages (group and 1-on-1).

Updates the URLs generated for email notifications and outgoing
webhook notification messages to use the new `/dm/...` format.
2023-04-25 09:07:08 -07:00
Lauryn Menard 33886575b2 narrow: Add backend support for `dm-including` operator.
Adds backend support for `dm-including` operator. This will
deprecate the `group-pm-with` operator, but we keep support
for backwards-compatibility.

For testing updates, because the messages returned by these
two operators are different, most of the tests for `group-pm-with`
remain unchanged, but added comments about deprecated state.

Also, cleans up remaining instance of "PM" in `narrow.py` to
be "DM".

The general API changelog and documentation updates will be done
in a final commit in the series of commits that adds support for
the various new direct message narrows.
2023-04-25 09:07:08 -07:00
Lauryn Menard 29832de5f6 narrow: Extract function for calculating huddle recipients.
Extracts the code from `by_group_pm_with` that calculates the
huddle recipients. Prep commit for implementing new `by_*`
method for "dm-including".
2023-04-25 09:07:08 -07:00
Lauryn Menard 665e435b58 narrow: Add backend support for `dm` operator.
Adds backend support for `dm` operator. This will deprecate the
`pm-with` operator, but we keep support for backwards-compatibility.

For testing updates, updates the existing tests for `pm-with` to
use `dm`, and adds one basic test for `pm-with` in the `add_term`
tests as the two operators refer to the same `by_*` method.

The general API changelog and documentation updates will be done
in a final commit in the series of commits that adds support for
the various new direct message narrows.
2023-04-25 09:07:08 -07:00
Lauryn Menard ece752014c narrow: Add backend support for `is:dm` narrow.
Adds backend support for `is` operator with the `dm` operand. This
will deprecate the `is` operator with the `private` operand, but we
keep support for backwards-compatibility.

Note that there is some clean up of references to private messages
in the updated backend test. In commit 43ec7ed, the documentation
for `build_narrow_filter` wasn't updated for the rename of
`BuildNarrowFilterTest` to `NarrowLibraryTest`, so that's also
corrected in these changes.

The general API changelog and documentation updates will be done
in a final commit in the series of commits that adds support for
the various new direct message narrows.
2023-04-25 09:07:08 -07:00