Commit Graph

8221 Commits

Author SHA1 Message Date
Alex Vandiver 936c2b54cb push_notifications: Use ignore_conflicts, over catching IntegrityError.
The IntegrityError shows up in the database logs, which looks
unnecessarily concerning.  Use `ON CONFLICT IGNORE` to mark this as
expected, especially since the return value is never used.
2024-04-12 11:36:40 -07:00
Tim Abbott 1cd11a4c61 zulip_updates: Adjust formatting for initial updates. 2024-04-11 16:13:42 -07:00
Prakhar Pratyush bc0325922e commands: Add 'skip-delay' option to send_zulip_update_announcements.
This commit adds a 'skip-delay' option to the
'send_zulip_update_announcements' management command.

It will be useful for self-hosted servers after 9.0 upgrade to
avoid the 24 hour delay to receive update messages after group
DM is sent to the admins.

One can run the management command with the --skip-delay flag
to immediately send the update messages.
2024-04-11 16:13:42 -07:00
Mateusz Mandera c9ca4e68e5 scim: Add config option to disable initial streams for guests.
When an organization (without open ability for anyone to join) invites a
guest user, the invitation prompts allows them to choose whether the
guest should be added to default streams or not. This is useful, because
since we don't have per-role default streams configs, they may want
default streams to be for full Members.

SCIM provisioning doesn't have this control, since a newly provisioned
user gets created via a direct do_create_user call, thus adding them to
the organization's default streams, with no workaround possible aside of
just getting rid of default streams in the organization.

To make provisioning guests in such an organization usable, we add a
simple config option to create them with no streams. It's configured by
adding
```
"create_guests_without_streams": True
```

to the config dict in settings.SCIM_CONFIG.
2024-04-11 12:28:26 -07:00
Sahil Singh 6c2535fe46 bots: Avoid multiple active bots with the same name.
Creating a bot with a name that is already in use
will raise an error. However, by deactivating
the existing bot, creating a new bot with the
same name, and then reactivating the original bot,
it is possible to have multiple bots with the same name.

To fix this, we check if the bot name is already
in use in the active bots list. If it is,
an error will be raised, prompting either the
name of the existing bot to be changed or
the bot to be deactivated.

Co-authored-by: Sujal Shah <sujalshah28092004@gmail.com>
2024-04-10 13:14:25 -07:00
Lauryn Menard e700e818e5 mentions: Add "channel" as a wildcard mention.
Adds "channel" to the `stream_wildcards` frozenset for stream
wildcard notifications on the backend/server.

Updates frontend/web-app to handle "channel" as the other stream
wildcards are handled in the typeahead and composebox modules.

Updates the API version and documentation for the addition of
"channel" as a wildcard mention. But does not change any of the
functionailty of (or deprecate) the "stream" wildcard at this
point.

Part of project to rename "stream" to "channel".
2024-04-04 11:16:07 -07:00
Prakhar Pratyush db214c8377 message: Add recipient_users parameter to internal_send_huddle_message.
This is a prep commit to add a 'recipient_users' parameter to
the 'internal_send_huddle_message' function.

'emails' is no longer a required parameter. We can use either
of the 'emails' or 'recipient_users' parameter. 'emails' is
eventually used to fetch 'recipient_users', so if the
'recipient_users' is already available we should use that to
skip database query.
2024-04-03 12:28:05 -07:00
Prakhar Pratyush bcebcb334c zulip_news: Wait for one day after sending group DM if stream is set.
For organizations with "Zulip update announcements" stream set
to a default value, we wait for one day after sending group
DM to admins to allow them to change the stream from it's
default value if they wish to.
2024-04-02 15:12:50 -07:00
Prakhar Pratyush d4fe4ca934 zulip_news: Refactor 'is_group_direct_...' to allow flexible timeframe.
This prep commit refactors the function
'is_group_direct_message_sent_to_admins_atleast_one_week_ago' to
'is_group_direct_message_sent_to_admins_within_days' allowing us
to use a flexible timeframe instead of hardcoded 1 week.

We will reuse this function as a part of determinig whether
group DM to admins was sent within 1 day.
2024-04-02 15:12:46 -07:00
Alya Abbott b8e5954c0b zulip_updates: Announce three features.
- Pasting formatted content.
- Quote-and-reply selection.
- Hide user list.
2024-04-02 15:07:22 -07:00
Tim Abbott 563485a37e zulip_updates: Fix line removal logic for bulleted lists. 2024-04-02 15:07:22 -07:00
roanster007 c7a08f3b77 settings: Add permission to enforce unique names in realm.
Previously, users were allowed to signup or change their names to
those which already existed in the realm.

This commit adds an Organization Permission, that shall enforce
users to use unique names while signing up or changing their
names. If a same or normalized full name is found in realm,
then a validation error is thrown.

Fixes #7830.
2024-04-02 14:55:59 -07:00
Anders Kaseorg 6e871e7731 ruff: Fix UP036 Version block is outdated for minimum Python version.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-01 18:32:52 -07:00
Anders Kaseorg a82a3eb4d7 ruff: Fix UP033 Use `@functools.cache`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-01 18:32:52 -07:00
Aman Agrawal d21f5c9b75 registration: Ask user how they found Zulip. 2024-04-01 12:44:12 -07:00
Prakhar Pratyush a7dc7c0734 message: Add function to remove single newline in triple quoted string.
For multiline strings in triple quotes, a '\n' is included
at the end of each line.

Earlier, to skip '\n' we used to add an escape character '\'
at the end of each line.

This commit adds a function to avoid manually adding '\'.
2024-03-28 09:03:59 -07:00
Prakhar Pratyush b9d9676b43 zulip_news: Update the group DM sent initially to admins.
As a part of the zulip news feature, we send an initial
group DM to admins suggesting them to update or set
the 'zulip_update_announcements_stream'.

This commit updates those messages to make it better.
2024-03-27 11:42:00 -07:00
Alex Vandiver 7de527f3bc soft_deactivation: Use much faster bulk_insert_all_ums.
As noted in the docstring for `bulk_insert_ums`, this is at least one
order of magnitude faster than using `bulk_create`.  This also
includes a `ON CONFLICT DO NOTHING` which allows multiple
soft-reactivations to run at once without failing.  We also adjust the
update of `last_active_message_id` to be safe against races.
2024-03-26 10:52:37 -07:00
Alex Vandiver 7988aad159 user_message: Use INSERT ... ON CONFLICT for historical UM creation.
Rather than use a bulk insert via Django, use the faster
`bulk_insert_all_ums` that we already have.  This also adds a `ON
CONFLICT` clause, to make the insert resilient to race conditions.

There are currently two callsites, with different desired `ON
CONFLICT` behaviours:
 - For `notify_reaction_update`, if the `UserMessage` had already been
   created, we would have done nothing to change it.
 - For `do_update_message_flags`, we would have ensured a specific bit
   was (un)set.

Extend `create_historical_user_messages` and `bulk_insert_all_ums` to
support `ON CONFLICT (...) UPDATE SET flags = ...`.
2024-03-26 10:52:37 -07:00
Alex Vandiver 52e3c8e1b2 user_message: Move create_historical_user_messages to zerver.lib.user_message.
It is not even used in zerver.actions.create_user.
2024-03-26 10:52:37 -07:00
Alex Vandiver a7ec7e924e import: Skip "active" check, and analytics, for crossrealm bots.
The bots do not exist in the user table to look up their active
status, and attempting to import them into the analytics table will
result in duplicate rows.
2024-03-25 10:21:48 -07:00
Aditya Bajaj a20b454ebb
notifications: Shorten string for redacted content.
Replace the long string for organisations that have notification
body/content disabled (settings.PUSH_NOTIFICATION_REDACT_CONTENT
set to true) with "New message".

This allows more of the limited space on the mobile device screen to
be used for additional messages rather than this verbose content.

Fixes #29152
2024-03-22 09:37:51 -07:00
Alex Vandiver fd5a091b30 messages: Only check the UserMessage row if necessary.
For the common case of not needing to reference the UserMessage row
later, and for being a stream without private history, the UserMessage
row is irrelevant.  Convert `has_user_message` to a thunk, and defer
loading it unless necessary.
2024-03-22 09:30:17 -07:00
Alex Vandiver f92d43c690 messages: Use overloads to only return a user_message if needed. 2024-03-22 09:30:17 -07:00
John Lu a5cf0ec526
refactor: Replace HUDDLE with DIRECT_MESSAGE_GROUP.
Replaced HUDDLE attribute with DIRECT_MESSAGE_GROUP using VS Code search,
part of a general renaming of the object class.

Fixes part of #28640.

Co-authored-by: JohnLu2004 <JohnLu10212004@gmail.com>
2024-03-21 16:39:33 -07:00
Anders Kaseorg 224b484195 test_classes: Skip OpenAPI validation on exceptions.
This gives more helpful error tracebacks from failing tests.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-21 16:36:48 -07:00
Alex Vandiver 6ad777c86f dev_urls: Make requests to /docs redirect to the index.
This makes it easier to browse to the docs in development mode, and
more closely matches ReadTheDocs production.
2024-03-21 11:27:14 -07:00
Vector73 f758ca596b custom_profile_fields: Add "required" parameter to the profile fields.
Fixes #28512.
2024-03-21 10:48:54 -07:00
Anders Kaseorg a1a341f0ae ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-21 08:59:17 -07:00
Prakhar Pratyush 118a7e8d9d zulip_updates: Send zulip updates based on zulip_update_*_level.
This commit adds a management command that will run regularly
as a cron job to send zulip updates to realms based on their
current and latest zulip_update_announcements_level.

For realms with:
* level = None: Send a group DM to admins notifying them about
this new feature & suggestion to set the stream accordingly.

* level = 0:
  * If stream is still not configured, wait for a week
    before setting their level to latest level. They will
    miss updates until their configure the stream.
  * If stream is configured, send updates.

* level > 0: Send one message/update per level & increase
  the level by 1 till the latest level.

Fixes #28604.
2024-03-20 11:48:06 -07:00
Prakhar Pratyush 6484532830 help: Migrate configure-notification-bot help page to a new URL.
This commit migrates the 'help/configure-notification-bot' help
page to a new URL i.e. 'help/configure-automated-notices'.
2024-03-20 11:48:06 -07:00
Alex Vandiver f15e006873 message_edit: Perform cross join in SQL, not Python.
The database can do this faster than we can, and it reduces the amount
of information transmitted over the wire.
2024-03-19 18:22:24 -04:00
Alex Vandiver 7ce6095003 message_edit: Handle previous subscribers and private-history streams. 2024-03-19 18:22:24 -04:00
Prakhar Pratyush ad3603c0aa onboarding: Show one-time modal to introduce recent conversations view.
To improve onboarding experience, this commit adds a
one-time modal which introduces the recent conversations view.

Users see this one-time modal on visiting the recent
conversations view.

Fixes #29073.
2024-03-19 09:32:59 -07:00
Prakhar Pratyush 9a7634d527 onboarding: Show one-time modal to introduce inbox view.
To improve onboarding experience, this commit adds
a one-time modal which introduces the inbox view.

Users see this one-time modal on visiting the inbox view.

Fixes part of #29073.
2024-03-19 09:32:59 -07:00
Alex Vandiver c13e3dee24 katex: Replace subprocess call with minimal external service.
Replace a separate call to subprocess, starting `node` from scratch,
with an optional standalone node Express service which performs the
rendering.  In benchmarking, this reduces the overhead of a KaTeX call
from 120ms to 2.8ms.  This is notable because enough calls to KaTeX in
a single message would previously time out the whole message
rendering.

The service is optional because he majority of deployments do not use
enough LaTeX to merit the additional memory usage (60Mb).

Fixes: #17425.
2024-03-15 15:34:12 -07:00
Alex Vandiver c7e92ee2f3 outgoing_http: Support fractional-second timeouts. 2024-03-15 15:34:12 -07:00
Aman Agrawal 2b12c8b4b9 narrow: Fix comment. 2024-03-11 14:13:26 -07:00
Mateusz Mandera b572fa93cd context_managers: Open file in write mode in lockfile_nonblocking.
Otherwise this fails if the file doesn't yet exist.
2024-03-06 10:35:02 -08:00
Mateusz Mandera 6593ccd7fc test_helpers: Remove /self-hosted-billing/ from exempt_patterns.
This was added early on and hasn't been removed since, even though we
have tests for the endpoint for a while now.
2024-03-05 11:53:51 -08:00
Mateusz Mandera e39f400f94 remote_billing: Make "plan management" always available.
Just shows a config error page if the bouncer is not enabled. Uses a new
endpoint for this so that it can work nicely for both browser and
desktop app clients.
It's necessary, because the desktop app expects to get a json response
with either an error or billing_access_url to redirect to. Showing a
nice config error page can't be done via the json error mechanism, so
instead we just serve a redirect to the new error page, which the app
will open in the browser in a new window or tab.
2024-03-05 11:53:51 -08:00
Mateusz Mandera 1ede8da46c events: Change format of authentication_methods realm_update_dict event.
Makes the event in line with state_data.realm_authentication_methods.
2024-03-05 11:48:58 -08:00
Mateusz Mandera da9e4e6e54 backends: Implementation of restricting certain backends by plan.
Only affects zulipchat, by being based on the BILLING_ENABLED setting.

The restricted backends in this commit are
- AzureAD - restricted to Standard plan
- SAML - restricted to Plus plan, although it was already practically
  restricted due to requiring server-side configuration to be done by us

This restriction is placed upon **enabling** a backend - so
organizations that already have a backend enabled, will continue to be
able to use it. This allows us to make exceptions and enable a backend
for an org manually via the shell, and to grandfather organizations into
keeping the backend they have been relying on.
2024-03-05 11:48:58 -08:00
David Rosa 3d63ea20b6 help: Add "View stream subscribers" page.
Documents how to view stream subscribers via settings and
the right sidebar.

Fixes #28887.
2024-03-05 10:22:18 -08:00
Mateusz Mandera 634015411a update_analytics_count: Use a correct lock mechanism.
Adds a re-usable lockfile_nonblocking helper to context_managers.

Relying on naive `os.mkdir` is not enough especially now that the
successful operation of this command is necessary for push notifications
to work for many servers.

We can't use `lockfile` context manager from
`zerver.lib.context_managers`, because we want the custom behavior of
failing if the lock can't be acquired, instead of waiting.
That's because if an instance of this gets stuck, we don't want to start
queueing up more processes waiting forever whenever the cronjob runs
again and fail->exit is preferrable instead.
2024-03-05 10:21:14 -08:00
Alex Vandiver 352dbf9387 caches: Only take the realm_id, not the Realm, as a cache key function.
This saves a hit to the database to fetch the Realm of a UserProfile
that we are trying to flush.
2024-03-04 16:35:57 -08:00
Anders Kaseorg 59b0548433 timezone: Only look up canonical time zones from the system.
Legacy time zone aliases were removed from the Debian tzdata package
in tzdata 2023c-8.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040997

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 17:38:08 -08:00
Anders Kaseorg 7b1bb984b3 ruff: Fix RUF022 `__all__` is not sorted.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 09:30:04 -08:00
Anders Kaseorg 3b114c516c ruff: Fix PLR2044 Line with empty comment.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 09:30:04 -08:00
Anders Kaseorg 87992b8b29 ruff: Fix PERF403 Use a dictionary comprehension instead of a for-loop.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 09:30:04 -08:00