Commit Graph

7108 Commits

Author SHA1 Message Date
Sahil Batra 440f9e397a message_edit: Apply topic edit restrictions to "(no topic)" messages.
Previously, editing topic of "(no topic)" messages was allowed
irrespective of time limit or the "edit_topic_policy" setting.
Since we are working in the direction of having "no topic" messages
feel reasonable, this commit changes the code to not consider them
as a special case and topic editing restrictions apply to them as
well now like all other messages.

We still highlight the topic edit icon in recipient bar without
hovering for "no topic" messages, but it is only shown when user
has permission to edit topics.
2023-04-11 11:07:23 -07:00
Sahil Batra 66693f2101 user_groups: Add allow_nobody_group to access_user_group_for_setting.
This commit adds allow_nobody_group parameter to
access_user_group_for_setting with a default value of True.
2023-04-11 09:02:09 -07:00
Alex Vandiver 3efc0c9af3 workers: Rewrite missedmessage_emails with a worker thread.
The previous implementation leaked database connections, as a new
thread (and thus a new thread-local database connection) was made for
each timer execution.  While these connections were relatively
lightweight in Python, they also incur memory overhead in the
PostgreSQL server itself.  The logic for managing the timer was also
unclear, and the unavoidable deadlock in the stopping logic was rather
unfortunate.

Rewrite with one explicit worker thread which handles the delayed
message sending.  The RabbitMQ consumer creates the database rows, and
notifies the worker to start its 5s timeout.  Because it is controlled
by a condition variable, it does not hold the lock while waiting, and
can be notified to exit.
2023-04-10 17:38:08 -07:00
Lauryn Menard f391bfeec6 emails: Add new onboarding email with guide for organization type.
Adds a new welcome email, `onboarding_zulip_guide`, to be sent four
days after a new user registers with a Zulip organization if the
organization has specified a particular organization type that has
a guide in the corporate `/for/.../` pages. If there is no guide,
then no email is scheduled or sent.

The current `for/communities/` page is not very useful for users
who are not organization administrators, so these onboarding guide
emails are further restricted for those organization types to
only go to new users who are invited/registered as admins for the
organzation.

Adds two database queries for new user registrations: one to get
the organization's type and one to create the scheduled email.

Adds two email logs because the email is sent both to a new user
who registers with an existing organization and to the organization
owner when they register a new organization.

Co-authored by: Alya Abbott <alya@zulip.com>
2023-04-10 08:38:09 -07:00
Lauryn Menard ab00648e3e email: Refactor calculation of delay for scheduled emails.
Refactors the logic for adjusting the delay for sending an email
to not land on a weekend so that it can be used to schedule any
number of onboarding emails we decide to send.

Consolidates duplicate testing into
`zerver/tests/test_email_notifications.py`. The initial test and
function were introduced in commit 610f2cbacf with the test
located in `zerver/tests/test_signup.py`.

Prep commit for adding new welcome / follow up email.
2023-04-10 08:38:09 -07:00
AcKindle3 b0ef8f0822 test: Replace occurences of `uri` with `url`.
In all the tests files, replaced all occurences of `uri` with `url`
appeared in comments, local variablles, function names and their callers.
2023-04-08 16:27:55 -07:00
Prakhar Pratyush d96048b0af test_classes: Rename and refactor 'tornado_redirected_to_list'.
This commit renames the 'tornado_redirected_to_list' context
manager to 'capture_send_event_calls' to improve readability.

It also refactors the function to yield a list of events
instead of passing in a list data structure as a parameter
and appending events to it.
2023-04-07 09:45:26 -07:00
Alex Vandiver e408f069fe uploads: Add a method to copy attachment contents out. 2023-04-07 09:13:48 -07:00
Akarsh Jain f122516e7d notifications: Add link to new location of single moved messages.
Improve the Notification Bot by adding a hyperlink to the new location
of a moved single message. The link will make it easier for users to
find the message in its new context.

Fixes #24604.
2023-04-06 19:04:40 -07:00
Ujjawal Modi d0dbdfa52d user_groups: Send a message on changing user-groups subscribers.
After this commit a notification message is sent to users if they are
added to user_groups by someone else or they are removed from user_groups
by someone else.

Fixes #23642.
2023-04-06 19:03:26 -07:00
Daniil Fadeev 2f203f4de1 emails: Inline CSS in emails in build_email.
Previously, we had an architecture where CSS inlining for emails was
done at provision time in inline_email_css.py. This was necessary
because the library we were using for this, Premailer, was extremely
slow, and doing the inlining for every outgoing email would have been
prohibitively expensive.

Now that we've migrated to a more modern library that inlines the
small amount of CSS we have into emails nearly instantly, we are able
to remove the complex architecture built to work around Premailer
being slow and just do the CSS inlining as the final step in sending
each individual email.

This has several significant benefits:

* Removes a fiddly provisioning step that made the edit/refresh cycle
  for modifying email templates confusing; there's no longer a CSS
  inlining step that, if you forget to do it, results in your testing a
  stale variant of the email templates.
* Fixes internationalization problems related to translators working
  with pre-CSS-inlined emails, and then Django trying to apply the
  translators to the post-CSS-inlined version.
* Makes the send_custom_email pipeline simpler and easier to improve.

Signed-off-by: Daniil Fadeev <fadeevd@zulip.com>
2023-04-05 12:22:29 -07:00
Lauryn Menard e95b784f6e backend-auth: Add user ID to fetch api key responses.
Adds the user ID to the return values for the `/fetch_api_key` and
`/dev_fetch_api_key` endpoints. This saves clients like mobile a
round trip to the server to get the user's unique ID as it is now
returned as part of the log in flow.

Fixes #24980.
2023-04-04 10:54:49 -07:00
Anders Kaseorg a881918a05 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-03 22:39:21 -07:00
Anders Kaseorg 2ce8357009 codespell: Fix newly found typos.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-03 22:39:21 -07:00
Prakhar Pratyush ab7cd3c40a user_topics: Update test to verify end-to-end behaviour.
This commit updates 'test_user_ids_unmuting_topic' to make
an api_post call to '/api/v1/user_topics' instead of
calling the internal function 'do_set_user_topic_visibility_policy'
to verify the logic.
2023-04-03 22:31:49 -07:00
Prakhar Pratyush f012d079c3 user_topics: Add a new endpoint to update visibility_policy.
This commit adds a new endpoint, 'POST /user_topics' which
is used to update the personal preferences for a topic.

Currently, it is used to update the visibility policy of
a user-topic row.
2023-04-03 22:31:49 -07:00
Prakhar Pratyush e33eb45507 user_topics: Update tests to use the 'time_machine' module.
This commit updates the tests to use the 'time_machine'
module for mocking datetime.
2023-04-03 22:31:49 -07:00
Prakhar Pratyush 5c86d3ca9e user_topics: Update tests to verify end-to-end behaviour.
This commit updates the existing tests so that they call
API endpoints instead of internal functions to verify logic.
2023-04-03 22:31:49 -07:00
Alex Vandiver 113a8c4782 export: Make --deactivate-realm exports be imported as active. 2023-04-03 16:08:43 -07:00
Daniil Fadeev 5c6f842b28 emails: Replace Premailer with CSS-inline.
Primary goal of library replacement is improving execution speed.
This commit should not affect the functionality of the system
or make any changes to it.
2023-04-03 15:15:05 -07:00
Mateusz Mandera f3db618cb5 actions: Implement do_delete_user_preserving_messages function.
With an extensive docstring by tabbott documenting its limitations.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2023-03-29 12:51:21 -07:00
Prakhar Pratyush d8a5ca5ddc user_topics: Update test_edit_muted_topic to verify events sent.
Previously, there was a stale code that didn't verify
if 'muted_topics' and 'user_topic' events are sent correctly.

This commit updates the test to verify if the expected
users are notified via 'muted_topics' and 'user_topic'
events.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 117264fb37 user_topics: Update move-topic codepath to utilise bulk database operations.
This commit updates the move-topic codepath to perform
bulk database operations on the UserTopic record using
user_profiles for each visibility_policy instead of
previously looping over each user_profile one by one.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 60762a750b user_topics: Update 'set visibility policy' codepath for bulk db operations.
This commit refactors 'set_user_topic_visibility_policy_in_database'
to perform bulk database operations and the related changes.

There is an increase in database query count because requests
to delete user_topic rows now take two queries instead of one.
This is required for logging the info for a request to delete
a non-existent user_topic row while performing bulk operations
at the same time.

The overall query count will be lower while performing
bulk operations (multiple user_profiles instead of one).
2023-03-29 09:50:06 -07:00
Prakhar Pratyush a890aaf34d user_topics: Update UserTopic records regardless of the visibility_policy.
This commit updates the 'do_update_message' codepath to
update the UserTopic records regardless of visibility policy
during the "move-topic" operation.

This is required before offering new visibility policies
in the UI.

Previously, UserTopic records were moved or deleted only
for objects with a MUTED visibility policy.

Fixes: #24574
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 0377085f15 user_topics: Update a comment regarding the database query count.
This commit updates a stale comment regarding query count
to its correct value.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush e1ec814507 user_topics: Add a test helper to improve readability.
This commit adds a test helper function that helps
to remove line wrappings and improve readability.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 0ec9ff17ce user_topics: Update lib functions to work with any visibility_policy.
This is a prep commit that renames 'set_topic_mutes' and
'topic_is_muted' to 'set_topic_visibility_policy' and
'topic_has_visibility_policy' respectively, and refactors
them to work with any visibility_policy, not only MUTED.
2023-03-29 09:50:06 -07:00
Lauryn Menard 7b225245c0 tests: Update ZulipTestCase.tearDown to remove local uploads.
Previously, tests that exercised code paths that added local
uploads did not always clean up `settings.LOCAL_UPLOADS_DIR`
after the test was complete.

Updates the `ZulipTestCase` class to remove any local uploads
in the unique `settings.LOCAL_UPLOADS_DIR` in `tearDown` for
all tests.
2023-03-28 14:38:06 -07:00
Sahil Batra bed2bf64c4 user_groups: Add "Nobody" system user group.
This commit adds code to create a "Nobody" system user group
to realms which will be used in settings to represent "Nobody"
option.

We also add a migration to add this group to existing realms.
2023-03-28 14:26:22 -07:00
Sahil Batra f684d36710 test_classes: Add submit_realm_creation_form helper.
This commit adds a new helper submit_realm_creation_form,
similar to existing submit_reg_form_for_user, to avoid
duplicate code for creating realms in tests.
2023-03-27 15:44:42 -07:00
Sahil Batra cf8d8db132 register: Update the user-registration form for realm creation.
In previous commits, we updated the realm creation flow to show
the realm name, type and subdomain fields in the first form
when asking for the email of the user. This commit updates the
user registration form to show the already filled realm details
as non-editable text and there is also a button to edit the
realm details before registration.

We also update the sub-heading for user registration form as
mentioned in the issue.

Fixes part of #24307.
2023-03-27 15:44:42 -07:00
Sahil Batra 791c3030eb backends: Update ZulipLDAPAuthBackend to use PreregistrationRealm.
This commit updates ZulipLDAPAuthBackend to use the recently added
PreregistrationRealm when creating realms with LDAP used as
authentication method.
2023-03-27 15:44:42 -07:00
Sahil Batra e60b6b68ea registration: Modify PreregistrationRealm objects after registration. 2023-03-27 15:44:42 -07:00
Sahil Batra 765e65f954 registration: Use PreregistrationRealm object for realm creation.
We now use PreregistrationRealm objects in registration_helper
function when creating new realms instead of PreregistrationUser
objects.

Fixes part of #24307.
2023-03-27 15:44:42 -07:00
Sahil Batra 54771cfe94 registration: Create PreregistrationRealm object when creating realm.
This commit adds code to create PreregistrationRealm object when
creating realm and set it to the content_object field of
Confirmation object.
2023-03-27 15:44:42 -07:00
Joelute 505c217db5 bots: Change logic for notification bot after moved message.
Previously, when a user moves a message to another topic, the Notification
bot will post a message saying "This topic was moved here from..." This is
confusing when the topic already contains messages. The changes aims to make
the messages more clear by changing the logic for the Notification bot. When
there is already messages in the topic, the bot will post "A message was
moved here from..." or "N messages were moved here from...". The bot will
post "This topic was moved here from (somewhere) by (someone)." when the
topic is empty.

Fixes #23267.
2023-03-27 10:00:34 -07:00
Zixuan James Li e331c356e4 user_groups: Use check_add_user_group instead in test cases.
"check_add_user_group" is a safer helper function than
"create_user_group" to use when creating user_groups. It does
error handling and notify the client with the appropriate event.

Note that the populate_db command still uses "create_user_group"
because we do not need to enqueue events at that point.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-03-27 09:05:00 -07:00
Zixuan James Li 0f5d6432a4 user_groups: Move create_user_group to zerver.actions.user_groups.
Since this function creates a new user group into the database,
it is more appropriate to have it not as a generic "lib" function
but as an "action".

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-03-27 09:05:00 -07:00
Alex Vandiver 164d58bec9 web: Save a 301 redirect from /help to /help/. 2023-03-24 14:51:01 -07:00
Alex Vandiver b5731c55b7 web: Save two 301 redirects from /privacy to /privacy/ to /policies/privacy. 2023-03-24 14:51:01 -07:00
Alex Vandiver ce847fd09c web: Save two 301 redirects from /terms to /terms/ to /policies/terms. 2023-03-24 14:51:01 -07:00
Alex Vandiver d9daeb13e0 web: Save a needless 301 redirect from /apps to /apps/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 61adc9cfcc web: Save a needless 301 redirect from /team to /team/. 2023-03-24 14:51:01 -07:00
Alex Vandiver c686c5ed0f web: Save a needless 301 redirect from /plans to /plans/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 5967dda35d web: Save a needless 301 redirect from /upgrade to /upgrade/. 2023-03-24 14:51:01 -07:00
Lauryn Menard 52801f3cd2 narrow: Support underscore version of "pm-with" and "group-pm-with".
Prior to commit a9b3a9c, the server implementation for documented
search operators with dashes, also implicitly supported clients
sending those same operators with underscores. This has been the
case sense the server side support for narrow filtering was
introduced in commit 3af2bf345a.

Updates the stricter version of mapping operator strings to `by*`
functions, to also include the underscore version of any operators
that have dashes. Adds a note that these undocumented versions are
tied to the support for the documented versions.
2023-03-24 11:54:50 -07:00
Sahil Batra ce562c4820 user_settings: Create RealmAuditLog entries for all user settings.
We previously created RealmAuditLog entries for user notification
settings only. This commit changes the code to create entries for
all user settings. We cannot backfill the entries since we don't
have the data to do that.
2023-03-23 10:06:47 -07:00
Lauryn Menard ffcdc13819 create-realm: Update notification message sent to admin realm.
When a new realm is created, a notification message is sent to
the realm configured as the settings.SYSTEM_BOT_REALM if there
is a "signups" stream that exists in that realm. This is used
for Zulip Cloud, but is an undocumented feature.

The topic of the message has been the subdomain of the new realm,
and the message content has been "Signups enabled" translated
into the default language of the new realm.

In order to make these messages more explicitly for Zulip Cloud,
the settings.CORPORATE_ENABLED is checked before sending these
messages.

To make these messages more useful, the topic for these
notifications is changed to be "new organizations". The content
of these messages is updated to have the new realm name (with a
link to the admin realm's activity support page for the realm),
subdomain (with a link to the realm), and organization type.
2023-03-22 17:48:53 -07:00
Alya Abbott e660ffbe07 portico: Add End Point Dev case study page. 2023-03-21 16:20:30 -07:00
Alex Vandiver fa6daee4e1 markdown: Fix use of pure_markdown for non-pure markdown rendering.
`render_markdown_path` renders Markdown, and also (since baff121115)
runs Jinja2 on the resulting HTML.

The `pure_markdown` flag was added in 0a99fa2fd6, and did two
things: retried the path directly in the filesystem if it wasn't found
by the Jinja2 resolver, and also skipped the subsequent Jinja2
templating step (regardless of where the content was found).  In this
context, the name `pure_markdown` made some sense.  The only two
callsites were the TOS and privacy policy renders, which might have
had user-supplied arbitrary paths, and we wished to handle absolute
paths in addition to ones inside `templates/`.

Unfortunately, the follow-up of 01bd55bbcb did not refactor the
logic -- it changed it, by making `pure_markdown` only do the former
of the two behaviors.  Passing `pure_markdown=True` after that commit
still caused it to always run Jinja2, but allowed it to look elsewhere
in the filesystem.

This set the stage for calls, such as the one introduced in
dedea23745, which passed both a context for Jinja2, as well as
`pure_markdown=True` implying that Jinja2 was not to be used.

Split the two previous behaviors of the `pure_markdown` flag, and use
pre-existing data to control them, rather than an explicit flag.  For
handling policy information which is stored at an absolute path
outside of the template root, we switch to using the template search
path if and only if the path is relative.  This also closes the
potential inconsistency based on CWD when `pure_markdown=True` was
passed and the path was relative, not absolute.

Decide whether to run Jinja2 based on if a context is passed in at
all.  This restores the behavior in the initial 0a99fa2fd6 where a
call to `rendar_markdown_path` could be made to just render markdown,
and not some other unmentioned and unrelated templating language as
well.
2023-03-17 08:46:25 -07:00
Alex Vandiver 330141f55d invites: Switch new LIMITED-plan heuristic to enforcing. 2023-03-16 11:41:49 -07:00
Alex Vandiver 50a2a54393 test_invite: Rework and expand invitation limit tests.
This adds tests for more corner cases, in exchange for dropping the
query count tests, which were of dubious utility.  It also adds the
time-machine library to mock the current time to test that the limits
do expire.
2023-03-16 11:41:49 -07:00
Alex Vandiver 6971c6d62d test_invite: Just import django.core.mail and use it. 2023-03-16 11:41:49 -07:00
Alex Vandiver 9e972bc82c test_invite: Clarify by filters that no *new* ScheduledEmails are made. 2023-03-16 11:41:49 -07:00
Alex Vandiver 83fdc59304 test_invite: Remove a totally uninteresting assert. 2023-03-16 11:41:49 -07:00
Alex Vandiver 566806fb41 test_invite: Fix a docstring. 2023-03-16 11:41:49 -07:00
Alex Vandiver 695730b609 test_invite: Explicitly test that no emails are actually sent. 2023-03-16 11:41:49 -07:00
Alex Vandiver 34c8cd1b74 tests: Split out test_invite from test_signup.
There is no good reason for this single test file to be more than 6k
lines.
2023-03-16 11:41:49 -07:00
Prakhar Pratyush 042bbf2936 UserTopic: Set visibility_policy or delete UserTopic row without error.
This commit updates 'set_user_topic_visibility_policy_in_database'
to not raise an error when deleting a UserTopic row and the user
doesn't have a visibility_policy for the topic yet, or when setting
the visibility_policy to its current value.

Also, it includes the changes to not send unnecessary events
in such cases.
2023-03-16 10:55:54 -07:00
Alya Abbott 584124bb45 help: Improve "Deactivate your account" help page. 2023-03-15 15:32:47 -07:00
Abhijeet Prasad Bodas ba0dd70904 user_topics: Introduce visibility policy enum subclass in models.py.
This commit gives more readable code than using the `VISIBILITY_POLICY`
prefix.
This commit does not alter the database schema.
2023-03-14 17:50:45 -07:00
Prakhar Pratyush ae72777c77 user_settings: Automate 'Include realm name in message email subject'.
Currently, there is a checkbox setting for whether to
"Include realm name in subject of message notification emails".

This commit replaces the checkbox setting with a dropdown
having values: Automatic [default], Always, Never.

The Automatic option includes the realm name if, and only if,
there are multiple Zulip realms associated with the user's email.

Tests are added and(or) modified.

Fixes: #19905.
2023-03-14 17:48:57 -07:00
Alya Abbott 005ca2b033 portico: Add landing page about trying Zulip by visiting chat.zulip.org. 2023-03-14 13:21:09 -07:00
Lauryn Menard 213d0f4990 create-user: Remove notifications sent to admin realm.
Removes the notification message that was sent if a stream named
"signups" exists in the `settings.SYSTEM_BOT_REALM`. This was a
undocumented feature that would send a notification message when
a new user registered with a Zulip organization that was hosted
by an admin realm like Zulip Cloud.

This removes two database queries when a new user is created: one
to get the system bot realm and the other to get the notification
bot in said realm.

Note that there are still notification messages sent when a new
organization is registered with the admin realm if the "signups"
stream exists.
2023-03-13 12:28:26 -07:00
Aman Agrawal dbe930394f footer: Integrate newly designed footer.
This footer was designed and mostly written by @terpimost.

This adds a new design of the footer for both corporate and not
corporate enabled pages.
2023-03-10 17:34:15 -08:00
Anders Kaseorg 3bfbfb014a zilencer: Switch a log message back from %r to %s.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Anders Kaseorg 2d9b2a2a05 models: Remove type prefixes from __str__ values.
The Django convention is for __repr__ to include the type and __str__
to omit it.  In fact its default __repr__ implementation for models
automatically adds a type prefix to __str__, which has resulted in the
type being duplicated:

    >>> UserProfile.objects.first()
    <UserProfile: <UserProfile: emailgateway@zulip.com <Realm: zulipinternal 1>>>

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Aman Agrawal c928c87645 google_analytics: Track realm registration separately from user signup.
While the function which processes the realm registration and
signup remains the same, we use different urls and functions to
call the process so that we can separately track them. This will
help us know the conversion rate of realm registration after
receiving the confirmation link.
2023-03-08 11:34:36 -08:00
Alex Vandiver 8f8a9f6f04 sentry: Add frontend event monitoring.
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors.  However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors.  Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.

Add a client-side Javascript sentry integration.  To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry.  Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
2023-03-07 10:51:45 -08:00
Alex Vandiver 73631950a5 report: Use ZULIP_VERSION rather than shelling out to git every time. 2023-03-07 10:51:45 -08:00
Alex Vandiver 90b1e0b8b9 report: Provide user information for browser-side errors.
b4dd118aa1 changed how the `user_info_str` parsed information out of
the events it received -- but only changed the server errors, not the
browser errors, though both use the same codepath.  As a result, all
browser errors since then have been incorrectly marked as being for
anonymous users.

Build and pass in the expected `user` dict into the event.
2023-03-07 10:51:45 -08:00
Kartik Srivastava ce5d13f9b2 message_send: Handle notifications for UNMUTED topic in a muted stream.
This commit adds 'visibility_policy' as a
parameter to user_allows_notifications_in_StreamTopic
function.

This adds logic inside the user_allows_notifications_in_StreamTopic
function, to not return False when a stream is muted
but the topic is UNMUTED.

Adds a method `user_id_to_visibility_policy_dict`
to 'StreamTopicTarget' class to fetch
(user_id => visibility_policy) in single db query.

Co-authored-by: Kartik Srivastava <kaushiksri0908@gmail.com>
Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-03-06 19:15:45 -08:00
Kartik Srivastava e9cf2659cf user_topics: Refactor remove_topic_mute.
This commit replaces 'remove_topic_mute' with
'set_user_topic_visibility_policy_in_database' and
updates it to delete UserTopic row with any configured
visibility_policy and not just muting.
2023-03-06 19:15:45 -08:00
Kartik Srivastava f844cb6dad user_topics: Refactor add_topic_mute.
In order to support different types of topic visibility policies,
this renames 'add_topic_mute' to
'set_user_topic_visibility_policy_in_database'
and refactors it to accept a parameter 'visibility_policy'.

Create a corresponding UserTopic row for any visibility policy,
not just muting topics.

When a UserTopic row for (user_profile, stream, topic, recipient_id)
exists already, it updates the row with the new visibility_policy.

In the event of a duplicate request, raises a JsonableError.
i.e., new_visibility_policy == existing_visibility_policy.

There is an increase in the database query count in the message-edit
code path.

Reason:
Earlier, 'add_topic_mute' used 'bulk_create' which either
creates or raises IntegrityError -- 1 query.

Now, 'set_user_topic_visibility_policy' uses get_or_create
-- 2 queries in the case of creating new row.

We can't use the previous approach, because now we have to
handle the case of updating the visibility_policy too.
Also, using bulk_* for a single row is not the correct way.

Co-authored-by: Kartik Srivastava <kaushiksri0908@gmail.com>
Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-03-06 19:15:45 -08:00
Prakhar Pratyush e9580f8c5a MutedTopicsTests: Remove real-time usage from tests.
This commit refactors the existing pattern (real-time usage)
used to assert 'date_muted' in tests.

A fixed value is used at the start of the test to
assert 'date_muted', replacing the timedelta or real-time usage pattern.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 826ea4162e user_topics: Refactor 'do_unmute_topic'.
Replaces 'do_unmute_topic' with 'do_set_user_topic_visibility_policy'
and associated minor changes.

This change is made to align with the plan to use a single function
'do_set_user_topic_visibility_policy' to manage
user_topic - visibility_policy changes and corresponding event
generation.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 2df2ef9f0f user_topics: Refactor 'do_mute_topic'.
This commit is a step in the direction of having a common
function to handle visibility_policy changes and event
generation instead of separate functions for each
visibility policy.

In order to support different types of topic visibility policies,
this renames 'do_topic_mute' to 'do_set_user_topic_visibility_policy'
and refactors it to accept a parameter 'visibility_policy'.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 9ecc610681 mute_topic: Update tests to not use lib functions directly.
The "add_topic_mute" and "remove_topic_mute" library functions
shouldn't be called directly from tests.
They should instead call "do_mute_topic" and "do_unmute_topic"

The reason being:
Library functions are meant to be internal interfaces
for just changing the database, and shouldn't generally be
called elsewhere.
2023-03-06 19:15:45 -08:00
Lauryn Menard e9bfdd1bf2 response: Implement ignored parameters with MutableJsonResponse class.
Creates `MutableJsonResponse` as a subclass of Django's `HttpResponse`
that we can modify for ignored parameters in the response content.

Updates responses to include `ignored_parameters_unsupported` in
the response data through `has_request_variables`. Creates unit
test for this implementation in `test_decorators.py`.

The `method` parameter processed in `rest_dispatch` is not in the
`REQ` framework, so for any tests that pass that parameter, assert
for the ignored parameter with a comment.

Updates OpenAPI documentation for `ignored_parameters_unsupported`
being returned in the JSON success response for all endpoints.
Adds detailed documentation in the error handling article, and
links to that page in relevant locations throughout the API docs.

For the majority of endpoints, the documentation does not include
the array in any examples of return values, and instead links to
the error handling page. The exceptions are the three endpoints
that had previously supported this return value. The changes note
and example for these endpoints is also used in the error
handling page.
2023-03-06 10:33:13 -08:00
Lauryn Menard 0f2472ed14 tests: Update `assert_json_success` to check for ignored parameters.
Updates `assert_json_success` so that it checks for an array of
ignored parameters in the success response.
2023-03-06 10:33:13 -08:00
Lauryn Menard 084dd216f0 tests: Remove ignored `realm_str` parameter from message send test.
In commit 8181ec4b56, we removed the `realm_str` as a parameter
for `send_message_backed`. This removes a missed test that included
this as a parameter for that endpoint/function.
2023-03-06 09:41:53 -08:00
Anders Kaseorg 087660a87e requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg d3efd4c095 python: Import F, Q, QuerySet from their canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg aa577a554b populate_db: Import timedelta from its canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Alex Vandiver 3bf3f47b49 delete_old_unclaimed_attachments: Add flag to clean up storage.
Actions like deleting realms may leave unreferenced uploads in the
attachment storage backend.

Fix these by walking the complete contents of the attachment storage
backend, and removing files which are no longer present in the
database.  This may take quite some time, as it is necessarily O(n) in
the number of files uploaded to the system.
2023-03-02 16:36:19 -08:00
Alex Vandiver c9d1755a12 delete_realm: Optimize attachment cleanup by batching. 2023-03-02 16:36:19 -08:00
Alex Vandiver cdda4bc089 delete_realm: Clean up attachments when deleting or scrubbing realms. 2023-03-02 16:36:19 -08:00
Alex Vandiver b31a6dc56c upload: Reorder functions into logical groupings. 2023-03-02 16:36:19 -08:00
Alex Vandiver 880a3f95a7 tests: Split out s3 and local tests.
This mirrors the split of the code in 7c0d414aff.
2023-03-02 16:36:19 -08:00
Alex Vandiver bd80c048be upload: Rename delete_message_image to use word "attachment".
The table is named Attachment, and not all of them are images.
2023-03-02 16:36:19 -08:00
Alex Vandiver 567d1d54e7 upload: Rename upload_message_file to use word "attachment".
For consistency with the table, which is named Attachment.
2023-03-02 16:36:19 -08:00
Sahil Batra 3950a8e19d test_helpers: Rename reset_emails_in_zulip_realm.
This commit renames reset_emails_in_zulip_realm function to
reset_email_visibility_to_everyone_in_zulip_realm which makes
it more clear to understand what the function actually does.

This commit also adds a comment explaining what this function
does.
2023-03-01 12:17:11 -08:00
Mateusz Mandera f0f02d05ab send_message_backend: Remove the realm_str API param.
This already became useless in 6e11754642,
as detailed in the API changelog entry here. At this point, we should
eliminate this param and the weird code around it.

This commit also deletes the associated tests added in
6e11754642, since with realm_str removed,
they make no sense anymore (and actually fail with an OpenAPI error due
to using params not used in the API). Hypothetically they could be
translated to use the subdomain= kwarg, but that also doesn't make
sense, since at that point they'd be just testing the case of a user
making an API request on a different subdomain than their current one
and that's just redundant and already tested generally in
test_decorators.
2023-02-28 12:12:15 -08:00
Lauryn Menard 514e5b990e templates: Update `reset_done.html` page text and title.
Updates the text and title used when the password reset done page
to work for situations where the user is resetting a forgotten
password and for situation where the user is setting a password
for the first time (e.g. SSO login, demo organizations).
2023-02-28 11:45:01 -08:00
Alex Vandiver 015a10637b docs: Document how to use SMTP without authentication.
This is the behaviour inherited from Django[^1].  While setting the
password to empty (`email_password = `) in
`/etc/zulip/zulip-secrets.conf` also would suffice, it's unclear what
the user would have been putting into `EMAIL_HOST_USER` in that
context.

Because we previously did not warn when `email_password` was not
present in `zulip-secrets.conf`, having the error message clarify the
correct configuration for disabling SMTP auth is important.

Fixes: #23938.

[^1]: https://docs.djangoproject.com/en/4.1/ref/settings/#std-setting-EMAIL_HOST_USER
2023-02-27 11:59:48 -08:00
Alex Vandiver fe654b76b7 data_import: Stop tar'ing up converted data.
`./manage.py import` does not take a tarball; it takes a directory.
Making a separate tarball is a waste of CPU time and disk, as it is
never used.

This was included in the commit of the initial Slack conversion code
in 5b37c5562b and propagated from there into every conversion tool.

Remove the unnecessary tarball creation.
2023-02-26 17:42:01 -08:00
m-e-l-u-h-a-n ab4e6a94c5 user groups: Make name and description optional in group update.
View that handled `PATCH user_groups/<int:user_group_id>` required
both name and description parameters to be passed. Due to this
clients had to pass values for both these parameters even if
one of them was changed.

To resolve this name description parameters to
`PATCH user_groups/<int:user_group_id>` are made optional.
2023-02-26 16:22:24 -08:00
Lauryn Menard e39c689f36 email-notifications: Update email subjects with "PM" to "DM".
Updates email notification subjects with "PM" to use "DM" instead.

Also, updates `tools/lib/capitalization.py` to include both "DM"
and "DMs".
2023-02-24 11:47:26 -08:00
Lauryn Menard 182e6c0730 push-notifications: Update strings for private messages.
Updates strings with "private message" in push notifications to
use "direct message" instead.
2023-02-24 11:47:26 -08:00
Lauryn Menard a0fd7b2afc private-messages: Update translated backend strings use "direct message".
Updates user-facing translated strings containing "private message" on
the backend to use "direct message" instead.
2023-02-24 11:47:26 -08:00
Sahil Batra 36584a3571 registration: Add code to set email_address_visibility during signup.
This commit adds backend code to set email_address_visibility when
registering a new user. The realm-level default and the value of
source profile gets overridden by the value user selected during
signup.
2023-02-24 09:23:34 -08:00
Anders Kaseorg cea1119423 node_tests: Move to web/tests.
This lets us simplify the long-ish ‘../../static/js’ paths, and will
remove the need for the ‘zrequire’ wrapper.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Alex Vandiver f78c2eab2c message_send: Use stream object for name in "no subscribers" message.
In the case where a stream existed but had no subscribers, the error
message used to send to the owner always used `stream_name`, which
may have been None.

Switch to using `stream.name` rather than `stream_name` for this case.
2023-02-23 11:02:55 -08:00
Anders Kaseorg ed069ebe0e docs: Remove spaces before commas.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-22 17:17:25 -08:00
Sahil Batra 0cf99cf5c3 streams: Refactor code to set group-based stream settings.
We add stream_permission_group_settings object which is
similar to property_types framework used for realm settings.

This commit also adds GroupPermissionSetting dataclass for
defining settings inside stream_permission_group_settings.

We add "do_change_stream_group_based_setting" function which
is called in loop to update all the group-based stream settings
and it is now used to update 'can_remove_subscribers_group'
setting instead of "do_change_can_remove_subscribers_group".

We also change the variable name for event_type field of
RealmAuditLog objects to STREAM_GROUP_BASED_SETTING_CHANGED
since this will be used for all group-based stream settings.

'property' field is also added to extra_data field to identify
the setting for which RealmAuditLog object was created.

We will add a migration in further commits which will add the
property field to existing RealmAuditLog objects created for
changing can_remove_subscribers_group setting.
2023-02-22 12:17:46 -08:00
Mateusz Mandera a593089770 presence: Return some presence params in the /register response. 2023-02-21 12:10:43 -08:00
Abhijeet Prasad Bodas 6e001d0672 user topic: Remove unnecessary check for double addition of muted topic.
This makes use of the new case insensitive UNIQUE index added in the
earlier commit. With that index present, we can now rely solely on the
database to correctly identify duplicates and throw integrity errors as
required.
2023-02-20 21:04:13 -08:00
Abhijeet Prasad Bodas 9fde88796a mute user: Remove unnecessary check for double muting.
In 141b0c4, we added code to handle races caused by duplicate muting
requests. That code can also handle the non-race condition, so we don't
require the first check.
2023-02-20 21:04:13 -08:00
Lauryn Menard 06dd6f8254 message-send: Deduplicate check of `settings.MAX_MESSAGE_LENGTH`.
Removes the initial check in `_internal_prep_message` of the length
of the message content because the `check_message` in the try block
will call `normalize_body` on the message content string, which
does a more robust check of the message content (empty string, null
bytes, length). If the message content length exceeds the value of
`settings.MAX_MESSAGE_LENGTH`, then it is truncated based on that
value. Updates associated backend test for these changes.

The removed length check would truncate the message content with a
hard coded value instead of using the value for
`settings.MAX_MESSAGE_LENGTH`.

Also, removes an extraneous comment about removing null bytes. If
there are null bytes in the message content, then `normalize_body`
will raise an error.

Note that the previous check had intentionally reduced any message over
the 10000 character limit to 3900 characters, with the code in
question dating to 2012's 100df7e349.

The 3900 character truncating rule was implemented for incoming emails
with the email gateway, and predated other features to help with
overly long messages (better stripping of email footers via Talon,
introduced in f1f48f305e, and
condensing, introduced in c92d664b44).
While we could preserve that logic if desired, it likely is no longer
a necessary or useful variation from our usual truncation rules.
2023-02-20 12:55:15 -08:00
Lauryn Menard b25cba16a7 tests: Remove ignored parameter from `authenticated_json_view` test.
The password parameter being passed in the `_do_test` helper
function for `TestAuthenticatedJsonPostViewDecorator` tests was
being ignored, as the user needs to be logged in. Removes the
parameter from the helper function and updates the success test
to use `assert_json_success` instead of just checking the status
code.

Also adds a test case for when a user is not logged in to confirm
that it returns an UnauthorizedError.
2023-02-20 11:56:00 -08:00
Alex Vandiver 8998aa00cd Revert "create_user: Use transaction.atomic decorator for do_create_user."
This reverts commit 851d68e0fc.

That commit widened how long the transaction is open, which made it
much more likely that after the user was created in the transaction,
and the memcached caches were flushed, some other request will fill
the `get_realm_user_dicts` cache with data which did not include the
new user (because it had not been committed yet).

If a user creation request lost this race, the user would, upon first
request to `/`, get a blank page and a Javascript error:

    Unknown user_id in get_by_user_id: 12345

...where 12345 was their own user-id.  This error would persist until
the cache expired (in 7 days) or something else expunged it.

Reverting this does not prevent the race, as the post_save hook's call
to flush_user_profile is still in a transaction (and has been since
168f241ff0), and thus leaves the potential race window open.
However, it much shortens the potential window of opportunity, and is
a reasonable short-term stopgap.
2023-02-17 21:20:52 -05:00
Alex Vandiver f8a16fb9af model: Handle parsed user-agents longer than 30 characters.
The Client.name field is only 30 characters long, but there is no
limit to the length of parsed User-Agent value which we may attempt to
store in it.  This can cause requests with long user-agents to 500
when the creation of the Client row fails.

Truncate the name at 30 characters for the cache key, and passing
`name` to `get_or_create`.
2023-02-14 14:58:15 -08:00
Alex Vandiver dd21e472e7 subdomains: Extend tests for for same-host when STATIC_URL has domain. 2023-02-14 17:17:06 -05:00
Alex Vandiver 6806bec2e9 email_notifications: Use stable, unhashed, URLs for emoji in emails. 2023-02-14 17:17:06 -05:00
Alex Vandiver e31767dda4 settings: Make DEFAULT_LOGO_URI/DEFAULT_AVATAR_URI use staticfiles. 2023-02-14 17:17:06 -05:00
Sahil Batra 9d1dc20e6e settings: Remove realm-level email_address_visibility setting.
This was replaced by the new user-level version in recent commits.

Fixes #20035.
Fixes #18149.
2023-02-10 17:40:33 -08:00
Sahil Batra 0ed5f76063 settings: Add backend code for using user email_address_visibility setting.
This commits update the code to use user-level email_address_visibility
setting instead of realm-level to set or update the value of UserProfile.email
field and to send the emails to clients.

Major changes are -

- UserProfile.email field is set while creating the user according to
RealmUserDefault.email_address_visbility.

- UserProfile.email field is updated according to change in the setting.

- 'email_address_visibility' is added to person objects in user add event
and in avatar change event.

- client_gravatar can be different for different users when computing
avatar_url for messages and user objects since email available to clients
is dependent on user-level setting.

- For bots, email_address_visibility is set to EVERYONE while creating
them irrespective of realm-default value.

- Test changes are basically setting user-level setting instead of realm
setting and modifying the checks accordingly.
2023-02-10 17:35:49 -08:00
Sahil Batra ea0b2286e0 settings: Add email_address_visbility user setting. 2023-02-10 17:35:49 -08:00
Sahil Batra 2396e6858f users: Always pass "delivery_email" field in user objects.
Previously, user objects contained delivery_email field
only when user had access to real email. Also, delivery_email
was not present if visibility setting is set to "everyone"
as email field was itself set to real email.

This commit changes the code to pass "delivery_email" field
always in the user objects with its value being "None" if
user does not have access to real email and real email otherwise.
The "delivery_email" field value is None for logged-out users.
For bots, the "delivery_email" is always set to real email
irrespective of email_address_visibility setting.

Also, since user has access to real email if visibility is set
to "everyone", "delivery_email" field is passed in that case
too.

There is no change in email field and it is same as before.

This commit also adds code to send event to update delivery_email
field when email_address_visibility setting changes to all the
users whose access to emails changes and also changes the code to
send event on changing delivery_email to users who have access
to email.
2023-02-10 17:35:49 -08:00
Anders Kaseorg 41aadc8c63 message_send: Convert RecipientInfoResult from TypedDict to dataclass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-10 15:51:22 -08:00
Abhijeet Prasad Bodas 7d02b20bdb user topics: Rename test file.
This makes the file name in line with the model name and other files,
for better discoverability.

This probably should have been a part of
48883f1580.
2023-02-10 15:39:57 -08:00
Abhijeet Prasad Bodas 9d1c131dc6 muted users: Make file naming consistent.
This makes the names of the relevant files consistant with the database
model name as well as the frontend JS files.
2023-02-10 15:39:57 -08:00
Anders Kaseorg 0a1904a6a7 markdown: Rewrite YouTube URL parser without regex spaghetti.
This also adds support for the new YouTube Shorts URLs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-09 22:34:51 -08:00
Sahil Batra 891f83601d message_edit: Use move_messages_between_streams_limit_seconds setting.
This commit adds time restriction on moving messages between streams
using the move_messages_between_streams_limit_seconds setting in the
backend. There is no time limit for admins and moderators.
2023-02-08 12:46:05 -08:00
Sahil Batra 2c4e076fef message_edit: Use move_messages_within_stream_limit_seconds setting.
We now use the newly added move_messages_within_stream_limit_seconds
setting to check for how long the user can edit the topic replacing
the previously used 3-day limit. As it was previously, there is no
time limit for admins and moderators.
2023-02-08 12:46:05 -08:00
Sahil Batra b919dfd489 realm: Add time limit setting for moving messages between streams.
This commit adds "move_messages_between_streams_limit_seconds"
setting which would be used to set a time limit to move messages
between streams.
2023-02-08 12:46:05 -08:00
Sahil Batra 73f0eae394 realm: Add time limit setting for moving message within stream.
This commit adds "move_messages_within_streams_limit_seconds"
setting which would be used to set a time limit to move messages
within stream.
2023-02-08 12:46:05 -08:00
Alex Vandiver e3c976fed9 missed_messages: Include messages from topic which differ by case.
In Zulip, message topics are case-insensitive but case-preserving.
The `get_context_for_message` function erroneously did a
case-sensitive search, and thus only messages whose topic matched
exactly were pulled in as context.

Make the missed-message pipeline aware that message topics are not
case-sensitive.  This means that, when collapsing adjacent messages,
we merge messages with topic headers which are "different"; create a
separate explicit "grouping" to know which to collapse.
2023-02-07 11:04:21 -08:00
Prakhar Pratyush 906ff9243a user_mutes: Rename 'muting.py' to 'user_mutes.py'.
Rename 'muting.py' to 'user_mutes.py' because it, now
, contains only user-mute related functions.

Includes minor refactoring needed after renaming the file.
2023-02-07 00:23:47 +05:30
Prakhar Pratyush 49577bbdcd user_topics: Move topic muting functions to user_topics.py.
This commit moves topic related stuff i.e. topic muting functions
to a separate file 'views/user_topics.py'.

'views/muting.py' contains functions related to user-mutes only.
2023-02-07 00:23:47 +05:30
Kartik Srivastava 48883f1580 tests: Rename test_muting_topics to test_topic_visibility_policies.
As we'll be adding support for more visibility policies for topics,
this renamed file will contain the necessary tests.
2023-02-07 00:23:47 +05:30
Sahil Batra 73f11853ec streams: Allow setting can_remove_subscribers_group_id while creating streams.
This commit adds API support to set can_remove_subscribers_group setting
when creating streams.
2023-02-05 14:46:36 -08:00
Sahil Batra c3759814be streams: Allow changing can_remove_subscribers_group through API.
This commit adds API support to change can_remove_subscribers_group
setting for a stream.
2023-02-05 14:46:36 -08:00
Anders Kaseorg da3cf5ea7a ruff: Fix RSE102 Unnecessary parentheses on raised exception.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-04 16:34:55 -08:00
Prakhar Pratyush 9997131df3 message_edit: Fix resolve topic notifications.
When 'resolve|unresolve' and 'move stream' actions occurs in
the same api call, 'This topic was marked as resolved|unresolved'
notification is not sent.

Both 'topic moved' and 'topic resolved' notification should be generated.

This commit updates the logic of when and where to send
'topic resolve|unresolve' notification. Unlike previous logic, notification
may be sent even in the case 'new_stream' is not None.

In general, 'topic resolved|unresolved' notification is sent to
'stream_being_edited'. In this particular case ('new_stream' is not None),
notification is sent to the 'new_stream' after check.

Test case is included.

Fixes: #22973
2023-02-03 18:20:39 -08:00
Alessandro Toppi ff89590558 auth: Add JWT-based user API key fetch.
This adds a new endpoint /jwt/fetch_api_key that accepts a JWT and can
be used to fetch API keys for a certain user. The target realm is
inferred from the request and the user email is part of the JWT.

A JSON containing an user API key, delivery email and (optionally)
raw user profile data is returned in response.
The profile data in the response is optional and can be retrieved by
setting the POST param "include_profile" to "true" (default=false).

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-02-03 15:23:35 -08:00
Mateusz Mandera 100f4a7152 auth: Add @require_post to remote_user_jwt. 2023-02-03 15:23:35 -08:00
Mateusz Mandera d466da1064 auth: Use REQ for getting token value and rename param in jwt paths.
This makes us use REQ properly instead of fetching from request.POST
manually - also renaming the param to "token" which is more standard.
2023-02-03 15:23:35 -08:00
Mateusz Mandera c9c255b3a8 auth: Improve JsonableError in get_..._jwt_authentication_request. 2023-02-03 15:23:35 -08:00
Mateusz Mandera 6c638a1057 auth: Extract token-check logic of remote_user_jwt.
This will be useful for re-use for implementation of an endpoint for
obtaining the API by submitting a JWT in the next commits.

It's not a pure refactor, as it requires some tweaks to remote_user_jwt
behavior:
1. The expected format of the request is changed a bit. It used to
   expect "user" and "realm" keys, from which the intended email was
   just generated by joining with @. Now it just expects "email"
   straight-up. The prior design was a bt strange to begin with, so this
   might be an improvement actually.
2. In the case of the codepath of new user signup, this will no longer
   pre-populate the Full Name in the registration form with the value
   from the "user" key. This should be a very minor lost of
   functionality, because the "user" value was not going to be a proper
   Full Name anyway. This functionality can be restored in a future
   commit if desired.

This is an API change, but this endpoint is nearly unused as far as
we're aware.
2023-02-03 15:23:35 -08:00
Aman Agrawal c3f0d5898a index: Remove `Loading...` text from the app loading overlay.
The new loading spinner animation conveys the message well
that the app is loading and this text is no longer required.
2023-02-03 10:51:39 -08:00
Lauryn Menard fc54ffd778 documentation: Move check for `api-doc-template.md`.
Moves the check for calling the `api-doc-template.md` directly,
so that we don't return a 500 error from the server, to happen
earlier with other checks for returning a 404 / missing page.

Also adds a specific test to `zerver/tests/test_urls` for this
template.

Prep commit for moving API documentation directory to be a top
level directory.
2023-02-02 17:23:31 -08:00
Alya Abbott 27b703f617
emails: Improve followup_day1 (registration confirmation) email.
- Clean up the language.
- Add a prominent "Go to organization" button.
- Link to guides for new users and admins.
- Fix duplication bug in text email version.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2023-02-02 17:16:43 -08:00
Anders Kaseorg df001db1a9 black: Reformat with Black 23.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.

(This does not actually upgrade our Python environment to Black 23
yet.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-02 10:40:13 -08:00
Lalit 4ab824dc4c
emoji: Disallow `.` in custom emoji names.
Until now, custom emojis with "periods" in their name were allowed, even though
they don't really fit the pattern of how we name them, and in fact the Markdown
processor would not render such custom emoji. Fix this by just disallowing the
character.

Also update the error strings accordingly.

Note that this does not include a migration to eliminate any existing custom emoji with this 
character in their name.

Fixes #24066.
2023-01-31 17:28:33 -08:00
Alex Vandiver 92c8c17190 import: Add the UTF-8 flag on file entries in zipfiles from Slack.
Fixes: #22533.
2023-01-31 16:07:48 -08:00
N-Shar-ma 282bf680f2 emoji: Replace `built-in` with `default` in emoji overriding API error. 2023-01-31 15:52:33 -08:00
Sahil Batra e14b1f03fa register: Refactor template for new account registration.
This commit refactors the template code for source-realm
select element to have same structure as other inputs
and select element in the page. Thus this change also
makes the styling of source-realm select element consistent
with other select element in the page.
2023-01-26 12:36:31 -08:00
Sahil Batra 851d68e0fc create_user: Use transaction.atomic decorator for do_create_user.
We change the do_create_user function to use transaction.atomic
decorator instead of using with block. Due to this change, all
send_event calls are made inside transaction.on_commit.

Some other changes -
- Remove transaction.atomic decorator from send_inital_realm_messages
since it is now called inside a transaction.
- Made changes in tests which tests message events and notifications
to make sure on_commit callbacks are executed.
2023-01-26 10:49:19 -08:00