Commit Graph

16644 Commits

Author SHA1 Message Date
Mateusz Mandera 2a45429a51 zilencer: Delete duplicate remote push registrations.
This fixes existing instances of the bug fixed in the previous commit.

Fixes #24969.
2023-04-13 15:17:20 -07:00
Mateusz Mandera ade2225f08 zilencer: Avoid creating duplicate remote push registrations.
Servers that had upgraded from a Zulip server version that did not yet
support the user_uuid field to one that did could end up with some
mobile devices having two push notifications registrations, one with a
user_id and the other with a user_uuid.

Fix this issue by sending both user_id and user_uuid, and clearing
2023-04-13 15:17:20 -07:00
Alex Vandiver d888bb3df2 error-bot: Remove ERROR_BOT support.
This isn't sufficiently useful to keep the added complexity.  Users
should use the email error reporting, or set up Sentry error
reporting.
2023-04-13 14:59:58 -07:00
Alex Vandiver db07b8668f error_notify: Consolidate email and zulip codepaths. 2023-04-13 14:59:58 -07:00
Alex Vandiver daba72c116 error_notify: Drop any remaining browser-side errors in RabbitMQ queue. 2023-04-13 14:59:58 -07:00
Alex Vandiver e536a14b61 report_error: Remove API endpoint for client error reporting. 2023-04-13 14:59:58 -07:00
Alex Vandiver cb7bc1b7b9 report_error: Remove reference to old non-existant path. 2023-04-13 14:59:58 -07:00
Alex Vandiver 52c4cae239 blueslip: Remove unused ui_message / show_ui_msg codepath.
This was last used in 71e14674aa, a decade ago.
2023-04-13 14:59:58 -07:00
Anders Kaseorg 0a3dc8a944 ruff: Fix DJ012 Order of model's inner classes, methods, and fields.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-12 17:32:38 -07:00
Sahil Batra 6e4c844907 tests: Fix comment about number of database queries.
This commit fixes the comment about number of database queries
when moving message from muted topic to mention clearly about
the number of queries added due to original topic being muted.

We do not include the queries that is executed to check whether
the topic is muted or not, as they will be executed in all cases.
2023-04-11 11:07:23 -07:00
Sahil Batra bd7f728796 message: Don't allow moving messages that have passed the time limit.
We previously allowed moving messages that have passed the time limit
using "change_all" value for "propagate_mode" parameter. This commit
changes the behavior to not allow moving messages (both stream and
topic edit) that have passed the time limit for non-admin and
non-moderator users.
2023-04-11 11:07:23 -07:00
Sahil Batra 440f9e397a message_edit: Apply topic edit restrictions to "(no topic)" messages.
Previously, editing topic of "(no topic)" messages was allowed
irrespective of time limit or the "edit_topic_policy" setting.
Since we are working in the direction of having "no topic" messages
feel reasonable, this commit changes the code to not consider them
as a special case and topic editing restrictions apply to them as
well now like all other messages.

We still highlight the topic edit icon in recipient bar without
hovering for "no topic" messages, but it is only shown when user
has permission to edit topics.
2023-04-11 11:07:23 -07:00
Sahil Batra 66693f2101 user_groups: Add allow_nobody_group to access_user_group_for_setting.
This commit adds allow_nobody_group parameter to
access_user_group_for_setting with a default value of True.
2023-04-11 09:02:09 -07:00
Sahil Batra 5237e9008f types: Add allow_nobody_group field to GroupPermissionSetting. 2023-04-11 09:02:09 -07:00
Alya Abbott aa65813294 email: Rewrite followup_day2 email.
Keep the focus of the email the same (topics), but improve how
the information is presented.
2023-04-10 17:44:18 -07:00
Lauryn Menard 2654af3675 welcome-emails: Separate context for followup day1 and day2.
Separates the context dictionary that is used for `send_email` for
the `followup_day1` and `followup_day2` emails.

Prep commit for updates to `followup_day2` email.
2023-04-10 17:44:18 -07:00
Alex Vandiver 3efc0c9af3 workers: Rewrite missedmessage_emails with a worker thread.
The previous implementation leaked database connections, as a new
thread (and thus a new thread-local database connection) was made for
each timer execution.  While these connections were relatively
lightweight in Python, they also incur memory overhead in the
PostgreSQL server itself.  The logic for managing the timer was also
unclear, and the unavoidable deadlock in the stopping logic was rather
unfortunate.

Rewrite with one explicit worker thread which handles the delayed
message sending.  The RabbitMQ consumer creates the database rows, and
notifies the worker to start its 5s timeout.  Because it is controlled
by a condition variable, it does not hold the lock while waiting, and
can be notified to exit.
2023-04-10 17:38:08 -07:00
AcKindle3 ff48f3c489 templates: Use `root_domain_url` Django templates.
Per the issue #25045, this commit changes some occurences of `uri`
appeared in variable `root_domain_uri`. Files affected are some
html files that used this variables and a backend file
`context_processors.py` that set it as a key.
2023-04-10 10:26:48 -07:00
Lauryn Menard f391bfeec6 emails: Add new onboarding email with guide for organization type.
Adds a new welcome email, `onboarding_zulip_guide`, to be sent four
days after a new user registers with a Zulip organization if the
organization has specified a particular organization type that has
a guide in the corporate `/for/.../` pages. If there is no guide,
then no email is scheduled or sent.

The current `for/communities/` page is not very useful for users
who are not organization administrators, so these onboarding guide
emails are further restricted for those organization types to
only go to new users who are invited/registered as admins for the
organzation.

Adds two database queries for new user registrations: one to get
the organization's type and one to create the scheduled email.

Adds two email logs because the email is sent both to a new user
who registers with an existing organization and to the organization
owner when they register a new organization.

Co-authored by: Alya Abbott <alya@zulip.com>
2023-04-10 08:38:09 -07:00
Lauryn Menard ab00648e3e email: Refactor calculation of delay for scheduled emails.
Refactors the logic for adjusting the delay for sending an email
to not land on a weekend so that it can be used to schedule any
number of onboarding emails we decide to send.

Consolidates duplicate testing into
`zerver/tests/test_email_notifications.py`. The initial test and
function were introduced in commit 610f2cbacf with the test
located in `zerver/tests/test_signup.py`.

Prep commit for adding new welcome / follow up email.
2023-04-10 08:38:09 -07:00
AcKindle3 b0ef8f0822 test: Replace occurences of `uri` with `url`.
In all the tests files, replaced all occurences of `uri` with `url`
appeared in comments, local variablles, function names and their callers.
2023-04-08 16:27:55 -07:00
Joelute 946b4e73ca integration: Remove branch names from Gitea edited PR event.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
2023-04-07 15:17:23 -07:00
Satyam Bansal 338436dfbd
integrations: Add Rundeck webhook integration.
Co-authored-by: sbansal1999 <sbansal1999@gmail.com>
2023-04-07 10:13:21 -07:00
Prakhar Pratyush d96048b0af test_classes: Rename and refactor 'tornado_redirected_to_list'.
This commit renames the 'tornado_redirected_to_list' context
manager to 'capture_send_event_calls' to improve readability.

It also refactors the function to yield a list of events
instead of passing in a list data structure as a parameter
and appending events to it.
2023-04-07 09:45:26 -07:00
Alex Vandiver e3f6e810dc export-search: Support writing out attachments as well. 2023-04-07 09:13:48 -07:00
Alex Vandiver e408f069fe uploads: Add a method to copy attachment contents out. 2023-04-07 09:13:48 -07:00
Alex Vandiver 885334a3ad export-search: Support CSV output, trim down output fields. 2023-04-07 09:13:48 -07:00
Alex Vandiver bde978d5b1 export-search: Move topic into recipient field. 2023-04-07 09:13:48 -07:00
Alex Vandiver 26bc112adc export-search: Add a --recipient filter. 2023-04-07 09:13:48 -07:00
Alex Vandiver e44ba1bef1 export-search: Write output size before beginning export. 2023-04-07 09:13:48 -07:00
Alex Vandiver 21db267202 export-search: Non-time-limited is fine with a sender. 2023-04-07 09:13:48 -07:00
Alex Vandiver ae20af84a9 export-search: Realm is required. 2023-04-07 09:13:48 -07:00
Anders Kaseorg d2af06f4df middleware: Remove ZulipCommonMiddleware patch.
My fix for the relevant performance bug was upstreamed in Django 4.2.

https://code.djangoproject.com/ticket/33700

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-07 09:13:20 -07:00
Akarsh Jain f122516e7d notifications: Add link to new location of single moved messages.
Improve the Notification Bot by adding a hyperlink to the new location
of a moved single message. The link will make it easier for users to
find the message in its new context.

Fixes #24604.
2023-04-06 19:04:40 -07:00
Ujjawal Modi d0dbdfa52d user_groups: Send a message on changing user-groups subscribers.
After this commit a notification message is sent to users if they are
added to user_groups by someone else or they are removed from user_groups
by someone else.

Fixes #23642.
2023-04-06 19:03:26 -07:00
Ujjawal Modi e163e3ced5 user_groups: Refactor the view function used for adding members.
This commit changes the variable names inside the view function
used for adding members to user_groups. This was done for better
readability of code.
2023-04-06 19:03:26 -07:00
Anders Kaseorg 52266cbd41 user_groups: Work around django-cte bug with Django 4.2.
https://github.com/dimagi/django-cte/issues/66

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-06 17:58:44 -07:00
Anders Kaseorg f1925487e8 db: Force use of TimeTrackingCursor to work around Django 4.2 bug.
Effectively revert commit b4cf9ad777 to
work around https://code.djangoproject.com/ticket/34466.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-06 17:58:44 -07:00
sbansal1999 f47a1c67a6 webhooks: Add body to PR Review Event message for GitHub Integration.
This commit passes the body of the PR Review as the message to
the helper function that generates the message to be sent by the
GitHub Integration.

Previously when a PR Review was done the message sent would just
include the link of the review but the message didn't include the
body the review. After this commit, the message also includes the
body of the review.

Fixes #24676
2023-04-05 15:48:26 -07:00
Daniil Fadeev 2f203f4de1 emails: Inline CSS in emails in build_email.
Previously, we had an architecture where CSS inlining for emails was
done at provision time in inline_email_css.py. This was necessary
because the library we were using for this, Premailer, was extremely
slow, and doing the inlining for every outgoing email would have been
prohibitively expensive.

Now that we've migrated to a more modern library that inlines the
small amount of CSS we have into emails nearly instantly, we are able
to remove the complex architecture built to work around Premailer
being slow and just do the CSS inlining as the final step in sending
each individual email.

This has several significant benefits:

* Removes a fiddly provisioning step that made the edit/refresh cycle
  for modifying email templates confusing; there's no longer a CSS
  inlining step that, if you forget to do it, results in your testing a
  stale variant of the email templates.
* Fixes internationalization problems related to translators working
  with pre-CSS-inlined emails, and then Django trying to apply the
  translators to the post-CSS-inlined version.
* Makes the send_custom_email pipeline simpler and easier to improve.

Signed-off-by: Daniil Fadeev <fadeevd@zulip.com>
2023-04-05 12:22:29 -07:00
Lauryn Menard e95b784f6e backend-auth: Add user ID to fetch api key responses.
Adds the user ID to the return values for the `/fetch_api_key` and
`/dev_fetch_api_key` endpoints. This saves clients like mobile a
round trip to the server to get the user's unique ID as it is now
returned as part of the log in flow.

Fixes #24980.
2023-04-04 10:54:49 -07:00
Anders Kaseorg a881918a05 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-03 22:39:21 -07:00
Anders Kaseorg 2ce8357009 codespell: Fix newly found typos.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-03 22:39:21 -07:00
Prakhar Pratyush ab7cd3c40a user_topics: Update test to verify end-to-end behaviour.
This commit updates 'test_user_ids_unmuting_topic' to make
an api_post call to '/api/v1/user_topics' instead of
calling the internal function 'do_set_user_topic_visibility_policy'
to verify the logic.
2023-04-03 22:31:49 -07:00
Prakhar Pratyush f012d079c3 user_topics: Add a new endpoint to update visibility_policy.
This commit adds a new endpoint, 'POST /user_topics' which
is used to update the personal preferences for a topic.

Currently, it is used to update the visibility policy of
a user-topic row.
2023-04-03 22:31:49 -07:00
Prakhar Pratyush 0b2fe5b163 user_topics: Rename the 'zerver/lib/user_topics' functions.
This is a prep commit that renames lib functions
so that they can be used while implementing view
for the new endpoint 'POST /user_topics'.

We use a more generic name when removing the visibility_policy of
a topic, i.e., 'access_stream_to_remove_visibility_policy_by_id/name'
instead of 'access_stream_for_unmute_topic_by_id/name' which focused
on removing MUTE from a topic.
2023-04-03 22:31:49 -07:00
Prakhar Pratyush e33eb45507 user_topics: Update tests to use the 'time_machine' module.
This commit updates the tests to use the 'time_machine'
module for mocking datetime.
2023-04-03 22:31:49 -07:00
Prakhar Pratyush 5c86d3ca9e user_topics: Update tests to verify end-to-end behaviour.
This commit updates the existing tests so that they call
API endpoints instead of internal functions to verify logic.
2023-04-03 22:31:49 -07:00
Alex Vandiver 113a8c4782 export: Make --deactivate-realm exports be imported as active. 2023-04-03 16:08:43 -07:00
Alex Vandiver 4b0b8abc6c import_realm: Keep the realm deactivated until import is complete. 2023-04-03 16:08:43 -07:00
Daniil Fadeev 5c6f842b28 emails: Replace Premailer with CSS-inline.
Primary goal of library replacement is improving execution speed.
This commit should not affect the functionality of the system
or make any changes to it.
2023-04-03 15:15:05 -07:00
Joelute 78a9d716ec integration: Remove branch names to Gogs edited PR event.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
2023-04-02 23:06:15 -07:00
Josh Klar ef0c281070 integrations: Add documentation for Mastodon via public RSS feeds.
This uses the Mastodon logo SVG as pulled from
https://blog.joinmastodon.org/logo.svg.
2023-03-30 09:36:16 -07:00
Mateusz Mandera e3f754fb9b help center: Document SAML configuration steps with Auth0.
Auth0 is another SAML IdP that we haven't detailed the instructions for
yet.
2023-03-30 09:11:55 -07:00
David Rosa 2b9374875d templates: Update text on /accounts/find.
- Being more specific about what the user will get.
- Putting less emphasis on entering multiple emails, since most
  people probably just have one email they need to check.
- Using more intuitive wording and hint that deactivated or
  deleted accounts won't be included.

Fixes: #24890.
2023-03-30 09:06:14 -07:00
Joelute 75611c6cee integration: Modify branch names for Bitbucket 3 events.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
2023-03-29 18:13:36 -07:00
Joelute 1ddfbc94c3 integration: Support reviewer in `get_pull_request_event_message`.
This is a prep commit to help make the changes to make changes to pull
event message easier. Our Bitbucket has been using a custom template to
render the reviewers. This means that values are fixed to how the templates
like it. These changes will allow `get_pull_request_event_message` to
support reviewer and allow for a easier and flexible adjustment to these
messages if needed.
2023-03-29 18:13:36 -07:00
Joelute 4dc85dddc1 integration: Move assignee message to the end for git integrations.
Previously, the assignee message would stick around in the middle of the
event message. This doesn't look as good as if we put it to the end of
the event message. These changes does just that and move the assignee
messages towards the end of the event message to make it look better
and cleaner for the readers.
2023-03-29 18:13:36 -07:00
Mateusz Mandera f3db618cb5 actions: Implement do_delete_user_preserving_messages function.
With an extensive docstring by tabbott documenting its limitations.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2023-03-29 12:51:21 -07:00
Prakhar Pratyush d8a5ca5ddc user_topics: Update test_edit_muted_topic to verify events sent.
Previously, there was a stale code that didn't verify
if 'muted_topics' and 'user_topic' events are sent correctly.

This commit updates the test to verify if the expected
users are notified via 'muted_topics' and 'user_topic'
events.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 117264fb37 user_topics: Update move-topic codepath to utilise bulk database operations.
This commit updates the move-topic codepath to perform
bulk database operations on the UserTopic record using
user_profiles for each visibility_policy instead of
previously looping over each user_profile one by one.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 60762a750b user_topics: Update 'set visibility policy' codepath for bulk db operations.
This commit refactors 'set_user_topic_visibility_policy_in_database'
to perform bulk database operations and the related changes.

There is an increase in database query count because requests
to delete user_topic rows now take two queries instead of one.
This is required for logging the info for a request to delete
a non-existent user_topic row while performing bulk operations
at the same time.

The overall query count will be lower while performing
bulk operations (multiple user_profiles instead of one).
2023-03-29 09:50:06 -07:00
Prakhar Pratyush a890aaf34d user_topics: Update UserTopic records regardless of the visibility_policy.
This commit updates the 'do_update_message' codepath to
update the UserTopic records regardless of visibility policy
during the "move-topic" operation.

This is required before offering new visibility policies
in the UI.

Previously, UserTopic records were moved or deleted only
for objects with a MUTED visibility policy.

Fixes: #24574
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 0377085f15 user_topics: Update a comment regarding the database query count.
This commit updates a stale comment regarding query count
to its correct value.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush e1ec814507 user_topics: Add a test helper to improve readability.
This commit adds a test helper function that helps
to remove line wrappings and improve readability.
2023-03-29 09:50:06 -07:00
Prakhar Pratyush 0ec9ff17ce user_topics: Update lib functions to work with any visibility_policy.
This is a prep commit that renames 'set_topic_mutes' and
'topic_is_muted' to 'set_topic_visibility_policy' and
'topic_has_visibility_policy' respectively, and refactors
them to work with any visibility_policy, not only MUTED.
2023-03-29 09:50:06 -07:00
Joelute a2a9e53423 integration: Refactor `get_issue_event_message` to use kwarg.
Previously, some call sites for the function provided optional
arguments as positional arguments. These changes will allow the
arguments to be passed as keyword arguments to the function and
fix up the call sites of the function to pass keyword arguments
instead.
2023-03-28 15:51:43 -07:00
Joelute a2c578e84d integration: Refactor `get_pull_request_event_message` to use kwarg.
Previously, some call sites for the function provided optional
arguments as positional arguments. These changes will allow the
arguments to be passed as keyword arguments to the function and
fix up the call sites of the function to pass keyword arguments
instead.
2023-03-28 15:51:43 -07:00
Lauryn Menard 7b225245c0 tests: Update ZulipTestCase.tearDown to remove local uploads.
Previously, tests that exercised code paths that added local
uploads did not always clean up `settings.LOCAL_UPLOADS_DIR`
after the test was complete.

Updates the `ZulipTestCase` class to remove any local uploads
in the unique `settings.LOCAL_UPLOADS_DIR` in `tearDown` for
all tests.
2023-03-28 14:38:06 -07:00
Sahil Batra a6fd41e012 migrations: Fix comment in 0403.
This commit fixes the file name in comment in
0403_create_role_based_groups_for_internal_realms.py.
2023-03-28 14:26:22 -07:00
Sahil Batra bed2bf64c4 user_groups: Add "Nobody" system user group.
This commit adds code to create a "Nobody" system user group
to realms which will be used in settings to represent "Nobody"
option.

We also add a migration to add this group to existing realms.
2023-03-28 14:26:22 -07:00
Prakhar Pratyush e45623fccc python: Update tuple handling pattern; returned by a delete() query.
This commit updates the pattern for dealing with tuples
returned by the delete() query.

The '(num_deleted, ignored) = ModelName.objects.filter().delete()'
pattern is preferred due to better readability.

We avoid the pattern '(num_deleted, _)' because Django uses _
for translation, which may lead to future bugs.
2023-03-27 16:18:23 -07:00
Sahil Batra f684d36710 test_classes: Add submit_realm_creation_form helper.
This commit adds a new helper submit_realm_creation_form,
similar to existing submit_reg_form_for_user, to avoid
duplicate code for creating realms in tests.
2023-03-27 15:44:42 -07:00
Sahil Batra 4d3c5fdc91 forms: Use RealmDetailsForm as subclass for RegistrationForm.
We now use RealmDetailsForm as subclass for RegistrationForm
such that we can avoid duplication of the checks for
realm_subdomain field.
2023-03-27 15:44:42 -07:00
Sahil Batra 96979e73c6 registration: Use get_realm_create_form_context for register.html.
This commit adds the fields related to realm creation form using
get_realm_create_form_context in the context passed to register.html
template to avoid duplication.
2023-03-27 15:44:42 -07:00
Sahil Batra 00ffa3e870 registration: Remove redundant code.
Since we have updated the registration code to use
PreregistrationRealm objects for realm creation in
previous commits, some of the code has become
redundant and this commit removes it.

We remove the following code -
- The modification to PreregistrationUser objects in
process_new_human_user can now be done unconditionally
because prereg_user is passed only during user creation
and not realm creation. And we anyway do not expect
any PreregistrationUser objects inside the realm
during the creation.
- There is no need of "realm_creation" parameter in
create_preregistration_user function, since we now
use create_preregistration_realm during realm creation.

Fixes part of #24307.
2023-03-27 15:44:42 -07:00
Sahil Batra cf8d8db132 register: Update the user-registration form for realm creation.
In previous commits, we updated the realm creation flow to show
the realm name, type and subdomain fields in the first form
when asking for the email of the user. This commit updates the
user registration form to show the already filled realm details
as non-editable text and there is also a button to edit the
realm details before registration.

We also update the sub-heading for user registration form as
mentioned in the issue.

Fixes part of #24307.
2023-03-27 15:44:42 -07:00
Sahil Batra c225de789e development: Use PreregistrationRealm object for creating realms. 2023-03-27 15:44:42 -07:00
Sahil Batra 791c3030eb backends: Update ZulipLDAPAuthBackend to use PreregistrationRealm.
This commit updates ZulipLDAPAuthBackend to use the recently added
PreregistrationRealm when creating realms with LDAP used as
authentication method.
2023-03-27 15:44:42 -07:00
Sahil Batra e60b6b68ea registration: Modify PreregistrationRealm objects after registration. 2023-03-27 15:44:42 -07:00
Sahil Batra 765e65f954 registration: Use PreregistrationRealm object for realm creation.
We now use PreregistrationRealm objects in registration_helper
function when creating new realms instead of PreregistrationUser
objects.

Fixes part of #24307.
2023-03-27 15:44:42 -07:00
Sahil Batra 54771cfe94 registration: Create PreregistrationRealm object when creating realm.
This commit adds code to create PreregistrationRealm object when
creating realm and set it to the content_object field of
Confirmation object.
2023-03-27 15:44:42 -07:00
Sahil Batra 80b00933b1 templates: Add realm information fields in create_realm.html.
We now show inputs for realm details like name, type and URL
in the create_realm.html template opened for "/new" url and
these information will be stored in PreregistrationRealm
objects in further commits.

We add a new class RealmDetailsForm in forms.py for this
such that it is used as a base class for RealmCreationForm
and we define RealmDetailsForm such that we can use it as
a subclass for RegistrationForm as well to avoid duplication.
2023-03-27 15:44:42 -07:00
Sahil Batra 7f1bf9d6ab models: Add PreregistrationRealm class.
This commit adds PreregistrationRealm class which will be
similar to PreregistrationUser and will store initial
information of the realm before its creation as we are
changing the organization creation flow as per #24307.

Fixes part of #24307.
2023-03-27 15:44:42 -07:00
Sahil Batra 8c6bf72c1f registration: Rename prereg_user to prereg_object.
This commit renames prereg_user variable in
check_prereg_key and get_prereg_key_and_redirect
functions in zerver/views/registration.py to
prereg_object as in further commits the
preregistration object could also be
PreregistrationRealm object as part of changes
for #24307.
2023-03-27 15:44:42 -07:00
Joelute 8747aa6fa1 integration: Modify branch names for Bitbucket 2 events.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
2023-03-27 10:09:25 -07:00
Joelute 505c217db5 bots: Change logic for notification bot after moved message.
Previously, when a user moves a message to another topic, the Notification
bot will post a message saying "This topic was moved here from..." This is
confusing when the topic already contains messages. The changes aims to make
the messages more clear by changing the logic for the Notification bot. When
there is already messages in the topic, the bot will post "A message was
moved here from..." or "N messages were moved here from...". The bot will
post "This topic was moved here from (somewhere) by (someone)." when the
topic is empty.

Fixes #23267.
2023-03-27 10:00:34 -07:00
Joelute c8bce6f8d3 bots: `maybe_send_resolve_topic_notiffications` return message_id or None 2023-03-27 10:00:34 -07:00
Zixuan James Li cf9b95b95a user_groups: rename create_user_group to create_user_group_in_database.
To avoid people calling "create_user_group" instead of
"check_add_user_group", we rename it to make its purpose clearer.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-03-27 09:05:00 -07:00
Zixuan James Li e331c356e4 user_groups: Use check_add_user_group instead in test cases.
"check_add_user_group" is a safer helper function than
"create_user_group" to use when creating user_groups. It does
error handling and notify the client with the appropriate event.

Note that the populate_db command still uses "create_user_group"
because we do not need to enqueue events at that point.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-03-27 09:05:00 -07:00
Zixuan James Li 0f5d6432a4 user_groups: Move create_user_group to zerver.actions.user_groups.
Since this function creates a new user group into the database,
it is more appropriate to have it not as a generic "lib" function
but as an "action".

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-03-27 09:05:00 -07:00
Alex Vandiver 93bfd3fb5f sentry: Provide a server-side tunnel.
Some well-intentioned adblockers also block Sentry client-side error
reporting.  Provide an endpoint on the Zulip server which forwards to
the Sentry server, so that these requests are not blocked.
2023-03-24 15:16:38 -07:00
Alex Vandiver 164d58bec9 web: Save a 301 redirect from /help to /help/. 2023-03-24 14:51:01 -07:00
Alex Vandiver ab7ff6b582 web: Save a 301 redirect from /billing to /billing/. 2023-03-24 14:51:01 -07:00
Alex Vandiver b5731c55b7 web: Save two 301 redirects from /privacy to /privacy/ to /policies/privacy. 2023-03-24 14:51:01 -07:00
Alex Vandiver 822683249d web: Save a 301 redirect from /terms/ to /policies/terms. 2023-03-24 14:51:01 -07:00
Alex Vandiver ce847fd09c web: Save two 301 redirects from /terms to /terms/ to /policies/terms. 2023-03-24 14:51:01 -07:00
Alex Vandiver d9daeb13e0 web: Save a needless 301 redirect from /apps to /apps/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 61adc9cfcc web: Save a needless 301 redirect from /team to /team/. 2023-03-24 14:51:01 -07:00
Alex Vandiver c686c5ed0f web: Save a needless 301 redirect from /plans to /plans/. 2023-03-24 14:51:01 -07:00
Alex Vandiver eb81e62eaa web: Save a needless 301 redirect from /hello to /hello/. 2023-03-24 14:51:01 -07:00
Alex Vandiver a1570ff3dd web: Save a needless 301 redirect from /integrations to /integrations/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 5967dda35d web: Save a needless 301 redirect from /upgrade to /upgrade/. 2023-03-24 14:51:01 -07:00
Anders Kaseorg b719227b04 ruff: Fix C416 Unnecessary `dict` comprehension.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-24 11:57:07 -07:00
Anders Kaseorg 869ea542fa ruff: Fix PLR1711 Useless `return` statement at end of function.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-24 11:57:07 -07:00
Anders Kaseorg 870238206c create-test-api-docs: Remove.
This tool plays fast and loose with HTML generation, and nobody seems
to use it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-24 11:55:12 -07:00
Lauryn Menard 52801f3cd2 narrow: Support underscore version of "pm-with" and "group-pm-with".
Prior to commit a9b3a9c, the server implementation for documented
search operators with dashes, also implicitly supported clients
sending those same operators with underscores. This has been the
case sense the server side support for narrow filtering was
introduced in commit 3af2bf345a.

Updates the stricter version of mapping operator strings to `by*`
functions, to also include the underscore version of any operators
that have dashes. Adds a note that these undocumented versions are
tied to the support for the documented versions.
2023-03-24 11:54:50 -07:00
Alex Vandiver bf532de8bb queue: Allow enabling TLS for the RabbitMQ connection.
This allows using cloud-based RabbitMQ services like AmazonMQ.

Fixes: #24699.
2023-03-23 16:02:10 -07:00
Joelute 0a3fcbb649 integration: Modify branch names for Gitlab MR events.
This is a follow up to #24673, we want to modify every webhook events to
follow the same pattern and consistency where branch name should only
show on opened and merged events.
2023-03-23 12:17:28 -07:00
Sahil Batra ce562c4820 user_settings: Create RealmAuditLog entries for all user settings.
We previously created RealmAuditLog entries for user notification
settings only. This commit changes the code to create entries for
all user settings. We cannot backfill the entries since we don't
have the data to do that.
2023-03-23 10:06:47 -07:00
Lauryn Menard ffcdc13819 create-realm: Update notification message sent to admin realm.
When a new realm is created, a notification message is sent to
the realm configured as the settings.SYSTEM_BOT_REALM if there
is a "signups" stream that exists in that realm. This is used
for Zulip Cloud, but is an undocumented feature.

The topic of the message has been the subdomain of the new realm,
and the message content has been "Signups enabled" translated
into the default language of the new realm.

In order to make these messages more explicitly for Zulip Cloud,
the settings.CORPORATE_ENABLED is checked before sending these
messages.

To make these messages more useful, the topic for these
notifications is changed to be "new organizations". The content
of these messages is updated to have the new realm name (with a
link to the admin realm's activity support page for the realm),
subdomain (with a link to the realm), and organization type.
2023-03-22 17:48:53 -07:00
Anders Kaseorg afa218fa2a semgrep: Detect some unsafe uses of markupsafe.Markup.
Use the built-in HTML escaping of Markup("…{var}…").format(), in order
to allow Semgrep to detect mistakes like Markup("…{var}…".format())
and Markup(f"…{var}…").

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-22 11:23:27 -07:00
Alya Abbott e660ffbe07 portico: Add End Point Dev case study page. 2023-03-21 16:20:30 -07:00
Anders Kaseorg 6db1e75fb9 test_classes: Don’t spew BAN_CONSOLE_OUTPUT errors over failed tests.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-21 12:27:32 -07:00
Alex Vandiver 8ad4ecb3a2 management: Add a command to send initial welcome bot messages. 2023-03-21 11:40:37 -07:00
Lauryn Menard a9b3a9c673 narrow: Use dict to map operator to by_* method in NarrowBuilder.
Updates the logic for identifying the method to use to extend the
query for the given term from a narrow to use a dictionary that
maps the operator string to the by_* method in the NarrowBuilder
class.

Previously, the by_* method was determined by building a string
based on the operator string and replacing dashes with underscores.
2023-03-20 15:37:09 -07:00
Joelute ff6ff1e014 integration: Add branch names to Azure Devops merged PR event.
This is a follow up to #24673, we want to modify every webhook
events to follow the same pattern and consistency where branch name
should only show on opened and merged events.
2023-03-20 12:33:47 -07:00
Alex Vandiver 33b61d9dc4 invite: Be more fair about the no-messages-sent warning flag.
The RealmCount statistics will be empty if the realm was created since
the last daily aggregation.  In cases where the daily stats have no
rows, it is likely fast enough to do the real count in the messages
table.  This stops unduly penalizing folks who have actually sent
messages, and are just inviting people within the first day.
2023-03-20 11:35:19 -07:00
Alex Vandiver e6eaa9a7f6 webhooks: Handle explicit null context_line in Sentry. 2023-03-20 11:34:17 -07:00
Alex Vandiver 311a76ed1c Move QOS configuration into connection, not queue verification.
Prior to aa032bf62c, QOS prefetch was set on every `publish` and
before every `start_json_consumer` -- which had a large and
unnecessary effect on publishing rates, which don't care about the
prefetch QOS settings at all, much less re-setting them before every
publish.

Unfortunately, that change had the effect of causing prefetch settings
to almost never be respected -- since the configuration happened in
`ensure_queue`s re-check that the connection was still live.  The
initial connection is established in `__init__` via `_connect`, and
the consumer only calls `ensure_queue` once, before setting up the
consumer.

Having no prefetch value set causes an unbounded prefetch; this
manifests itself as the server attempting to shove every event down to
the worker as soon as it starts consuming; if the client cannot keep
up, the server closes the connection.  The worker observes the
connection has been shut down, and restarts.  While this does make
forward progress, it causes large queues to make progress more slowly,
as they suffer from sporadic restarts.

Shift the QOS configuration to when the connection is set up, which is
a more sensible place for it in general -- and ensures that it is set
on consumers and producers alike, but only once per connection
establishment.
2023-03-20 11:28:29 -07:00
Alex Vandiver fa6daee4e1 markdown: Fix use of pure_markdown for non-pure markdown rendering.
`render_markdown_path` renders Markdown, and also (since baff121115)
runs Jinja2 on the resulting HTML.

The `pure_markdown` flag was added in 0a99fa2fd6, and did two
things: retried the path directly in the filesystem if it wasn't found
by the Jinja2 resolver, and also skipped the subsequent Jinja2
templating step (regardless of where the content was found).  In this
context, the name `pure_markdown` made some sense.  The only two
callsites were the TOS and privacy policy renders, which might have
had user-supplied arbitrary paths, and we wished to handle absolute
paths in addition to ones inside `templates/`.

Unfortunately, the follow-up of 01bd55bbcb did not refactor the
logic -- it changed it, by making `pure_markdown` only do the former
of the two behaviors.  Passing `pure_markdown=True` after that commit
still caused it to always run Jinja2, but allowed it to look elsewhere
in the filesystem.

This set the stage for calls, such as the one introduced in
dedea23745, which passed both a context for Jinja2, as well as
`pure_markdown=True` implying that Jinja2 was not to be used.

Split the two previous behaviors of the `pure_markdown` flag, and use
pre-existing data to control them, rather than an explicit flag.  For
handling policy information which is stored at an absolute path
outside of the template root, we switch to using the template search
path if and only if the path is relative.  This also closes the
potential inconsistency based on CWD when `pure_markdown=True` was
passed and the path was relative, not absolute.

Decide whether to run Jinja2 based on if a context is passed in at
all.  This restores the behavior in the initial 0a99fa2fd6 where a
call to `rendar_markdown_path` could be made to just render markdown,
and not some other unmentioned and unrelated templating language as
well.
2023-03-17 08:46:25 -07:00
Zixuan James Li 24f24d236d cache: Use QuerySetAny for isinstance check.
Previously, `QuerySet` does not support isinstance check since it is
defined to be generic in django-stubs. In a recent update, such check is
possible by using `QuerySetAny`, a non-generic alias of `QuerySet`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-03-17 08:38:20 -07:00
Alex Vandiver 330141f55d invites: Switch new LIMITED-plan heuristic to enforcing. 2023-03-16 11:41:49 -07:00
Alex Vandiver 50a2a54393 test_invite: Rework and expand invitation limit tests.
This adds tests for more corner cases, in exchange for dropping the
query count tests, which were of dubious utility.  It also adds the
time-machine library to mock the current time to test that the limits
do expire.
2023-03-16 11:41:49 -07:00
Alex Vandiver 6971c6d62d test_invite: Just import django.core.mail and use it. 2023-03-16 11:41:49 -07:00
Alex Vandiver 9e972bc82c test_invite: Clarify by filters that no *new* ScheduledEmails are made. 2023-03-16 11:41:49 -07:00
Alex Vandiver 83fdc59304 test_invite: Remove a totally uninteresting assert. 2023-03-16 11:41:49 -07:00
Alex Vandiver 566806fb41 test_invite: Fix a docstring. 2023-03-16 11:41:49 -07:00
Alex Vandiver 695730b609 test_invite: Explicitly test that no emails are actually sent. 2023-03-16 11:41:49 -07:00
Alex Vandiver 34c8cd1b74 tests: Split out test_invite from test_signup.
There is no good reason for this single test file to be more than 6k
lines.
2023-03-16 11:41:49 -07:00
Joelute 869cb6dc34 integration: Modify branch names on pull request events.
Previously when Github bot receives an update pull request event,it
will produce the following message:

user updated PR #1 Start writing unit tests from test to main

"from test to main" is improper and causes unnecessary confusion.
These changes will update the logic to remove the phrase from
update events. These changes will also include the org: prefix to
the branch names to keep it consistent with Github and further
reduce confusions on branch names.

Fixes #24536.
2023-03-16 11:31:48 -07:00
Anders Kaseorg 8cd78d356f templates: Allow the same line-wrapping in {{t}} as in {{#tr}}.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-16 11:26:45 -07:00
Prakhar Pratyush 042bbf2936 UserTopic: Set visibility_policy or delete UserTopic row without error.
This commit updates 'set_user_topic_visibility_policy_in_database'
to not raise an error when deleting a UserTopic row and the user
doesn't have a visibility_policy for the topic yet, or when setting
the visibility_policy to its current value.

Also, it includes the changes to not send unnecessary events
in such cases.
2023-03-16 10:55:54 -07:00
David Rosa f783e8b6ca help: Add a dedicated page on how to add emphasis to text.
- Adds "Via Markdown" instructions block.
- Adds "Via compose box buttons" instructions block.

Fixes part of #24209.
2023-03-16 10:51:57 -07:00
Zixuan James Li 234771d5db integrations: Add a link to GitHub webhooks documentation.
The list of supported events for filtering itself does not document what
each of the events does. Adding a link to GitHub's documentation would
be a pointer to get people started. But ideally we need to establish a
better system to document the events in general.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-03-15 15:37:42 -07:00
Alya Abbott 584124bb45 help: Improve "Deactivate your account" help page. 2023-03-15 15:32:47 -07:00
Abhijeet Prasad Bodas ba0dd70904 user_topics: Introduce visibility policy enum subclass in models.py.
This commit gives more readable code than using the `VISIBILITY_POLICY`
prefix.
This commit does not alter the database schema.
2023-03-14 17:50:45 -07:00
Prakhar Pratyush ae72777c77 user_settings: Automate 'Include realm name in message email subject'.
Currently, there is a checkbox setting for whether to
"Include realm name in subject of message notification emails".

This commit replaces the checkbox setting with a dropdown
having values: Automatic [default], Always, Never.

The Automatic option includes the realm name if, and only if,
there are multiple Zulip realms associated with the user's email.

Tests are added and(or) modified.

Fixes: #19905.
2023-03-14 17:48:57 -07:00
Alya Abbott 005ca2b033 portico: Add landing page about trying Zulip by visiting chat.zulip.org. 2023-03-14 13:21:09 -07:00
Lauryn Menard 213d0f4990 create-user: Remove notifications sent to admin realm.
Removes the notification message that was sent if a stream named
"signups" exists in the `settings.SYSTEM_BOT_REALM`. This was a
undocumented feature that would send a notification message when
a new user registered with a Zulip organization that was hosted
by an admin realm like Zulip Cloud.

This removes two database queries when a new user is created: one
to get the system bot realm and the other to get the notification
bot in said realm.

Note that there are still notification messages sent when a new
organization is registered with the admin realm if the "signups"
stream exists.
2023-03-13 12:28:26 -07:00
Aman Agrawal dbe930394f footer: Integrate newly designed footer.
This footer was designed and mostly written by @terpimost.

This adds a new design of the footer for both corporate and not
corporate enabled pages.
2023-03-10 17:34:15 -08:00
Prakhar Pratyush 1b7cbd9ef0 UserTopic: Refactor 'do_set_user_topic_visibility_policy'.
This commit refactors 'do_set_user_topic_visibility_policy'
to remove the if/else block and just have a single call to
'set_user_topic_visibility_policy_in_database'.

The branching out behaviour based on the user_topic
visibility_policy is reduced to one place, i.e.,
'set_user_topic_visibility_policy_in_database'.
2023-03-10 14:39:14 -08:00
palashb01 dee63680df docs: Update time format and Emoticon help page.
Updated the title and description in the 'enable-emoticon-translation'
file and renamed the file accordingly. Added a new bullet point for
'time format' in the 'configure-new-user-settings.md' file and updated
the sidebar index by replacing the title 'Use 24-hour time' with
'Change the time format'.
2023-03-09 16:12:45 -08:00
Anders Kaseorg 3bfbfb014a zilencer: Switch a log message back from %r to %s.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Anders Kaseorg 2d9b2a2a05 models: Remove type prefixes from __str__ values.
The Django convention is for __repr__ to include the type and __str__
to omit it.  In fact its default __repr__ implementation for models
automatically adds a type prefix to __str__, which has resulted in the
type being duplicated:

    >>> UserProfile.objects.first()
    <UserProfile: <UserProfile: emailgateway@zulip.com <Realm: zulipinternal 1>>>

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Aman Agrawal c928c87645 google_analytics: Track realm registration separately from user signup.
While the function which processes the realm registration and
signup remains the same, we use different urls and functions to
call the process so that we can separately track them. This will
help us know the conversion rate of realm registration after
receiving the confirmation link.
2023-03-08 11:34:36 -08:00
Sahil Batra 0488dfdd9e openapi: Add links to email_address_visibility help documentation. 2023-03-07 15:11:28 -08:00
Sahil Batra a48faec887 create_user: Refactor notify_created_user function.
This commit refactors the notify_created_user function to
call format_user_row twice with different parameters instead
of modifying the person object returned by format_user_row.

This change makes the code somewhat more easy to understand
than it was before.
2023-03-07 15:11:03 -08:00
Alex Vandiver a06a699c35 invites: The aggregate is null if there are no rows to sum. 2023-03-07 15:04:39 -08:00
Alex Vandiver 36da7783ce invites: _max_invites is currently never None.
dc1eeef30a made the column nullable, with the meaning for null of
"use the current `settings.INVITES_DEFAULT_REALM_DAILY_MAX`."
However, 8a95526ced switched to calling `do_change_plan_type` during
realm creation, which sets `realm.max_invites` based on the plan type,
thus ensuring that no new realms have their `_max_invites` set to
null.

Check `max_invites` instead of `_max_invites`.  This requires test
adjustments for the fact that `apply_invite_realm_heuristics` is now
run.
2023-03-07 15:04:39 -08:00
Alex Vandiver 75138102f4 invites: Extract new realm invitation heuristic. 2023-03-07 15:04:39 -08:00
Alex Vandiver a253c478f6 test_classes: Provide path to test-backend for ease of copy/paste. 2023-03-07 15:02:31 -08:00
Alex Vandiver 739d527b16 sentry: Enable reporting on portico and other unauth pages. 2023-03-07 15:01:27 -08:00
Lauryn Menard dfb9f74017 api_docs: Add page documenting HTTP headers in the api docs.
Adds a page to the general api documentation about HTTP headers,
so that information about the special response headers for rate
limits have a more logical location in the docs and so that other
HTTP header information can be shared, such as `User-Agent`
conventions.

Adjusts some text and linking on the rest-error-handling page and
overview page for the REST API for the addition of the HTTP headers
page.
2023-03-07 12:51:17 -08:00
Alex Vandiver 8f8a9f6f04 sentry: Add frontend event monitoring.
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors.  However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors.  Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.

Add a client-side Javascript sentry integration.  To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry.  Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
2023-03-07 10:51:45 -08:00
Alex Vandiver 19691b170b blueslip: Report the built client version, as well as server version. 2023-03-07 10:51:45 -08:00
Alex Vandiver 73631950a5 report: Use ZULIP_VERSION rather than shelling out to git every time. 2023-03-07 10:51:45 -08:00
Alex Vandiver 90b1e0b8b9 report: Provide user information for browser-side errors.
b4dd118aa1 changed how the `user_info_str` parsed information out of
the events it received -- but only changed the server errors, not the
browser errors, though both use the same codepath.  As a result, all
browser errors since then have been incorrectly marked as being for
anonymous users.

Build and pass in the expected `user` dict into the event.
2023-03-07 10:51:45 -08:00
Kartik Srivastava ce5d13f9b2 message_send: Handle notifications for UNMUTED topic in a muted stream.
This commit adds 'visibility_policy' as a
parameter to user_allows_notifications_in_StreamTopic
function.

This adds logic inside the user_allows_notifications_in_StreamTopic
function, to not return False when a stream is muted
but the topic is UNMUTED.

Adds a method `user_id_to_visibility_policy_dict`
to 'StreamTopicTarget' class to fetch
(user_id => visibility_policy) in single db query.

Co-authored-by: Kartik Srivastava <kaushiksri0908@gmail.com>
Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-03-06 19:15:45 -08:00
Kartik Srivastava e9cf2659cf user_topics: Refactor remove_topic_mute.
This commit replaces 'remove_topic_mute' with
'set_user_topic_visibility_policy_in_database' and
updates it to delete UserTopic row with any configured
visibility_policy and not just muting.
2023-03-06 19:15:45 -08:00
Kartik Srivastava f844cb6dad user_topics: Refactor add_topic_mute.
In order to support different types of topic visibility policies,
this renames 'add_topic_mute' to
'set_user_topic_visibility_policy_in_database'
and refactors it to accept a parameter 'visibility_policy'.

Create a corresponding UserTopic row for any visibility policy,
not just muting topics.

When a UserTopic row for (user_profile, stream, topic, recipient_id)
exists already, it updates the row with the new visibility_policy.

In the event of a duplicate request, raises a JsonableError.
i.e., new_visibility_policy == existing_visibility_policy.

There is an increase in the database query count in the message-edit
code path.

Reason:
Earlier, 'add_topic_mute' used 'bulk_create' which either
creates or raises IntegrityError -- 1 query.

Now, 'set_user_topic_visibility_policy' uses get_or_create
-- 2 queries in the case of creating new row.

We can't use the previous approach, because now we have to
handle the case of updating the visibility_policy too.
Also, using bulk_* for a single row is not the correct way.

Co-authored-by: Kartik Srivastava <kaushiksri0908@gmail.com>
Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-03-06 19:15:45 -08:00
Prakhar Pratyush e9580f8c5a MutedTopicsTests: Remove real-time usage from tests.
This commit refactors the existing pattern (real-time usage)
used to assert 'date_muted' in tests.

A fixed value is used at the start of the test to
assert 'date_muted', replacing the timedelta or real-time usage pattern.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 826ea4162e user_topics: Refactor 'do_unmute_topic'.
Replaces 'do_unmute_topic' with 'do_set_user_topic_visibility_policy'
and associated minor changes.

This change is made to align with the plan to use a single function
'do_set_user_topic_visibility_policy' to manage
user_topic - visibility_policy changes and corresponding event
generation.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 2df2ef9f0f user_topics: Refactor 'do_mute_topic'.
This commit is a step in the direction of having a common
function to handle visibility_policy changes and event
generation instead of separate functions for each
visibility policy.

In order to support different types of topic visibility policies,
this renames 'do_topic_mute' to 'do_set_user_topic_visibility_policy'
and refactors it to accept a parameter 'visibility_policy'.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 9ecc610681 mute_topic: Update tests to not use lib functions directly.
The "add_topic_mute" and "remove_topic_mute" library functions
shouldn't be called directly from tests.
They should instead call "do_mute_topic" and "do_unmute_topic"

The reason being:
Library functions are meant to be internal interfaces
for just changing the database, and shouldn't generally be
called elsewhere.
2023-03-06 19:15:45 -08:00
Author: Tim Abbott e161b36b21 user_topics: Delete stale do_update_message comment.
The issue this comment refers to was addressed in
bd04733.
2023-03-06 19:15:45 -08:00
Daniil Fadeev dcb72448d8 giphy: Fix "PG13" typo. 2023-03-06 10:52:52 -08:00
Daniil Fadeev d606f5ba42 i18n: Add new markup for lazy translation of giphy ratings.
Also, it adds phrases such as 'rated Y' to the ignored list.
Otherwise, the linter would require to write it as 'rated y'.
2023-03-06 10:52:52 -08:00
Lauryn Menard e9bfdd1bf2 response: Implement ignored parameters with MutableJsonResponse class.
Creates `MutableJsonResponse` as a subclass of Django's `HttpResponse`
that we can modify for ignored parameters in the response content.

Updates responses to include `ignored_parameters_unsupported` in
the response data through `has_request_variables`. Creates unit
test for this implementation in `test_decorators.py`.

The `method` parameter processed in `rest_dispatch` is not in the
`REQ` framework, so for any tests that pass that parameter, assert
for the ignored parameter with a comment.

Updates OpenAPI documentation for `ignored_parameters_unsupported`
being returned in the JSON success response for all endpoints.
Adds detailed documentation in the error handling article, and
links to that page in relevant locations throughout the API docs.

For the majority of endpoints, the documentation does not include
the array in any examples of return values, and instead links to
the error handling page. The exceptions are the three endpoints
that had previously supported this return value. The changes note
and example for these endpoints is also used in the error
handling page.
2023-03-06 10:33:13 -08:00
Lauryn Menard 0f2472ed14 tests: Update `assert_json_success` to check for ignored parameters.
Updates `assert_json_success` so that it checks for an array of
ignored parameters in the success response.
2023-03-06 10:33:13 -08:00
Lauryn Menard b8197e06bc request-notes: Add `is_webhook_view` boolean to fields.
Adds `is_webhook_view` boolean field to the RequestNotes class so
that (when implemented) `ignored_parameters_unsupported` feature
is not something that is applied to webhooks.
2023-03-06 10:33:13 -08:00
Lauryn Menard 084dd216f0 tests: Remove ignored `realm_str` parameter from message send test.
In commit 8181ec4b56, we removed the `realm_str` as a parameter
for `send_message_backed`. This removes a missed test that included
this as a parameter for that endpoint/function.
2023-03-06 09:41:53 -08:00
Anders Kaseorg 087660a87e requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg 11e94c4bb7 html_diff: Fix lxml import.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg d3efd4c095 python: Import F, Q, QuerySet from their canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg aa577a554b populate_db: Import timedelta from its canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg 0628c3cac8 migrations: Import BaseDatabaseSchemaEditor from its canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg 43b4f10578 run-dev: Drop .py from script name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:37 -08:00
Alex Vandiver 3bf3f47b49 delete_old_unclaimed_attachments: Add flag to clean up storage.
Actions like deleting realms may leave unreferenced uploads in the
attachment storage backend.

Fix these by walking the complete contents of the attachment storage
backend, and removing files which are no longer present in the
database.  This may take quite some time, as it is necessarily O(n) in
the number of files uploaded to the system.
2023-03-02 16:36:19 -08:00
Alex Vandiver c9d1755a12 delete_realm: Optimize attachment cleanup by batching. 2023-03-02 16:36:19 -08:00
Alex Vandiver cdda4bc089 delete_realm: Clean up attachments when deleting or scrubbing realms. 2023-03-02 16:36:19 -08:00
Alex Vandiver b31a6dc56c upload: Reorder functions into logical groupings. 2023-03-02 16:36:19 -08:00
Alex Vandiver 880a3f95a7 tests: Split out s3 and local tests.
This mirrors the split of the code in 7c0d414aff.
2023-03-02 16:36:19 -08:00
Alex Vandiver 04e7621668 upload: Rename upload_message_image_from_request.
The table is named Attachment, and not all of them are images.
2023-03-02 16:36:19 -08:00
Alex Vandiver bd80c048be upload: Rename delete_message_image to use word "attachment".
The table is named Attachment, and not all of them are images.
2023-03-02 16:36:19 -08:00
Alex Vandiver 567d1d54e7 upload: Rename upload_message_file to use word "attachment".
For consistency with the table, which is named Attachment.
2023-03-02 16:36:19 -08:00
Lauryn Menard 96aca49f9f integrations: Update Asana integration doc to point to Zapier.
Updates the Asana documentation, which was a detailed version
of the Zapier documentation with screenshots specifically for
Asana, to instead start with the basic incoming webhook steps
and then point to the general Zapier documentation to complete
the integration.

This will be easier to maintain moving forward in the short
term as ideally we'll migrate to a system that documents all
of the integrations with Zulip that are available via Zapier.

Also, updates the current Zapier documentation to mention
Asana as one of the apps that can be integrated with Zulip.
2023-03-02 13:51:34 -08:00
Sahil Batra 3950a8e19d test_helpers: Rename reset_emails_in_zulip_realm.
This commit renames reset_emails_in_zulip_realm function to
reset_email_visibility_to_everyone_in_zulip_realm which makes
it more clear to understand what the function actually does.

This commit also adds a comment explaining what this function
does.
2023-03-01 12:17:11 -08:00
Sahil Batra 4e01449cfd register: Rename creating_new_team to creating_new_realm.
This commit renames creating_new_team variable to
creating_new_realm as "realm" seems better to explain
new realm creation than "team".
2023-03-01 12:17:11 -08:00
Alex Vandiver 43800b4c55 invites: Limit invites per day as a function of current users. 2023-03-01 11:44:25 -08:00
David Rosa 15fd5299af help: Update filenames "private message" -> "direct message".
Renames files, updates references, and adds URL redirects.
2023-02-28 16:55:13 -08:00
Anders Kaseorg 738667b39e templates: Convert config errors to HTML.
Markdown and Jinja don’t mix correctly, and templating is not an
appropriate use of Markdown.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-28 16:54:15 -08:00
Lauryn Menard d851662008 demo-orgs: Fix link to help center in Welcome bot message.
The inital Welcome bot message has an extra section if the user is
joining a demo organization, but the link in that section was not
being formatted correctly. Fixes the formatting so that the link
works.
2023-02-28 16:52:20 -08:00
Mateusz Mandera f0f02d05ab send_message_backend: Remove the realm_str API param.
This already became useless in 6e11754642,
as detailed in the API changelog entry here. At this point, we should
eliminate this param and the weird code around it.

This commit also deletes the associated tests added in
6e11754642, since with realm_str removed,
they make no sense anymore (and actually fail with an OpenAPI error due
to using params not used in the API). Hypothetically they could be
translated to use the subdomain= kwarg, but that also doesn't make
sense, since at that point they'd be just testing the case of a user
making an API request on a different subdomain than their current one
and that's just redundant and already tested generally in
test_decorators.
2023-02-28 12:12:15 -08:00
Mateusz Mandera 82379c31e4 send_message_backend: Set the realm value passed down to check_message.
This leftover variable, as a result of older changes, was just always
set to None. That was fine, because when realm=None reaches
check_message further down the codepath, it just infers from
sender.realm. We want to stop passing None like that though, so let's
just set this to user_profile.realm.
2023-02-28 12:12:15 -08:00
Lauryn Menard 514e5b990e templates: Update `reset_done.html` page text and title.
Updates the text and title used when the password reset done page
to work for situations where the user is resetting a forgotten
password and for situation where the user is setting a password
for the first time (e.g. SSO login, demo organizations).
2023-02-28 11:45:01 -08:00
Lauryn Menard 8eaac945b2 integrations-docs: Fix indentation issues in existing documentation.
Fixes some integrations documentation so that code blocks and
images are rendered in the correct HTML list elements.
2023-02-28 11:31:12 -08:00
Alex Vandiver 015a10637b docs: Document how to use SMTP without authentication.
This is the behaviour inherited from Django[^1].  While setting the
password to empty (`email_password = `) in
`/etc/zulip/zulip-secrets.conf` also would suffice, it's unclear what
the user would have been putting into `EMAIL_HOST_USER` in that
context.

Because we previously did not warn when `email_password` was not
present in `zulip-secrets.conf`, having the error message clarify the
correct configuration for disabling SMTP auth is important.

Fixes: #23938.

[^1]: https://docs.djangoproject.com/en/4.1/ref/settings/#std-setting-EMAIL_HOST_USER
2023-02-27 11:59:48 -08:00
Anders Kaseorg cdfa294b9a alertmanager: Indent a documentation code block into its list item.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-27 11:49:36 -08:00
Alex Vandiver fe654b76b7 data_import: Stop tar'ing up converted data.
`./manage.py import` does not take a tarball; it takes a directory.
Making a separate tarball is a waste of CPU time and disk, as it is
never used.

This was included in the commit of the initial Slack conversion code
in 5b37c5562b and propagated from there into every conversion tool.

Remove the unnecessary tarball creation.
2023-02-26 17:42:01 -08:00