Commit Graph

50256 Commits

Author SHA1 Message Date
Alex Vandiver cd288af4d5 sentry: Better URL filtering.
Always add `webpack_public_path` to all of the URL filters, and make
it a regex.  Also adjust to respect undef realm_uri, such as when a
spectator.
2023-03-07 15:01:27 -08:00
Lauryn Menard dfb9f74017 api_docs: Add page documenting HTTP headers in the api docs.
Adds a page to the general api documentation about HTTP headers,
so that information about the special response headers for rate
limits have a more logical location in the docs and so that other
HTTP header information can be shared, such as `User-Agent`
conventions.

Adjusts some text and linking on the rest-error-handling page and
overview page for the REST API for the addition of the HTTP headers
page.
2023-03-07 12:51:17 -08:00
Lauryn Menard d9d17eb87e help: Add article for supported browsers.
Creates new help article for supported web browsers. Also updates
links in the `/apps` landing page as well as the unsupported-browser
template and the `set-up-your-account` article.

Fixes #14732.
2023-03-07 12:01:48 -08:00
David Rosa 1429c3cf10 help: Document "Who can unsubscribe others from this stream?" setting.
- Updates "Remove others" row in "Stream permissions" tables.
- Adds a section to "Add or remove users from a stream" describing
  how to configure the new setting.
- Documents the new setting in "Create a stream".
- Documents "Message retention period" setting in "Create a stream".

Fixes #24340.
2023-03-07 11:06:52 -08:00
Alya Abbott 2b50428ac5 help: Add context/motivation to Support the Zulip project help page. 2023-03-07 11:03:04 -08:00
Hardik Dharmani 74ff9a4beb
popovers: Add disabled button CSS for light theme.
Previously, we only had disabled button CSS for the dark theme in dialog widgets.

Fixes #24567.
2023-03-07 11:01:01 -08:00
Alex Vandiver 8f8a9f6f04 sentry: Add frontend event monitoring.
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors.  However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors.  Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.

Add a client-side Javascript sentry integration.  To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry.  Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
2023-03-07 10:51:45 -08:00
Alex Vandiver fc40d74cda hooks: Remove --project from sentry when not necessary. 2023-03-07 10:51:45 -08:00
Alex Vandiver 08251ac53b hooks: Fix typo in sentry error message. 2023-03-07 10:51:45 -08:00
Alex Vandiver 19691b170b blueslip: Report the built client version, as well as server version. 2023-03-07 10:51:45 -08:00
Alex Vandiver a8181152b7 webpack: Build a ZULIP_VERSION global constant into the built product. 2023-03-07 10:51:45 -08:00
Alex Vandiver 73631950a5 report: Use ZULIP_VERSION rather than shelling out to git every time. 2023-03-07 10:51:45 -08:00
Alex Vandiver 90b1e0b8b9 report: Provide user information for browser-side errors.
b4dd118aa1 changed how the `user_info_str` parsed information out of
the events it received -- but only changed the server errors, not the
browser errors, though both use the same codepath.  As a result, all
browser errors since then have been incorrectly marked as being for
anonymous users.

Build and pass in the expected `user` dict into the event.
2023-03-07 10:51:45 -08:00
Kartik Srivastava ce5d13f9b2 message_send: Handle notifications for UNMUTED topic in a muted stream.
This commit adds 'visibility_policy' as a
parameter to user_allows_notifications_in_StreamTopic
function.

This adds logic inside the user_allows_notifications_in_StreamTopic
function, to not return False when a stream is muted
but the topic is UNMUTED.

Adds a method `user_id_to_visibility_policy_dict`
to 'StreamTopicTarget' class to fetch
(user_id => visibility_policy) in single db query.

Co-authored-by: Kartik Srivastava <kaushiksri0908@gmail.com>
Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-03-06 19:15:45 -08:00
Kartik Srivastava e9cf2659cf user_topics: Refactor remove_topic_mute.
This commit replaces 'remove_topic_mute' with
'set_user_topic_visibility_policy_in_database' and
updates it to delete UserTopic row with any configured
visibility_policy and not just muting.
2023-03-06 19:15:45 -08:00
Kartik Srivastava f844cb6dad user_topics: Refactor add_topic_mute.
In order to support different types of topic visibility policies,
this renames 'add_topic_mute' to
'set_user_topic_visibility_policy_in_database'
and refactors it to accept a parameter 'visibility_policy'.

Create a corresponding UserTopic row for any visibility policy,
not just muting topics.

When a UserTopic row for (user_profile, stream, topic, recipient_id)
exists already, it updates the row with the new visibility_policy.

In the event of a duplicate request, raises a JsonableError.
i.e., new_visibility_policy == existing_visibility_policy.

There is an increase in the database query count in the message-edit
code path.

Reason:
Earlier, 'add_topic_mute' used 'bulk_create' which either
creates or raises IntegrityError -- 1 query.

Now, 'set_user_topic_visibility_policy' uses get_or_create
-- 2 queries in the case of creating new row.

We can't use the previous approach, because now we have to
handle the case of updating the visibility_policy too.
Also, using bulk_* for a single row is not the correct way.

Co-authored-by: Kartik Srivastava <kaushiksri0908@gmail.com>
Co-authored-by: Prakhar Pratyush <prakhar841301@gmail.com>
2023-03-06 19:15:45 -08:00
Prakhar Pratyush e9580f8c5a MutedTopicsTests: Remove real-time usage from tests.
This commit refactors the existing pattern (real-time usage)
used to assert 'date_muted' in tests.

A fixed value is used at the start of the test to
assert 'date_muted', replacing the timedelta or real-time usage pattern.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 826ea4162e user_topics: Refactor 'do_unmute_topic'.
Replaces 'do_unmute_topic' with 'do_set_user_topic_visibility_policy'
and associated minor changes.

This change is made to align with the plan to use a single function
'do_set_user_topic_visibility_policy' to manage
user_topic - visibility_policy changes and corresponding event
generation.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 2df2ef9f0f user_topics: Refactor 'do_mute_topic'.
This commit is a step in the direction of having a common
function to handle visibility_policy changes and event
generation instead of separate functions for each
visibility policy.

In order to support different types of topic visibility policies,
this renames 'do_topic_mute' to 'do_set_user_topic_visibility_policy'
and refactors it to accept a parameter 'visibility_policy'.
2023-03-06 19:15:45 -08:00
Prakhar Pratyush 9ecc610681 mute_topic: Update tests to not use lib functions directly.
The "add_topic_mute" and "remove_topic_mute" library functions
shouldn't be called directly from tests.
They should instead call "do_mute_topic" and "do_unmute_topic"

The reason being:
Library functions are meant to be internal interfaces
for just changing the database, and shouldn't generally be
called elsewhere.
2023-03-06 19:15:45 -08:00
Author: Tim Abbott e161b36b21 user_topics: Delete stale do_update_message comment.
The issue this comment refers to was addressed in
bd04733.
2023-03-06 19:15:45 -08:00
Joelute b2b86b5e6f message_feed: Hide compose banner when unsubbing from stream.
Currently, when a user marks messages as unread in a stream/topic
and unsubscribes from the stream, both subscribe button and compose
banner will remain visible. This change will hide the compose banner
when the user unsubscribes from the stream and hopes to create a
better flow and reduce confusion.
2023-03-06 18:08:41 -08:00
Alex Vandiver dcd8947c1b tests: Set LOCAL_AVATARS_DIR / LOCAL_FILES_DIR in tests.
7ad06473b6 split out `LOCAL_AVATARS_DIR` and `LOCAL_FILES_DIR` as
derived values from `LOCAL_UPLOADS_DIR`.  However, this means that all
places which set `LOCAL_UPLOADS_DIR` need to potentially propagate
that change into the derived values if they come *after*
`computed_settings.py` is applied.  It did this successfully in
`zerver/lib/test_runner.py` and the `use_s3_backend` decorator, but
did not adjust the late-set `zproject/test_extra_settings.py`.

This causes tests to share a single common set of avatars and
attachments directories.  In puppeteer tests, this leads to assertion
failures checking `assert_is_local_storage_path`; in backend tests,
this leads to races when checking the contents of the local storage
directory when run in parallel mode.

Set `LOCAL_AVATARS_DIR` and `LOCAL_FILES_DIR` based on
`LOCAL_UPLOADS_DIR` when it is pulled from the environment during
testing.
2023-03-06 11:52:23 -08:00
Tim Abbott c58e893cb6 linkifiers: Add missing coverage ignore.
This was incorrectly missing in the last linkifiers commit.
2023-03-06 10:57:50 -08:00
Daniil Fadeev dcb72448d8 giphy: Fix "PG13" typo. 2023-03-06 10:52:52 -08:00
Daniil Fadeev d606f5ba42 i18n: Add new markup for lazy translation of giphy ratings.
Also, it adds phrases such as 'rated Y' to the ignored list.
Otherwise, the linter would require to write it as 'rated y'.
2023-03-06 10:52:52 -08:00
Haridarshan Choudhary d335cb34af
message_events: Avoid full rerender when marking unread.
Previously, when you did a "mark as unread from here" operation, we triggered 
a full rerender of both the recent topics view and the message feed. This was 
needlessly expensive, on and a large server with a somewhat busy CPU from 
other applications, can cause a visible lag, even when the message feed that 
you're looking at only has like 3 messages in it.

Improve this by passing the set of modified messages to the rerender.

There's likely further improvements to be made here -- we shouldn't need to
do more than toggle the unread markers -- but this should be good enough to
eliminate the visible lag.

Fixes #24263.
2023-03-06 10:39:47 -08:00
Lauryn Menard e9bfdd1bf2 response: Implement ignored parameters with MutableJsonResponse class.
Creates `MutableJsonResponse` as a subclass of Django's `HttpResponse`
that we can modify for ignored parameters in the response content.

Updates responses to include `ignored_parameters_unsupported` in
the response data through `has_request_variables`. Creates unit
test for this implementation in `test_decorators.py`.

The `method` parameter processed in `rest_dispatch` is not in the
`REQ` framework, so for any tests that pass that parameter, assert
for the ignored parameter with a comment.

Updates OpenAPI documentation for `ignored_parameters_unsupported`
being returned in the JSON success response for all endpoints.
Adds detailed documentation in the error handling article, and
links to that page in relevant locations throughout the API docs.

For the majority of endpoints, the documentation does not include
the array in any examples of return values, and instead links to
the error handling page. The exceptions are the three endpoints
that had previously supported this return value. The changes note
and example for these endpoints is also used in the error
handling page.
2023-03-06 10:33:13 -08:00
Lauryn Menard 0f2472ed14 tests: Update `assert_json_success` to check for ignored parameters.
Updates `assert_json_success` so that it checks for an array of
ignored parameters in the success response.
2023-03-06 10:33:13 -08:00
Lauryn Menard b8197e06bc request-notes: Add `is_webhook_view` boolean to fields.
Adds `is_webhook_view` boolean field to the RequestNotes class so
that (when implemented) `ignored_parameters_unsupported` feature
is not something that is applied to webhooks.
2023-03-06 10:33:13 -08:00
sbansal1999 d1253043a5 ts: Convert linkifiers.js to TypeScript. 2023-03-06 10:30:00 -08:00
Lauryn Menard b5f5ab948a puppeteer: Fix small logging copy and paste error in `admin.test.ts`. 2023-03-06 09:42:11 -08:00
Lauryn Menard 084dd216f0 tests: Remove ignored `realm_str` parameter from message send test.
In commit 8181ec4b56, we removed the `realm_str` as a parameter
for `send_message_backed`. This removes a missed test that included
this as a parameter for that endpoint/function.
2023-03-06 09:41:53 -08:00
Pranav Joglekar 2c316ce1ac
gear_menu: Update 'Support Zulip' link.
Previously the 'Support Zulip' option linked to Github Sponsors.
We now link to our own https://zulip.com/help/support-zulip-project,
which describes more options for how to support the project.

Fixes #24230.
2023-03-06 08:01:56 -08:00
Anders Kaseorg c382904638 docs: Remove bad advice about pinning versions.
Because `yarn.lock` includes transitive dependencies, it already pins
our dependencies more comprehensively than `package.json` would if we
followed this bad advice, which we don’t, as of commit
9b0401b76d (#13118).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 21:26:55 -08:00
Anders Kaseorg 087660a87e requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg 11e94c4bb7 html_diff: Fix lxml import.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg d3efd4c095 python: Import F, Q, QuerySet from their canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg aa577a554b populate_db: Import timedelta from its canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg 0628c3cac8 migrations: Import BaseDatabaseSchemaEditor from its canonical module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-05 14:46:28 -08:00
Anders Kaseorg cca8417ad4 docs: Add missing link to outreach TOC.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 23:06:58 -08:00
Anders Kaseorg 7b47b6a311 docs: Move homepage link to TOC tree.
This lets us avoid maintaining a forked copy of sphinx_rtd_theme’s
sidebartitle block.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 23:06:17 -08:00
Anders Kaseorg 821584b8fc gitlint-rules: Rename to gitlint_rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:38 -08:00
Anders Kaseorg 43b4f10578 run-dev: Drop .py from script name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:37 -08:00
Anders Kaseorg 5a79ca251b check-database-compatibility: Drop .py from script name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:37 -08:00
Alex Vandiver 26eb1d7371 puppet: Also set systemd limits. 2023-03-03 16:39:47 -08:00
Alex Vandiver 1c76036c61 puppet: Increase maximum file descriptors.
The current threshold of 40k descriptors was set in 2016, chosen to be
"at least 40x our current scale."  At present, that only provides a
50% safety margin.  Increase to 1 million to provide the same 40x
buffer as previously.

The highest value currently allowed by the kernels in
production (linux 5.3.0) is 1048576.  This is set as the hard limit.

The 1 million limit is likely far above what the system can handle for
other reasons (memory, cpu, etc).  While this removes a potential
safeguard on overload due to too many connections, due to the longpoll
architecture we would generally prefer to service more connections at
lower quality (due to CPU limitations) rather than randomly reject
additional connections.

Relevant prior commits:
 - 836f313e69
 - f2f97dd335
 - ec23996538
 - 8806ec698a
 - e4fce10f46
2023-03-03 16:39:47 -08:00
Alex Vandiver a20bb54cbb puppet: Move limits.conf to maintain more of the installation structure. 2023-03-03 16:39:47 -08:00
David Rosa 140e62d024 help: Add open link icon in "Add a custom linkifier". 2023-03-03 14:19:20 -08:00
Lauryn Menard f1e60014fa css: Move `pre code` reset rules to `rendered_markdown` class block.
Moves and simplifies the `pre code` rules in `rendered_markdown.css`
so that they are all part of the `rendered_markdown` class block.
2023-03-03 14:10:06 -08:00