Commit Graph

4872 Commits

Author SHA1 Message Date
Anders Kaseorg afa218fa2a semgrep: Detect some unsafe uses of markupsafe.Markup.
Use the built-in HTML escaping of Markup("…{var}…").format(), in order
to allow Semgrep to detect mistakes like Markup("…{var}…".format())
and Markup(f"…{var}…").

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-22 11:23:27 -07:00
Anders Kaseorg 3a27b12a7d dependencies: Switch to pnpm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-20 15:48:29 -07:00
Anders Kaseorg b3f9fdf99a dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-20 15:48:29 -07:00
Anders Kaseorg 14352547cd install-transifex-cli: Upgrade Transifex CLI from 1.6.4 to 1.6.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-17 13:11:02 -07:00
Prakhar Pratyush ab8222c4ae custom_check: Update FILES_WITH_LEGACY_SUBJECT set.
This commit adds 'zerver/lib/email_notifications.py'
to the FILES_WITH_LEGACY_SUBJECT set.

Because the file can have 'subject' in the email sense,
it should be exempted from the 'avoid subject as a var' lint rule.
2023-03-14 17:48:57 -07:00
sbansal1999 5f4bdfe6bc ts: Convert css_variables.js to TypeScript. 2023-03-10 14:35:23 -08:00
palashb01 dee63680df docs: Update time format and Emoticon help page.
Updated the title and description in the 'enable-emoticon-translation'
file and renamed the file accordingly. Added a new bullet point for
'time format' in the 'configure-new-user-settings.md' file and updated
the sidebar index by replacing the title 'Use 24-hour time' with
'Change the time format'.
2023-03-09 16:12:45 -08:00
Anders Kaseorg 2d9b2a2a05 models: Remove type prefixes from __str__ values.
The Django convention is for __repr__ to include the type and __str__
to omit it.  In fact its default __repr__ implementation for models
automatically adds a type prefix to __str__, which has resulted in the
type being duplicated:

    >>> UserProfile.objects.first()
    <UserProfile: <UserProfile: emailgateway@zulip.com <Realm: zulipinternal 1>>>

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Alex Vandiver 8f8a9f6f04 sentry: Add frontend event monitoring.
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors.  However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors.  Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.

Add a client-side Javascript sentry integration.  To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry.  Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
2023-03-07 10:51:45 -08:00
Alex Vandiver a8181152b7 webpack: Build a ZULIP_VERSION global constant into the built product. 2023-03-07 10:51:45 -08:00
Daniil Fadeev d606f5ba42 i18n: Add new markup for lazy translation of giphy ratings.
Also, it adds phrases such as 'rated Y' to the ignored list.
Otherwise, the linter would require to write it as 'rated y'.
2023-03-06 10:52:52 -08:00
Anders Kaseorg 7b47b6a311 docs: Move homepage link to TOC tree.
This lets us avoid maintaining a forked copy of sphinx_rtd_theme’s
sidebartitle block.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 23:06:17 -08:00
Anders Kaseorg 821584b8fc gitlint-rules: Rename to gitlint_rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:38 -08:00
Anders Kaseorg 43b4f10578 run-dev: Drop .py from script name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:37 -08:00
Sahil Batra 3950a8e19d test_helpers: Rename reset_emails_in_zulip_realm.
This commit renames reset_emails_in_zulip_realm function to
reset_email_visibility_to_everyone_in_zulip_realm which makes
it more clear to understand what the function actually does.

This commit also adds a comment explaining what this function
does.
2023-03-01 12:17:11 -08:00
Lauryn Menard 42775219b3 private-messages: Update frontend strings to use "direct message" or "DM".
Updates frontend user-facing strings with "private message" or "PM" to
use "direct message" or "DM" respectively instead.

Note that this updates translated strings as well as a few that
are not translated like search suggestions.

Updates `tools/lib/capitalization.py` for some specific strings
that are impacted by these changes, and removes "PM" and "PMs"
from checked strings.
2023-02-24 11:47:26 -08:00
Lauryn Menard e39c689f36 email-notifications: Update email subjects with "PM" to "DM".
Updates email notification subjects with "PM" to use "DM" instead.

Also, updates `tools/lib/capitalization.py` to include both "DM"
and "DMs".
2023-02-24 11:47:26 -08:00
Anders Kaseorg 0ef8e88b17 webpack: Move webpack configuration to web.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-24 06:35:58 -08:00
Anders Kaseorg 377b8b22c0 puppeteer_tests: Move to web/e2e-tests.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Anders Kaseorg cea1119423 node_tests: Move to web/tests.
This lets us simplify the long-ish ‘../../static/js’ paths, and will
remove the need for the ‘zrequire’ wrapper.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Anders Kaseorg c1675913a2 web: Move web app to ‘web’ directory.
Ever since we started bundling the app with webpack, there’s been less
and less overlap between our ‘static’ directory (files belonging to
the frontend app) and Django’s interpretation of the ‘static’
directory (files served directly to the web).

Split the app out to its own ‘web’ directory outside of ‘static’, and
remove all the custom collectstatic --ignore rules.  This makes it
much clearer what’s actually being served to the web, and what’s being
bundled by webpack.  It also shrinks the release tarball by 3%.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Alex Vandiver e0a7658db7 ts: Pull in static assets via import. 2023-02-14 17:17:06 -05:00
Anders Kaseorg ec58b6790d install-node: Upgrade Node.js to 18.14.0; manage Yarn with Corepack.
Corepack manages multiple per-project version of Yarn and PNPM, which
means we have to maintain less installation code, and could help us
switch away from Yarn 1 without making the system unusable for
development of other Yarn 1 projects.

https://nodejs.org/api/corepack.html

The Unicode spaces in the timerender test resulted from an ICU
upgrade: https://github.com/nodejs/node/pull/45068.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-09 15:50:47 -08:00
Anders Kaseorg 6992d3297a ruff: Fix PIE810 Call `startswith` once with a `tuple`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-08 16:40:35 -08:00
Sahil Batra 34fa712220 settings: Clarify message-delete-time-limit setting rule.
This commit adds note to the message-delete-time-limit
setting label that the time limit does not apply to
administrators.
2023-02-08 12:46:05 -08:00
Sahil Batra 4378b1639d settings: Add move_messages_within_stream_limit_seconds element in UI.
This commit adds dropdown for move_messages_within_stream_limit_seconds
setting which is used to control for how long the user is allowed to
edit topic.
2023-02-08 12:46:05 -08:00
Aman Agrawal 4dfe3d36e0 floating_recipient_bar: Replace with sticky header. 2023-02-07 16:25:39 -08:00
evykassirer d54618fb4b node tests: Exclude zjquery_element.js and upload.js.
CZO discussion
[here](https://chat.zulip.org/#narrow/stream/6-frontend/topic/code.20coverage/near/1487773).

This allows `zjquery_element.js` to have test utils that aren't always being used.
`upload.js` is mostly UI focused and has upcoming commits that are hard
to write unit tests for (and for which unit tests wouldn't make much sense).
2023-02-04 19:50:52 -08:00
evykassirer 5f59ea0036 compose banner: Consolidate two functions that clear success banners.
Previously notifications.clear_compose_notifications was used accross
the codebase. Since introducing the new
compose_banner.clear_message_sent_banners function, the two functions
are similar enough that we can just use clear_message_sent_banners
everywhere. This commit also moves scroll_to_message_banner_message_id
to compose_banner.
2023-02-04 18:53:35 -08:00
Anders Kaseorg 5aaa36b8fc ruff: Enable logging format rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-04 16:36:20 -08:00
Anders Kaseorg da3cf5ea7a ruff: Fix RSE102 Unnecessary parentheses on raised exception.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-04 16:34:55 -08:00
Anders Kaseorg 8dc3e086c9 semgrep: Remove typing.Text rule superseded by Ruff UP019.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-03 17:37:31 -08:00
Lauryn Menard dbacc00f0f api-docs: Move markdown files to top level directory.
- Updates `.prettierignore` for the new directory.
- Updates any reference to the API documentation directory for
  markdown files to be `api_docs/` instead of `zerver/api/`.
- Removes a reference link from `docs/documentation/api.md` that
  hasn't referenced anything in the text since commit 0542c60.
- Update rendering of API documentation for new directory.
2023-02-02 17:25:40 -08:00
Alex Vandiver a8d0908893 tail-ses: Support noreply address of `Real Name <noreply@example.com>`. 2023-02-02 11:33:40 -08:00
Alex Vandiver 98711b0e45 tail-ses: Set AWS region based on S3_REGION value. 2023-02-02 11:33:40 -08:00
Anders Kaseorg df001db1a9 black: Reformat with Black 23.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.

(This does not actually upgrade our Python environment to Black 23
yet.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-02 10:40:13 -08:00
Anders Kaseorg 4eda29bd86 ruff: Fix RUF005 Consider spread instead of concatenation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-26 10:16:30 -08:00
Anders Kaseorg 7e3a681f80 ruff: Fix S108 Probable insecure usage of temporary file.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-26 10:14:56 -08:00
Lauryn Menard dedea23745
help-docs: Move help center documentation to top level directory.
These files are not Jinja2 templates, so there's no reason that they needed
to be inside `templates/zerver`. Moving them to the top level reflects their
importance and also makes it feel nicer to work on editing the help center content, 
without it being unnecessary buried deep in the codebase.
2023-01-25 14:08:29 -08:00
Alex Vandiver ad8041253c release: Add tests for spelling and links in changelog. 2023-01-24 13:30:04 -08:00
Alex Vandiver 1eb31f4f54 upload-release: Update SHA256SUM for existing files if changed.
In the very infrequent case that we re-upload a file, the logic for
upload-release would have the existing SHA256SUM take precedence over
the new one -- despite uploading the new file.

Skip existing filenames (i.e. from the new file) when gathering
SHA256SUM metadata used to assemble `SHA256SUMS.txt`.
2023-01-23 17:41:29 -08:00
Anders Kaseorg 7a7513f6e0 ruff: Fix SIM201 Use `… != …` instead of `not … == …`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg 6303ebfc2f ruff: Fix SIM115 Use context handler for opening files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg c6b6004432 ruff: Fix SIM111 Use `return all(…)` instead of `for` loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg ff1971f5ad ruff: Fix SIM105 Use `contextlib.suppress` instead of try-except-pass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg b0e569f07c ruff: Fix SIM102 nested `if` statements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Lalit 112df91fbd provision: Do not require that no other yarn precedes us in $PATH.
`check_version` in `install-yarn` had the rather careful check that
the yarn it installed into `/usr/bin/yarn` was the yarn which was
first in the user's `$PATH`.  This caused problems when the user had a
pre-existing `/usr/local/bin/yarn`; however, those problems are
limited to the `install-yarn` script itself, since the nearly all
calls to yarn from Zulip's code already hardcode the `/srv/zulip-yarn`
location, and do not depend on what is in `$PATH`.

Remove the checks in `install-yarn` that depend on the local `$PATH`,
and stop installing our `yarn` into it.  We also adjust the two
callsites which did not specify the full path to `yarn`, so use
`/srv/zulip-yarn`.

Fixes: #23993

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2023-01-19 17:51:52 -05:00
Anders Kaseorg 8f7a7877fe python: Clean up janky URL matching code with urlsplit.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-18 17:25:46 -05:00
Josh Klar 70b30e7792 i18n: Unescape Unicode sequences in JSON.
This greatly improves the readability of the diffs and in-codebase
translation strings over using ASCII sequences for unicode in the JSON.

We've previously noticed [^1] some JSON translation files ending up with
escaped Unicode sequences on disk, which Transifex indicates is expected
behavior [^2], though it is sometimes fixed by `manage.py
compilemessages` [^3]. Further, as noted in #23932 [^4], some JSON
translation files include HTML-escaped entities like quotation marks.

This script will ingest valid JSON files and output them as proper UTF-8
files with appropriately unescaped (unless otherwise necessary, like
double quotes being backslash-escaped) sequences, except when the key
itself contains HTML escape sequences (as it's presumed the value of
such entries must be pre-escaped before being passed to consumers).

[^1]: https://chat.zulip.org/#narrow/stream/58-translation/topic/Transifex.20client/near/1479205

[^2]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481287

[^3]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481908

[^4]: Which is not end-to-end fixed yet by this commit: that will
require a new release of Zulip Server.

gitlint-ignore: B1, title-trailing-punctuation, body-min-length, body-is-missing
2023-01-17 13:19:45 -08:00
Alex Vandiver 7c0d414aff uploads: Split out S3 and local file backends into separate files.
The uploads file is large, and conceptually the S3 and local-file
backends are separable.
2023-01-09 18:23:58 -05:00