Commit Graph

5120 Commits

Author SHA1 Message Date
Lalit e132a68193 refactor: Create `compose_textarea` and extracted related handlers from `ui.js`.
Created the new `compose_textarea` and moved event handlers for `#compose_textarea`
from `ui` to this new module so that it is now responsible for initializing event handlers
for compose_textarea instead of `ui` module.
2023-04-26 12:59:08 -07:00
sbansal1999 aad2f7c7c5 compose: Remove empty message warning border after closing.
Adding this made the line coverage of
"web/tests/lib/zjquery_element.js" equal to 100%, hence it
was required to remove it from the exempt files list.
2023-04-26 10:58:08 -07:00
Anders Kaseorg b8023da17a dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 22:18:48 -07:00
Anders Kaseorg 03b3c8522d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 21:20:33 -07:00
Anders Kaseorg 9db3451333 Remove statsd support.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 19:58:16 -07:00
Prakhar Pratyush a079bb811b user_topics: Rename 'settings_muted_topics.js'.
This commit renames 'settings_muted_topics.js'
to 'settings_user_topics.js' because the file now
supports the settings for topics with any visibility_policy,
not just MUTED.

It also renames the corresponding test file.
2023-04-25 12:46:05 -07:00
AcKindle3 70b8eb3423 TS migration: Convert `localstorage.js` to `localstorage.ts`.
Added function parameter types, return type, and types of local
varaibles. Added a `null` check for `raw_data` before `JSON.parse`.
Created a type `FormData` and an export type `LocalStorage` to
imporve conciseness and clearity.

Type `LocalStorage` is exported because it might be used in other
files based on an observation that many files have imported `localstorage`.
2023-04-24 11:59:02 -07:00
evykassirer 0c706aeafc compose: Extract a compose_recipient module.
This helps reduce the amount of import cycles we have in the compose
code path following the migration to a fancier stream input.

`compose_closed_ui.initialize()` was moved further down in the
initialization order because it relies on the dropdown widget
to be defined.
2023-04-20 16:09:55 -07:00
Zixuan James Li 6f42542320 migrations: Remove filter_format_validator.
This removes the validator argument for 0423_realmfilter_url_template,
which do not really alter the database schema. It otherwise fails
the migration because the filter_format_validator function is removed.

Migration 0094_realm_filter_url_validator is modified because we can no
longer refer to filter_format_validator.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
Zixuan James Li 268f858f39 linkifier: Support URL templates for linkifiers.
This swaps out url_format_string from all of our APIs and replaces it
with url_template. Note that the documentation changes in the following
commits  will be squashed with this commit.

We change the "url_format" key to "url_template" for the
realm_linkifiers events in event_schema, along with updating
LinkifierDict. "url_template" is the name chosen to normalize
mixed usages of "url_format_string" and "url_format" throughout
the backend.

The markdown processor is updated to stop handling the format string
interpolation and delegate the task template expansion to the uri_template
library instead.

This change affects many test cases. We mostly just replace "%(name)s"
with "{name}", "url_format_string" with "url_template" to make sure that
they still pass. There are some test cases dedicated for testing "%"
escaping, which aren't relevant anymore and are subject to removal.
But for now we keep most of them as-is, and make sure that "%" is always
escaped since we do not use it for variable substitution any more.

Since url_format_string is not populated anymore, a migration is created
to remove this field entirely, and make url_template non-nullable since
we will always populate it. Note that it is possible to have
url_template being null after migration 0422 and before 0424, but
in practice, url_template will not be None after backfilling and the
backend now is always setting url_template.

With the removal of url_format_string, RealmFilter model will now be cleaned
with URL template checks, and the old checks for escapes are removed.

We also modified RealmFilter.clean to skip the validation when the
url_template is invalid. This avoids raising mulitple ValidationError's
when calling full_clean on a linkifier. But we might eventually want to
have a more centric approach to data validation instead of having
the same validation in both the clean method and the validator.

Fixes #23124.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
Zixuan James Li a19c80df43 linkifiers: Add url_template field.
This will later be used to expand matching linkifier patterns.
Making it nullable for now, but we will make it required in
the APIs.

As a part of this transition, we temporarily make url_format_string
nullable as well, which will be later removed. This allows us to
switch to populating url_template without caring about passing
url_format_string.

Note that the validators are imported in the migration because Django
otherwise diffs it and considers the schema to be different, generating
a migration, failing the "tools/test-migrations" test.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
Aman Agrawal 043d54d170 scheduled_messages: Add overlay to display and edit them.
Fixes #20971
2023-04-17 16:59:25 -07:00
Aman Agrawal c0ef1c360a message_send: Edit scheduled message if its ID is present.
If the ID of the scheduled message is passed by the client, we
edit the existing scheduled message instead of creating a new one.

However, this will soon be moved into its own API endpoint.
2023-04-14 17:38:37 -07:00
Tim Abbott 1d2cf734f3 lint: Limit JS assert rule to web/tests. 2023-04-14 12:26:33 -07:00
Alex Vandiver e536a14b61 report_error: Remove API endpoint for client error reporting. 2023-04-13 14:59:58 -07:00
Anders Kaseorg b3a185103d ruff: Enable new lints INT, PYI.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-12 17:32:38 -07:00
Hardik Dharmani 46cabdd59e user_topics: Use new API endpoint to update topics_visibility_policy.
Removed functions mute_topic and unmute_topic in muted_topics_ui.js
which were using old API endpoints to mute/unmute topics.

In user_topics.js added new function set_user_topic_visibility_policy
which uses new API endpoint `/user_topics` to update the topic's
visibility_policy to visibility_policy passed to
set_user_topic_visibility_policy function as a parameter.

In functions toggle_topic_mute and mute_or_unmute_topic, replaced
the calls of mute_topic and unmute_topic with the new
user_topics.set_user_topic_visibility_policy function.

Added "web/src/user_topics.js" in EXEMPT_FILES in test-js-with-node
as adding function `set_user_topic_visibility_policy` resulted in
user_topics.js losing 100% test coverage.

Fixes #24244
2023-04-11 09:51:01 -07:00
Sahil Batra 9278a219ea user_groups: Refactor get_realm_user_groups_for_dropdown_list_widget.
This commit refactors get_realm_user_groups_for_dropdown_list_widget
function to use "group_permission_settings" module instead of
passing variables like "allow_internet_group", etc. individually
for each settings.

We lose some test coverage due to this commit, but that would be
fixed when we add some more group-based settings and allow groups
other than system groups in the group based settings.
2023-04-11 09:02:09 -07:00
Sahil Batra a5cbf95c12 settings: Add group_permission_settings.ts.
This commit adds a new module group_permission_settings.ts
which is used to define configuration options like
allow_owners_group, allow_nobody_group, etc. for group-based
settings similar to GroupPermissionSetting dataclass we have
in backend.
2023-04-11 09:02:09 -07:00
Aman Agrawal 8c744752ed stream_privacy: Use new lock and hashtag icons.
This commit doesn't modify the lock and hashtag icon in settings.
2023-04-10 23:41:30 -07:00
AcKindle3 ac4ec8888a ts migration: Convert `favicon.js` to `favicon.ts`.
Added type annotations to function parameters, function return
values and local variables. Added neccessary `if` statements to
enforce the objects having proper type before executing the later
operations.

An error type check in the catch block in `update_favicon`
function is added because the statement in the catch block requires
error to be a type `Error` in order to access `error.stack`.
2023-04-06 13:56:59 -07:00
Daniil Fadeev 2f203f4de1 emails: Inline CSS in emails in build_email.
Previously, we had an architecture where CSS inlining for emails was
done at provision time in inline_email_css.py. This was necessary
because the library we were using for this, Premailer, was extremely
slow, and doing the inlining for every outgoing email would have been
prohibitively expensive.

Now that we've migrated to a more modern library that inlines the
small amount of CSS we have into emails nearly instantly, we are able
to remove the complex architecture built to work around Premailer
being slow and just do the CSS inlining as the final step in sending
each individual email.

This has several significant benefits:

* Removes a fiddly provisioning step that made the edit/refresh cycle
  for modifying email templates confusing; there's no longer a CSS
  inlining step that, if you forget to do it, results in your testing a
  stale variant of the email templates.
* Fixes internationalization problems related to translators working
  with pre-CSS-inlined emails, and then Django trying to apply the
  translators to the post-CSS-inlined version.
* Makes the send_custom_email pipeline simpler and easier to improve.

Signed-off-by: Daniil Fadeev <fadeevd@zulip.com>
2023-04-05 12:22:29 -07:00
Anders Kaseorg 620b1aa1bd Revert "ts: Convert css_variables.js to TypeScript."
This reverts commit 5f4bdfe6bc (#24630).

This was causing Webpack to trigger type checking with the TypeScript
compiler, which is very expensive.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-04 10:54:01 -07:00
Anders Kaseorg 0d4dcc8583 install-transifex-cli: Upgrade Transifex CLI from 1.6.5 to 1.6.7.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-04 00:16:04 -07:00
Daniil Fadeev 5c6f842b28 emails: Replace Premailer with CSS-inline.
Primary goal of library replacement is improving execution speed.
This commit should not affect the functionality of the system
or make any changes to it.
2023-04-03 15:15:05 -07:00
Hardik Dharmani ebc07c0dd7 ts: Covert `dialog_widget.js` to typescript.
Created `WidgetConfig` type which has all the parameter that can be
passed launch(). Except html_heading, html_body, and on_click all
other fields are optional. Therefore, removed checking of undefined
mandatory fields as it will be done by typescript.

Co-authored-by: Hardik Dharmani <Ddharmani99@gmail.com>
2023-04-01 11:54:53 -07:00
Lalit fcbc6dcae4 ts: Convert `overlays.js` to TypeScript.
Refactored some code as well to avoid unneccesaary `undefined`
checks in `overlays`. To be exact created an action_overlay
object of type `Overlay` such that the `reset_state` is just
`action_overlay = undefined` and having attributes like `$element`
and `close_handler`.

This would ensure that if you have an `active_overlay`, there will
be a registered `close handler` attached to it without needing to
write additional checks for `close_handler` just to satisfy type
checker.
2023-04-01 11:54:53 -07:00
human 0de0743e91
ts: Migrate debug.js to TypeScript.
TypeScript doesn't understand Array.prototype.slice.call; rather than dealing with the
type-checking issues around it, we remove it; it was only necessary for ancient browsers where
NodeList isn’t directly iterable.
2023-04-01 11:50:34 -07:00
Adarsh Tiwari eed6f67fca
ts: Migrate scroll_bar.js to typescript.
Remove the winjs specific `getScrollbarWidth` property while we're at it; 
WinJS isn't relevant anymore.

Reorders methods due to TypeScript files enforcing define-before-use when
this commit was drafted.
2023-04-01 11:43:58 -07:00
Adarsh Tiwari 78fdb6135a
ts: Migrate gear_menu.js to TypeScript.
Includes adding various missing values to page_params.
2023-03-29 09:36:52 -07:00
Sahil Batra 1874491178 templates: Extract realm creation form html in new template file.
This commit extracts realm creation form html in a new file
realm_creation_form.html as we are reworking the organization
flow as per #24307 and this change would allow us to avoid
duplicating the code.
2023-03-27 15:44:42 -07:00
Alex Vandiver 93bfd3fb5f sentry: Provide a server-side tunnel.
Some well-intentioned adblockers also block Sentry client-side error
reporting.  Provide an endpoint on the Zulip server which forwards to
the Sentry server, so that these requests are not blocked.
2023-03-24 15:16:38 -07:00
Alex Vandiver 164d58bec9 web: Save a 301 redirect from /help to /help/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 8da1c51a06 web: Save a 301 redirect from /security to /security/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 3e307781dd web: Save a 301 redirect from /for/x to /for/x/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 4b21852e72 web: Save a 301 redirect from /why-zulip to /why-zulip/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 14d19c00d7 web: Save a 301 redirect from /features to /features/. 2023-03-24 14:51:01 -07:00
Alex Vandiver b5731c55b7 web: Save two 301 redirects from /privacy to /privacy/ to /policies/privacy. 2023-03-24 14:51:01 -07:00
Alex Vandiver ce847fd09c web: Save two 301 redirects from /terms to /terms/ to /policies/terms. 2023-03-24 14:51:01 -07:00
Alex Vandiver d9daeb13e0 web: Save a needless 301 redirect from /apps to /apps/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 61adc9cfcc web: Save a needless 301 redirect from /team to /team/. 2023-03-24 14:51:01 -07:00
Alex Vandiver c686c5ed0f web: Save a needless 301 redirect from /plans to /plans/. 2023-03-24 14:51:01 -07:00
Alex Vandiver bf06d493ee web: Save a needless 301 redirect from /history to /history/. 2023-03-24 14:51:01 -07:00
Alex Vandiver eb81e62eaa web: Save a needless 301 redirect from /hello to /hello/. 2023-03-24 14:51:01 -07:00
Alex Vandiver a1570ff3dd web: Save a needless 301 redirect from /integrations to /integrations/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 7448ab6234 bootstrap-aws-installer: Update the github.com SSH key.
Per https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/.
2023-03-24 12:01:55 -07:00
Anders Kaseorg 870238206c create-test-api-docs: Remove.
This tool plays fast and loose with HTML generation, and nobody seems
to use it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-24 11:55:12 -07:00
evykassirer 12a61fcec6 emoji: Remove parens and apostrophes from emoji names.
This was breaking rendering of the emoji with these names.
More context on CZO: https://chat.zulip.org/#narrow/stream/9-issues/topic/issue.20with.20built-in.20emoji.20name/near/1517735
2023-03-23 15:58:43 -07:00
Anders Kaseorg afa218fa2a semgrep: Detect some unsafe uses of markupsafe.Markup.
Use the built-in HTML escaping of Markup("…{var}…").format(), in order
to allow Semgrep to detect mistakes like Markup("…{var}…".format())
and Markup(f"…{var}…").

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-22 11:23:27 -07:00
Anders Kaseorg 3a27b12a7d dependencies: Switch to pnpm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-20 15:48:29 -07:00
Anders Kaseorg b3f9fdf99a dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-20 15:48:29 -07:00
Anders Kaseorg 14352547cd install-transifex-cli: Upgrade Transifex CLI from 1.6.4 to 1.6.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-17 13:11:02 -07:00
Prakhar Pratyush ab8222c4ae custom_check: Update FILES_WITH_LEGACY_SUBJECT set.
This commit adds 'zerver/lib/email_notifications.py'
to the FILES_WITH_LEGACY_SUBJECT set.

Because the file can have 'subject' in the email sense,
it should be exempted from the 'avoid subject as a var' lint rule.
2023-03-14 17:48:57 -07:00
sbansal1999 5f4bdfe6bc ts: Convert css_variables.js to TypeScript. 2023-03-10 14:35:23 -08:00
palashb01 dee63680df docs: Update time format and Emoticon help page.
Updated the title and description in the 'enable-emoticon-translation'
file and renamed the file accordingly. Added a new bullet point for
'time format' in the 'configure-new-user-settings.md' file and updated
the sidebar index by replacing the title 'Use 24-hour time' with
'Change the time format'.
2023-03-09 16:12:45 -08:00
Anders Kaseorg 2d9b2a2a05 models: Remove type prefixes from __str__ values.
The Django convention is for __repr__ to include the type and __str__
to omit it.  In fact its default __repr__ implementation for models
automatically adds a type prefix to __str__, which has resulted in the
type being duplicated:

    >>> UserProfile.objects.first()
    <UserProfile: <UserProfile: emailgateway@zulip.com <Realm: zulipinternal 1>>>

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Alex Vandiver 8f8a9f6f04 sentry: Add frontend event monitoring.
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors.  However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors.  Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.

Add a client-side Javascript sentry integration.  To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry.  Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
2023-03-07 10:51:45 -08:00
Alex Vandiver a8181152b7 webpack: Build a ZULIP_VERSION global constant into the built product. 2023-03-07 10:51:45 -08:00
Daniil Fadeev d606f5ba42 i18n: Add new markup for lazy translation of giphy ratings.
Also, it adds phrases such as 'rated Y' to the ignored list.
Otherwise, the linter would require to write it as 'rated y'.
2023-03-06 10:52:52 -08:00
Anders Kaseorg 7b47b6a311 docs: Move homepage link to TOC tree.
This lets us avoid maintaining a forked copy of sphinx_rtd_theme’s
sidebartitle block.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 23:06:17 -08:00
Anders Kaseorg 821584b8fc gitlint-rules: Rename to gitlint_rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:38 -08:00
Anders Kaseorg 43b4f10578 run-dev: Drop .py from script name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:37 -08:00
Sahil Batra 3950a8e19d test_helpers: Rename reset_emails_in_zulip_realm.
This commit renames reset_emails_in_zulip_realm function to
reset_email_visibility_to_everyone_in_zulip_realm which makes
it more clear to understand what the function actually does.

This commit also adds a comment explaining what this function
does.
2023-03-01 12:17:11 -08:00
Lauryn Menard 42775219b3 private-messages: Update frontend strings to use "direct message" or "DM".
Updates frontend user-facing strings with "private message" or "PM" to
use "direct message" or "DM" respectively instead.

Note that this updates translated strings as well as a few that
are not translated like search suggestions.

Updates `tools/lib/capitalization.py` for some specific strings
that are impacted by these changes, and removes "PM" and "PMs"
from checked strings.
2023-02-24 11:47:26 -08:00
Lauryn Menard e39c689f36 email-notifications: Update email subjects with "PM" to "DM".
Updates email notification subjects with "PM" to use "DM" instead.

Also, updates `tools/lib/capitalization.py` to include both "DM"
and "DMs".
2023-02-24 11:47:26 -08:00
Anders Kaseorg 0ef8e88b17 webpack: Move webpack configuration to web.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-24 06:35:58 -08:00
Anders Kaseorg 377b8b22c0 puppeteer_tests: Move to web/e2e-tests.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Anders Kaseorg cea1119423 node_tests: Move to web/tests.
This lets us simplify the long-ish ‘../../static/js’ paths, and will
remove the need for the ‘zrequire’ wrapper.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Anders Kaseorg c1675913a2 web: Move web app to ‘web’ directory.
Ever since we started bundling the app with webpack, there’s been less
and less overlap between our ‘static’ directory (files belonging to
the frontend app) and Django’s interpretation of the ‘static’
directory (files served directly to the web).

Split the app out to its own ‘web’ directory outside of ‘static’, and
remove all the custom collectstatic --ignore rules.  This makes it
much clearer what’s actually being served to the web, and what’s being
bundled by webpack.  It also shrinks the release tarball by 3%.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-23 16:04:17 -08:00
Alex Vandiver e0a7658db7 ts: Pull in static assets via import. 2023-02-14 17:17:06 -05:00
Anders Kaseorg ec58b6790d install-node: Upgrade Node.js to 18.14.0; manage Yarn with Corepack.
Corepack manages multiple per-project version of Yarn and PNPM, which
means we have to maintain less installation code, and could help us
switch away from Yarn 1 without making the system unusable for
development of other Yarn 1 projects.

https://nodejs.org/api/corepack.html

The Unicode spaces in the timerender test resulted from an ICU
upgrade: https://github.com/nodejs/node/pull/45068.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-09 15:50:47 -08:00
Anders Kaseorg 6992d3297a ruff: Fix PIE810 Call `startswith` once with a `tuple`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-08 16:40:35 -08:00
Sahil Batra 34fa712220 settings: Clarify message-delete-time-limit setting rule.
This commit adds note to the message-delete-time-limit
setting label that the time limit does not apply to
administrators.
2023-02-08 12:46:05 -08:00
Sahil Batra 4378b1639d settings: Add move_messages_within_stream_limit_seconds element in UI.
This commit adds dropdown for move_messages_within_stream_limit_seconds
setting which is used to control for how long the user is allowed to
edit topic.
2023-02-08 12:46:05 -08:00
Aman Agrawal 4dfe3d36e0 floating_recipient_bar: Replace with sticky header. 2023-02-07 16:25:39 -08:00
evykassirer d54618fb4b node tests: Exclude zjquery_element.js and upload.js.
CZO discussion
[here](https://chat.zulip.org/#narrow/stream/6-frontend/topic/code.20coverage/near/1487773).

This allows `zjquery_element.js` to have test utils that aren't always being used.
`upload.js` is mostly UI focused and has upcoming commits that are hard
to write unit tests for (and for which unit tests wouldn't make much sense).
2023-02-04 19:50:52 -08:00
evykassirer 5f59ea0036 compose banner: Consolidate two functions that clear success banners.
Previously notifications.clear_compose_notifications was used accross
the codebase. Since introducing the new
compose_banner.clear_message_sent_banners function, the two functions
are similar enough that we can just use clear_message_sent_banners
everywhere. This commit also moves scroll_to_message_banner_message_id
to compose_banner.
2023-02-04 18:53:35 -08:00
Anders Kaseorg 5aaa36b8fc ruff: Enable logging format rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-04 16:36:20 -08:00
Anders Kaseorg da3cf5ea7a ruff: Fix RSE102 Unnecessary parentheses on raised exception.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-04 16:34:55 -08:00
Anders Kaseorg 8dc3e086c9 semgrep: Remove typing.Text rule superseded by Ruff UP019.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-03 17:37:31 -08:00
Lauryn Menard dbacc00f0f api-docs: Move markdown files to top level directory.
- Updates `.prettierignore` for the new directory.
- Updates any reference to the API documentation directory for
  markdown files to be `api_docs/` instead of `zerver/api/`.
- Removes a reference link from `docs/documentation/api.md` that
  hasn't referenced anything in the text since commit 0542c60.
- Update rendering of API documentation for new directory.
2023-02-02 17:25:40 -08:00
Alex Vandiver a8d0908893 tail-ses: Support noreply address of `Real Name <noreply@example.com>`. 2023-02-02 11:33:40 -08:00
Alex Vandiver 98711b0e45 tail-ses: Set AWS region based on S3_REGION value. 2023-02-02 11:33:40 -08:00
Anders Kaseorg df001db1a9 black: Reformat with Black 23.
Black 23 enforces some slightly more specific rules about empty line
counts and redundant parenthesis removal, but the result is still
compatible with Black 22.

(This does not actually upgrade our Python environment to Black 23
yet.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-02-02 10:40:13 -08:00
Anders Kaseorg 4eda29bd86 ruff: Fix RUF005 Consider spread instead of concatenation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-26 10:16:30 -08:00
Anders Kaseorg 7e3a681f80 ruff: Fix S108 Probable insecure usage of temporary file.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-26 10:14:56 -08:00
Lauryn Menard dedea23745
help-docs: Move help center documentation to top level directory.
These files are not Jinja2 templates, so there's no reason that they needed
to be inside `templates/zerver`. Moving them to the top level reflects their
importance and also makes it feel nicer to work on editing the help center content, 
without it being unnecessary buried deep in the codebase.
2023-01-25 14:08:29 -08:00
Alex Vandiver ad8041253c release: Add tests for spelling and links in changelog. 2023-01-24 13:30:04 -08:00
Alex Vandiver 1eb31f4f54 upload-release: Update SHA256SUM for existing files if changed.
In the very infrequent case that we re-upload a file, the logic for
upload-release would have the existing SHA256SUM take precedence over
the new one -- despite uploading the new file.

Skip existing filenames (i.e. from the new file) when gathering
SHA256SUM metadata used to assemble `SHA256SUMS.txt`.
2023-01-23 17:41:29 -08:00
Anders Kaseorg 7a7513f6e0 ruff: Fix SIM201 Use `… != …` instead of `not … == …`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg 6303ebfc2f ruff: Fix SIM115 Use context handler for opening files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg c6b6004432 ruff: Fix SIM111 Use `return all(…)` instead of `for` loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg ff1971f5ad ruff: Fix SIM105 Use `contextlib.suppress` instead of try-except-pass.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Anders Kaseorg b0e569f07c ruff: Fix SIM102 nested `if` statements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Lalit 112df91fbd provision: Do not require that no other yarn precedes us in $PATH.
`check_version` in `install-yarn` had the rather careful check that
the yarn it installed into `/usr/bin/yarn` was the yarn which was
first in the user's `$PATH`.  This caused problems when the user had a
pre-existing `/usr/local/bin/yarn`; however, those problems are
limited to the `install-yarn` script itself, since the nearly all
calls to yarn from Zulip's code already hardcode the `/srv/zulip-yarn`
location, and do not depend on what is in `$PATH`.

Remove the checks in `install-yarn` that depend on the local `$PATH`,
and stop installing our `yarn` into it.  We also adjust the two
callsites which did not specify the full path to `yarn`, so use
`/srv/zulip-yarn`.

Fixes: #23993

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2023-01-19 17:51:52 -05:00
Anders Kaseorg 8f7a7877fe python: Clean up janky URL matching code with urlsplit.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-18 17:25:46 -05:00
Josh Klar 70b30e7792 i18n: Unescape Unicode sequences in JSON.
This greatly improves the readability of the diffs and in-codebase
translation strings over using ASCII sequences for unicode in the JSON.

We've previously noticed [^1] some JSON translation files ending up with
escaped Unicode sequences on disk, which Transifex indicates is expected
behavior [^2], though it is sometimes fixed by `manage.py
compilemessages` [^3]. Further, as noted in #23932 [^4], some JSON
translation files include HTML-escaped entities like quotation marks.

This script will ingest valid JSON files and output them as proper UTF-8
files with appropriately unescaped (unless otherwise necessary, like
double quotes being backslash-escaped) sequences, except when the key
itself contains HTML escape sequences (as it's presumed the value of
such entries must be pre-escaped before being passed to consumers).

[^1]: https://chat.zulip.org/#narrow/stream/58-translation/topic/Transifex.20client/near/1479205

[^2]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481287

[^3]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481908

[^4]: Which is not end-to-end fixed yet by this commit: that will
require a new release of Zulip Server.

gitlint-ignore: B1, title-trailing-punctuation, body-min-length, body-is-missing
2023-01-17 13:19:45 -08:00
Alex Vandiver 7c0d414aff uploads: Split out S3 and local file backends into separate files.
The uploads file is large, and conceptually the S3 and local-file
backends are separable.
2023-01-09 18:23:58 -05:00
Anders Kaseorg 2afdb46095 ruff: Enable new lints DTZ, ISC, PIE, PLW, Q, S, SIM.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg 17300f196c ruff: Fix ISC003 Explicitly concatenated string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg 2c5e114f8b ruff: Fix ISC001 Implicitly concatenated string literals on one line.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg d9f2f60f4b ruff: Fix DTZ011 `datetime.date.today()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg e1ed44907b ruff: Fix SIM118 Use `key in dict` instead of `key in dict.keys()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg cd66448311 install-shfmt: Upgrade shfmt to 3.6.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 14:01:34 -08:00
Anders Kaseorg 986bad8f90 install-shellcheck: Upgrade ShellCheck to 0.9.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 14:01:34 -08:00
Anders Kaseorg 8a8e538af7 install-aws-server: Use correct fd and exit code for usage message.
On --help, write to stdout and exits with 0; on error, write to stderr
and exit with 1.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 14:01:34 -08:00
Anders Kaseorg bd884c88ed Fix typos caught by typos.
https://github.com/crate-ci/typos

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-03 11:09:50 -08:00
Yuri Sulyma 911b3b5365 ts: Convert util.js to TypeScript; add (Raw)Message type in types.ts.
Co-authored-by: Zixuan James Li <p359101898@gmail.com>
2022-12-13 15:23:29 -08:00
Josh Klar 0acb96ea9d provision: Respect proxy settings when installing shellcheck+shfmt. 2022-12-13 12:34:08 -08:00
Josh Klar 0c44b933c8 provision: Replace transifex-client with new transifex-cli.
transifex-client went EOL on November 30, 2022, replaced by
transifex/cli [^1].

Swap this in-place, since per the upstream README [^2]:

> The current version of the client maintains backwards compatibility
> for the tx push and tx pull commands. So, if you have a CI setup that
> uses them, you should not have to change anything.

As the mobile team found out, this is a partial truth if one previously
used some of the more advanced CLI flags, but all workflows referenced
in tools/ and docs/ use forwards-compatible flags to the new version.

[^1]: https://github.com/transifex/transifex-client/

[^2]: a0f28a1cf3/README.md
2022-12-13 12:34:08 -08:00
Greg Price 214eec007a tools: Add script apns/prep-cert.
This prepares server certificates for talking to APNs.
2022-12-07 22:07:36 -08:00
Anders Kaseorg 8b6bd961e6 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-07 09:55:05 -08:00
Anders Kaseorg 872f4b41c1 ci: Check that non-scripts aren’t marked executable.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-07 09:54:01 -08:00
Tim Abbott 158352c339 lint: Remove stale compose_error linter rule.
The module has been renamed, but also the original linter rule was
likely ineffective because of the length of compose_banner calls
anyway.

I'm not particularly worried about future internationalization bugs
here, so removing seems better than replacing it.
2022-12-06 15:48:22 -08:00
evykassirer 606258e75b template_parser: Handle handlebar partial blocks. 2022-12-06 15:40:40 -08:00
evykassirer 8848359887 template_parser: Rename handlebars_singleton for consistency. 2022-12-06 15:40:40 -08:00
evykassirer 4b175514aa emoji: Remove variation selector when generating emoji.
Fixes #23625.

The U+FE0F character was appearing in the data supplied by
emoji-datasource-google, but isn't in the CLDR data. By
removing that variation selector character, all remaining
emoji (that were previously marked as "not found in custom
emoji name maps, but also not found in CLDR data. Skipping.")
are now supported.
2022-12-06 12:33:59 -08:00
Aman Agrawal eb45925b1a actions_popover: Use tippyjs instead of bootstrap to display popover.
Fixes #23494
Popover now automatically displays on left when there is not
enough space top or bottom of the reference element.
2022-12-06 09:58:40 -08:00
Aman Agrawal 4593fcf8d1 actions_popover: Move data to display the popover to a new module.
Since we are switching to tippyjs to display popovers, extracting
this data in a commit of its own, makes further changes easier to
review and merge.

Also, we could add tests with full coverage on the data
provided to popovers, possibly a good first issue for beginners.
2022-12-06 09:58:40 -08:00
Josh Klar 3a0620a40c tools: Reimplement CI failure script without using CircleCI endpoint.
Using curl to POST to the CircleCI workflow endpoint on CZO:

- Doesn't work on zulip/zulip@main (CZO runs a revert)
- Sets a bad example for other orgs
- Robs us of an opportunity to dogfood our own zulip/github-actions-zulip

Refactor the Actions workflows in this repo to report failure states
using the Zulip Action, and reimplement the related helper scripts in
Python, since they'd previously mostly shelled out to Python anyway.
2022-12-05 14:33:15 -05:00
Anders Kaseorg 4898fdb89d ruff: Enable some new errors.
PGH001 forbids eval().

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Anders Kaseorg c850a006d9 emoji: Import modules relative to ZULIP_PATH.
Newer mypy is less confused this way.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Anders Kaseorg 84bdd6371d emoji: Remove old broken CSV import/export scripts.
These have been broken at least since commit
e331a758c3 (#12787), so clearly nobody
cares.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Sahil Batra 00ad7e5aeb development: Add CSS file for emails page in dev server.
Previously, CSS rules were added inline for emails page in
dev server. This commit adds a new file which contains
CSS rules for the emails page in dev server. This will
also help us in adding focus for the radio buttons in
the page, which cannot be added inline to the elements,
when we remove the use of bootstrap for this page.
2022-11-30 17:35:22 -08:00
evykassirer 512705f267 emoji: Remove commas from emoji names.
Fixes part of #23626.

This was preventing emoji like `:family_man,_woman,_girl,_boy:`
from displaying.
2022-11-29 16:26:15 -08:00
Mateusz Mandera 51ed2442fa oneclick: Wait in a loop until the image build droplet gets an IP.
The script will fail if we process with a droplet object with no
.ip_address, which can sometimes happen if we just droplet.load().
Sometimes that gives droplet.ip_address set to None. If we wait in a
loop, we eventuall get the .ip_address set.
2022-11-18 12:45:08 -08:00
Anders Kaseorg 9972fcc85b custom_check: Remove exclusions for comment annotations check.
Commit 2762121162 (#21552) made them
unnecessary.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-18 10:08:24 -08:00
Anders Kaseorg 4dce3c54d9 custom_check: Remove assertEquals check.
Ruff checks this as U005 (and even auto-fixes it).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-18 10:08:24 -08:00
Anders Kaseorg 73c4da7974 ruff: Fix N818 exception name should be named with an Error suffix.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-17 16:52:00 -08:00
Anders Kaseorg 9a8a2bd345 ruff: Enable import sorting, replacing isort.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-16 09:29:11 -08:00
Alex Vandiver f3fd0b6975 release: Support -rc1 style suffixes for releases.
These suffixes suppress some checks in the process, but still generate
and upload a tarball, push a tag, and make a Github prerelease.
`upload-release` already understands that anything with a suffix never
becomes the "latest" release.
2022-11-15 12:50:41 -08:00
Alex Vandiver 1f1e1e4ec2 build-release-tarball: Override merge-base for full official releases. 2022-11-15 12:50:41 -08:00
Alex Vandiver e95e8b985f release: Be explicit about which branch was expected. 2022-11-15 12:50:41 -08:00
Alex Vandiver abb7a63b62 release: Be consistent about TZ-setting. 2022-11-15 12:50:41 -08:00
rht 5ce2103b87 Slack import: Cache emoji.json in static/generated/emoji.
Previously, emoji.json was read from
"$ZULIP_PATH/node_modules/emoji-datasource-google/emoji.json".
This path doesn't exist in production when installing from scratch from
a release tarball. And so, we ensure emoji.json exists by copying it to
`static/generated/emoji`.

With tweaks to comments by tabbott.

Fixes: #23469
2022-11-15 10:43:11 -08:00
Lauryn Menard 53df35e698 tools: Remove obsolete `_set_error_state` method in linter spider.
In 2f547ea, the custom `has_error` logic in BaseDocumentationSpider
was removed in favor of checking whether any errors were logged.

One reference to that method was missed in UnusedImagesLinterSpider
and went unremarked as no images were added to the documentation
that were not used.

Replaces that reference with an error log.
2022-11-09 11:54:19 -08:00
David Rosa 233e690fb4 tools: Upgrade scrapy's settings.py to the latest version.
Fixes a warning caused by using the deprecated `2.6` value
of the `REQUEST_FINGERPRINTER_IMPLEMENTATION` setting.

Upgrades `settings.py` to what would have been generated
by starting the documentation_crawler project using the
`startproject` command of scrapy v2.7.
2022-11-09 11:40:07 -08:00
Aman Agrawal d15b29f056 list_widget: Add functions to add and remove a rendered item in the list.
This a refactoring commit as a followup from #23385.
2022-11-08 13:27:48 -08:00
Alex Vandiver 6f46faa980 bootstrap-awscli: Upgrade awscli version. 2022-11-08 13:26:32 -08:00
Anders Kaseorg fc9f23a85f scripts: Pass --retry 3 to curl.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-08 08:07:36 -08:00
Anders Kaseorg 51d1ccb064 build_emoji: Guard mutable default with read-only type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-04 14:42:33 -07:00
Anders Kaseorg 7021852627 install-node: Silence expected “node: command not found” on first run.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:11:08 -07:00
Anders Kaseorg e1a2894af9 ruff: Fix B011 Do not `assert False` (`python -O` removes these calls).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Anders Kaseorg adffad384c lint: Replace pycodestyle and pyflakes with ruff.
https://github.com/charliermarsh/ruff

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Tim Abbott 931ed069b3 tools: Don't display release candidate as forks. 2022-11-02 21:38:24 -07:00
Anders Kaseorg ccfb516a79 lint: Enable caching for Prettier.
Caching is new in Prettier 2.7.0, but broken before 3.0.0-alpha.4.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-02 16:50:37 -07:00
Anders Kaseorg 683a6b0411 dependencies: Upgrade Prettier to 3.0.0-alpha.4.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-02 16:50:37 -07:00
Tim Abbott 8023a646de tools: Update source repos in total-contributions.
Otherwise, the output will incorrectly exclude any new commits in
repositories you'd used this tool with previously.
2022-10-31 16:27:31 -07:00
Tim Abbott 8b352e9321 total-contributions: Fix broken pathlib/os.path combination.
This is still messy, in that it does the `chdir` using a different
library than the rest, but it's at least more consistent with the
codebase and it should be possible to bulk migrate the `ZULIP_PATH`
calculation, which we have in a dozen files, to pathlib later.
2022-10-31 16:15:38 -07:00
Alex Vandiver 12a5317b8c bootstrap-aws-installer: Switch to an IMDSv2-compatible URL.
We don't use the token we request for anything, but a straight GET
request would fail in an IMDSv2-only environment.
2022-10-28 16:52:54 -07:00
Alex Vandiver 721a1d7a10 bootstrap-aws-installer: Drop "credential_source" in .aws/config.
Setting `credential_source` is used when assuming role credentials --
that is, when running as one role, use the AssumeRole right to become
someone else.

The AWS command-line tools only do this if `role_arn`, the role to
assume, is also set -- if it is not set, it transparently falls
through to IAM role attached to the EC2 instance profile.  However,
with the `aws-sdk-go` package, used by Teleport, this configuration
produces an error.

Remove the `credential_source = Ec2InstanceMetadata` line, which isn't
necessary for the AWS CLI, and interferes with Teleport operation.
2022-10-28 16:52:54 -07:00
Alex Vandiver 88fcf6862f bootstrap-aws-installer: Add ed-25519 host key. 2022-10-28 16:52:54 -07:00
jai2201 6f9e97921d pm_section: Create collapsible private messages section.
This commit introduces the change of rendering private messages
section as collapsible, whose data-fetching logic came with zulip#21357.

We now have separated out `Private messages` from `top_left_corner`
section and shifted it below the `global_filters` in a different
separate section along with stream list with common scroll bar
in left-sidebar.

The new PM section will be opened by-default on loading the page
and will have a toggle-icon in its header, clicking on which makes the
section collapse/expand accordingly.

In default view, only recent 5 PM threads would be shown
and would append the active conversation as the 6th one at last
if not present in those 5, similar to how topics list work.

In PM section with unreads, a maximum of 8 conversations
would be shown and rest of them would be hidden behind
the 'more conversations' li-item, clicking on which takes
to the zoomedIn view of PM section where all the present
PM threads would be visible and rest of the sections of left-sidebar
will get collapsed.

Fixes #20870.

Co-authored-by: Aman Agrawal <amanagr@zulip.com>
2022-10-26 13:47:08 -07:00
Barissa-Imran 15b9e9c7cc bots: Customize reactivation confirmation modal for bots.
Since bots can't join user groups, and end users don't think of bots
as a special class of users, it's worth having special text here.

Fixes: #23270.
2022-10-26 13:26:16 -07:00
Alex Vandiver 63d2565467 ci: Do not pre-install rabbitmq-server in Docker images.
Before Zulip 4.9, the Zulip install process left any already-installed
rabbitmq with whatever nodename it had previously configured.  Wince
this encodes the name of the host when it was installed, this does not
function well with containers.

Leave rabbitmq-server uninstalled, which lets the Zulip installation
process set the nodename to `localhost`, which ensures that it is
usable across container restarts.
2022-10-25 14:53:32 -04:00
Alex Vandiver b161cba078 ci: Backport two patches to the 3.4 prod install process. 2022-10-25 14:53:32 -04:00
Anders Kaseorg d41afb2b25 tools: Add missing spaces in concatenated string literals.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-24 14:52:49 -07:00
Aman Agrawal 587f52e351 documentation_crawler: Exclude communities page.
Since /communities/#all was failing the spider test due to `#all`
not being present as an `id` on the page, we exclude it from
this check saying that it is not really a documentation page.
2022-10-20 15:52:18 -07:00
Aman Agrawal e3f22a9907 recent_topics: Add mention indicator in row for unread topics.
Fixes #22984

Add an `@` icon in unread topics where user is mentioned.

We track a new set of `stream_id:topic` pairs for the unread mentions
so that recent topics instantly knows if a topic is unread and mentioned
or not.
2022-10-19 16:26:42 -07:00
madrix01 550a32bb20 recent_topics: Add Private message to recent_topics.
This commit adds private messages to the Recent topics view, to make
it an all-encompassing overview of recent activity visible to the user.

We add a filter "Include PM" to toggle whether PMs should be shown in
recent topics.

Fixes #19449.
2022-10-17 15:02:01 -07:00
Ganesh Pawar 4df8c6610f gear menu: Add organization info.
Fixes #22205
2022-10-13 23:45:25 -07:00
Anders Kaseorg 0affc7ac6f test_script: Don’t crash on second Puppeteer run.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-13 19:56:47 -07:00
Anders Kaseorg 762a2d8da4 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-11 10:50:57 -07:00
Anders Kaseorg 9cb908682b test_script: Run Puppeteer install.js in its own cwd.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-11 10:50:57 -07:00
Anders Kaseorg 0027d7dea7 test_script: Move Puppeteer downloads outside node_modules.
This way Puppeteer doesn’t have to re-download Chromium every time we
install new JavaScript dependencies.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-11 10:50:57 -07:00
Anders Kaseorg e0032223c8 test_script: Correct glob for puppeteer failure screenshots.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-11 10:50:57 -07:00
Alex Vandiver 993fdc3311 bootstrap-awscli: Upgrade awscli version. 2022-10-10 08:46:29 -07:00
Sahil Batra aaa5d5cb0e settings: Show typeahead for pronouns type custom profile field.
User can add any pronouns in this field, but we just show some
commonly used pronouns as suggestions in typeahead.
2022-10-06 17:56:26 -07:00
Aman Agrawal ab9c5ae482 actions_popover: Only wrap content to next line on small width.
Fixes #23017

We try to keep the width of actions popover at `max-content` till
small widths.
2022-10-06 16:58:33 -07:00
evykassirer c321f57785 settings: Explain that Google blob emojis are deprecated.
Part of fixing #19371.
2022-10-04 12:29:35 -07:00
evykassirer b2c8ca295f emoji: Support new emoji and fallback from blob to Google for new emoji.
Fixes #19371.
2022-10-04 12:29:35 -07:00
evykassirer 8a9e68e026 emoji: Finish script to generate emoji_names.py with CLDR data.
This script pulls from our previously custom-written emoji strings
and fills in the rest from CLDR. It also removes 4 custom emoji which
collide with some of the new CLDR names (they will now just be called
by their CLDR name).
2022-10-04 12:29:35 -07:00
Sahil Batra fda471ab80 settings: Italicize text inside parens for presence_enabled setting. 2022-09-27 10:43:51 -07:00
Anders Kaseorg b267b17677 python: Use ‘not in’ for more negated membership tests.
Fixes “E713 Test for membership should be `not in`” found by ruff (now
that I’ve fixed it not to ignore scripts lacking a .py extension).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:09:46 -07:00
Riken Shah ff899e5c31 read_receipts: Disable showing read receipts for Notification Bot msgs.
The implementation is simple, we just check if the
the message sender is a notification bot to decide if we
should show the read receipts list.

We also update the modal content styling to match the padding at the
top of the modal.

Fixes #22905
2022-09-23 16:20:25 -07:00
Lauryn Menard d5b7551f09 frontend: Implement 'invisible mode' feature.
Transitions the frontend of the web app to no longer use the
user status `away` field for setting a user's activity status
to be 'unavailable' (which is now a deprecated way to access
a user's `presence_enabled` setting).

Instead we now directly use and update the user's `presence_enabled`
setting for this feature.

Renames frontend code related to the feature to `invisible_mode`
vs `away`.

We lose node test coverage in `user_status.js` because we are now
using `channel.patch` to send these user setting updates to the
server.

Removes the temporary updates to `server_events_dispatch.py` (and
related tests) made in a previous commit, since we no longer have
or need the `away_user_ids` set.
2022-09-23 12:27:54 -07:00
David Rosa 02eec29433 docs: Rename setup-vagrant.md -> setup-recommended.md.
Renames the filename so that it accurately reflects its contents
given the changes to the "Recommended setup" page in the previous
commit, and updates all links accordingly.
2022-09-15 15:54:14 -07:00
Anders Kaseorg 8da2eb58ec template_parser: Spell “Handlebars” correctly.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-09 00:34:36 -07:00
Anders Kaseorg 7d485aa58f docs: Correct more “webapp” to “web app”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-08 20:23:28 -07:00
Anders Kaseorg 0cd76d17d7 requirements: Remove unused mypy.txt lock file.
It’s unused since commit b7ec1b55dd.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-08 11:08:06 -07:00
Purushottam Tiwari 7879f78917 user groups: Implement edit features in user group settings overlay.
Follow up for #22214.
2022-09-07 12:09:28 -07:00
Anders Kaseorg 4fb97576e9 run-codespell: Update codespell exclude list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-30 17:36:21 -07:00
Aman Agrawal 379c6acbea portico: Add page to list open organizations.
Fixes #22020
2022-08-30 16:02:06 -07:00
Lauryn Menard 8a20884230 tippy: Initialize tippy in portico.
As a prep-commit for updating the billing / corporate pages for
demo organizations, initialize tippy.js with a default setting
for portico pages to use in general.
2022-08-29 11:43:45 -07:00
evykassirer efee77b41f emoji: Add which emoji are supported to the /register call.
Fixes #21037.

This is part of fixing #19371. To bulk-add new emoji regularly,
mobile needs to know which servers support which emoji.
`staticfiles_storage.url` generates a unique URL with a hash
based on the file content, which lets mobile know if it needs
to update its locally stored data.
2022-08-26 17:58:31 -07:00
Anders Kaseorg 81d0f5bdd9 computed_settings: Set STATICFILES_DIRS to an existing absolute path.
Django has always expected this, but Django 4.0 added a system check
that spews warnings in production.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-26 15:55:59 -07:00
Adam Sah c6130ce4b1 email_notifications: Complete test coverage for module.
Following the last commit, there was this WELCOME_EMAIL_SENDER code
without an existing automated test.
2022-08-25 13:04:51 -07:00
Tim Abbott b8c362c991 release: Don't remove OUTPUT_DIR on exit.
This ends up deleting your local copy of the release tarball, which
can be annoying if you need to upload it manually to the GitHub
releases page.
2022-08-25 12:50:40 -07:00
Tim Abbott c07212ad60 release: Fix pushing new release tags. 2022-08-25 12:50:39 -07:00
Adam Sah b1293a84f8 testing: 100% coverage for zerver/webhooks/{librato,pivotal}. 2022-08-25 12:04:24 -07:00
David Rosa 5cac44be06 docs: Reorganize sections and pages about contributing to Zulip.
We should rearrange Zulip's developer docs to make it easier to
find the documentation that new contributors need.

Name changes
Rename "Code contribution guide" section -> "Contributing to Zulip".
Rename "Contributing to Zulip" page -> "Contributing guide".

Organizational changes to the newly-named "Contributing to Zulip":
Move up "Contributing to Zulip", as the third link in sidebar index.
Move up renamed "Contributing guide" page to the top of this section.
Move up "Zulip code of Conduct", as the second link of this section.
Move down "Licensing", as the last link of this section.
Move "Accessibility" just below "HTML and CSS" in Subsystems section.

Update all links according to the changes above.
Redirects should be added as needed.

Fixes: #22517.
2022-08-25 11:24:57 -07:00
Purushottam Tiwari c0acffb8a8 user groups: Set up right panel for user group edit.
This is preparatory commit that does basic UI set up for
user group edit in group settings overlay. This allows us to
write proper hashchange logic for user group settings overlay
under diffrent situations.
The work in this commit will be extended in further commits
to add proper UI and group edit logic.
2022-08-23 17:40:02 -07:00
Purushottam Tiwari a8c3be7fee user groups: Add new UI support for user group creation.
Add support for creation of user groups using right panel
of new user group settings overlay being developed as part
of https://github.com/zulip/zulip/issues/19526.

In further commits we will add support for editing user
groups using right panel of the overlay.

This commit also introduces a minor bug related hashchange
for #groups which would be a quick fix once we have UI
for group edit on #groups overlay.
2022-08-23 17:40:02 -07:00
Purushottam Tiwari 76aae8c5a0 user groups: Add user group rows in left for user group overlay. 2022-08-23 17:30:39 -07:00
Purushottam Tiwari 656f6a0fe9 user groups: Add overlay for user group settings.
Dedicated overlay for user group settings is added as part of
addressing zulip#19526.

The newely added overlay is currently empty and more UI
related to settings is to be added in further commits.
2022-08-23 17:30:39 -07:00
Purushottam Tiwari ff2db9c235 user groups: Rename settings_user_groups to settings_user_groups_legacy.
A preparatory commit to have legacy user group settings logic
as we move forward to redesign the user group settings.

This is done so that current user group settings are functional
while we are working on the redesign, and also to make it clear
that most of the code in this file will be deleted and developers
should avoid spending much time on it.
2022-08-23 17:30:39 -07:00
Tim Abbott de275da70a release: Fix OUTDIR typo.
This was missed in 58962f0b7b.
2022-08-22 22:40:01 -07:00
Aman Agrawal ef21f9107c urls: Move /hello files to the corporate folder. 2022-08-22 15:53:43 -07:00
Aman Agrawal c9ac233911 urls: Move /team files to the corporate folder. 2022-08-22 15:53:43 -07:00
Aman Agrawal 4a9b4fb91b urls: Move /features files to corporate folder. 2022-08-22 15:53:43 -07:00
Aman Agrawal afd31b739d templates: Move case-studies and /for pages to corporate folder.
Since these pages don't belong in a self-hosted server.
2022-08-22 15:53:43 -07:00
Anders Kaseorg dad0d51ada node_tests: Reenable asynchronous event_status test.
Fixes #20180.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-18 08:55:31 -07:00
Ganesh Pawar e758ec400b alert word: Convert inline form to modal. 2022-08-15 19:07:38 -07:00
Dinesh 48d2783559 read_receipts: Add support for displaying read receipts.
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.

Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.

Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.

Fixes #3618.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2022-08-12 13:16:35 -07:00
Julia Bichler 0a278c39d2 settings: Send email after deactivating user.
This adds a feature where an admin can choose to send an email
with custom content to an user after they deactivated them.

Fixes #18943.
2022-08-06 21:41:53 -07:00
Anders Kaseorg 707ae0e7c1 Revert "provision: Binary-patch OpenSSL ARM64 assembly bug."
This reverts commit 924df5aaf5.

The fix was released in Ubuntu 20.04 and Debian 11.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-05 22:45:46 -07:00
Anders Kaseorg 2508b579a6 upload: Replace boto3.Session with boto3.session.Session.
boto3-stubs seems to have dropped the former for some reason.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-30 06:46:34 -07:00
Anders Kaseorg b945aa3443 python: Use a real parser for email addresses.
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:47:33 -07:00
Zixuan James Li 62cbacbfe7 test_runner: Write failed tests with a side effect.
`DiscoverRunner.run_tests` has a return type of `int`. While
`Runner.run_tests` has a wildly different `Tuple[bool, List[str]]`.

This refactors it so that we have the correct return type, by passing
the additional information about failed tests through a side effect to directly
write the failed tests to a file.

Note that we have to make `failed_tests_path` optional as otherwise the method
signature will not be compatible with the supertype.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 13:59:30 -07:00
Zixuan James Li f88530575a test-backend: Cast test runner to the custom Runner type.
Mypy does not know the acccurate return type of `get_runner` that is
determined by the `TEST_RUNNER` setting. We need to cast it to the
correct type to use methods like `get_shallow_tested_templates`. Note
that we import conditionally to avoid adding runtime dependency on
`zerver`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-28 09:48:16 -07:00
Anders Kaseorg a2f43beef4 commit-message-lint: Take release branches into account.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-22 10:39:05 -07:00
Matt Keller 44e93613fb tools: Add executable bit to release tool. 2022-07-21 19:08:10 -07:00
evykassirer afab7002bf emoji: Add script to generate emoji_names.py, only with custom names.
This script pulls from our previously custom-written emoji strings
to prepare to fill in the rest from CLDR.

This commit has no user-facing changes.
2022-07-21 13:22:07 -07:00
evykassirer 190c0737a7 emoji: Store customized emoji in file separate from emoji_names.py.
Emoji that we specifically choose names or aliases for will be
stored in this new file so that we can generate emoji_names.py
from the custom names and also CLDR (for the rest).

This new file isn't used for anything yet (no user facing change);
it will be used in future commits.
2022-07-21 13:22:07 -07:00
Anders Kaseorg 412e90f601 provision: Avoid distutils; keep PROVISION_VERSION as a tuple.
distutils is deprecated in Python 3.10 and will be removed in Python
3.12.  We don’t need a full-powered version parser for this anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-20 22:03:30 -07:00
Alex Vandiver 58962f0b7b release: Use the right variable to build predictable tarball path.
`tools/build-release-tarball` respects `OUTPUT_DIR`, not `OUTDIR`.
2022-07-19 17:29:03 -07:00
Alex Vandiver 39efe6f312 ci: Remove references to CircleCI, which we no longer use. 2022-07-16 10:43:40 -07:00
Alex Vandiver f8e2d652e1 ci: Test upgrades from the minimum of each major version, not the max. 2022-07-16 10:43:40 -07:00
Zixuan James Li 3404282927 tail-ses: Add IdentityArgsDict.
This fixes having incompatible types of different items in
the dictionary, and makes `**identity_args` type-safe.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 13:49:11 -07:00
Anders Kaseorg c520890f54 node_tests: Remove low-hanging uses of __Rewire__.
When we were preparing the conversion to ES modules in 2019, the
primary obstacle was that the Node tests extensively relied on the
ability to reach into modules and mutate their CommonJS exports in
order to mock things.  ES module bindings are not mutable, so in
commit 173c9cee42 we added
babel-plugin-rewire-ts as a kludgy transpilation-based workaround for
this to unblock the conversion.

However, babel-plugin-rewire-ts is slow, buggy, nonstandard,
confusing, and unmaintained.  It’s incompatible with running our ES
modules as native ES modules, and prevents us from taking advantage of
modern tools for ES modules.  So we want to excise all use of
__Rewire__ (and the disallow_rewire, override_rewire helper functions
that rely on it) from the tests and remove babel-plugin-rewire-ts.

Commits 64abdc199e and
e17ba5260a (#20730) prepared for this by
letting us see where __Rewire__ is being used.  Now we go through and
remove most of the uses that are easy to remove without modifying the
production code at all.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-13 16:27:30 -07:00
Anders Kaseorg 81892df176 requirements: Upgrade to Django 4.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-13 16:07:17 -07:00
Ganesh Pawar c6d636ef48 realm_domains_modal: Migrate modal to dialog_widget. 2022-07-07 14:56:45 -07:00
Alex Vandiver c915400101 release: Write a tool to automate final release checks and steps. 2022-07-07 13:19:00 -07:00
Alex Vandiver d9958618a9 build-release-tarball: Use OUTPUT_DIR environment variable, if set. 2022-07-07 13:19:00 -07:00
Alex Vandiver 2d132bf342 build-release-tarball: Verify full releases are tagged and pushed.
This ensures that anything that looks like a full release will
identify itself as such.  It requires that the version have a tag at
the current commit (though it doesn't need to be pushed yet), as well
as the commits exist in `main` or a release branch in the remote.
2022-07-07 13:19:00 -07:00
Zixuan James Li b3f27b1843 mypy: Enable local_partial_types.
dmypy is innately incompatible with `local_partial_types` being
`False` as it uses fined-grained incremental mode for caching.

We need to enable `local_partial_types` for mypy as well so that the two
will behave the same way.

This requires us to add additional type annotation in certain siuations
but that's fine.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-06 17:33:13 -07:00
Zixuan James Li bf9f9c8b5d tools: Support running mypy daemon for better performance.
mypy daemon performs significantly better than running the regular
mypy cli tool when we type check the entire codebase multiple
times locally.

This adds running mypy daemon as an option for both
`tools/run-mypy` and `tools/lint`.

To ensure daemon messages like "Daemon started", "Daemon stopped"
won't get printed we filter any output that starts with "Daemon".

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-06 17:33:13 -07:00
Zixuan James Li ea3478e377 tools: Extract ROOT_DIR.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-06 17:33:13 -07:00
Anders Kaseorg e4e51b0eb4 install-shfmt: Upgrade shfmt from 3.5.0 to 3.5.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-06 17:23:16 -07:00
Anders Kaseorg feff1d0411 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-05 17:54:17 -07:00
Alex Vandiver aabbb2ad10 bootstrap-awscli: Leave CWD unchanged. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1b5e058d3b install-aws-server: Add a --debug-key to allow logins during bootstrap. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1a1061e77e install-aws-server: Switch to getopt. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1be9ab2690 install-aws-server: Assume zulip_ops::profile:: prefix on all roles.
This will require that any profile-specific sections of
`$HOME/.zulip-install-server.conf` be renamed to their short form.
2022-06-28 09:39:31 -04:00
Alex Vandiver 9a3be2af64 install-aws-server: Fix default_branch config name. 2022-06-28 09:39:31 -04:00
Alex Vandiver f5d4dea2f0 bootstrap-aws-installer: Leave `ubuntu` user deletion until the last.
This makes it easier to log in and figure out what went wrong if
something failed during the bootstrapping process.
2022-06-28 09:39:31 -04:00
Anders Kaseorg ef3510fa6d nginx: Remove legacy X-XSS-Protection header.
Support for this header was removed in Chrome 78, Safari 15.4, and
Edge 17.  It was never supported in Firefox.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 17:38:18 -07:00
Anders Kaseorg 25f6b53e90 docs: Capitalize Handlebars consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-27 13:35:10 -07:00
Anders Kaseorg d5c302feef run-dev: Remove unnecessary finish calls.
Tornado finishes the request automatically.  Avoids this error, hidden
until commit 81f7192ca3 (#22301), when
the browser tab is closed:

Traceback (most recent call last):
  File "/srv/zulip-py3-venv/lib/python3.8/site-packages/tornado/web.py", line 1683, in _execute
    result = await result
  File "tools/run-dev.py", line 280, in prepare
    await self.finish()
tornado.iostream.StreamClosedError: Stream is closed

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:38:43 -07:00
Anders Kaseorg 0dfde9db94 python: Simplify unnecessary getattr calls.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:38:22 -07:00
Anders Kaseorg 3bf8ee2156 python: Unquote some unnecessarily quoted type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:37:41 -07:00
Anders Kaseorg e069330084 check-thirdparty: License under GPLv2+.
Since python-debian is GPL licensed, our script that imports it should
arguably be GPL licensed as well.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-26 17:33:11 -07:00
Alex Vandiver 6cd4d907ba upload-release: Only copy full releases to zulip-server-latest.tar.gz.
`zulip-server-6.0-pre0.tar.gz` should not be copied to
`zulip-server-latest.tar.gz`.
2022-06-23 19:24:54 -07:00
Anders Kaseorg 81f7192ca3 tornado: Add missing await for finish calls.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-23 19:22:12 -07:00
Anders Kaseorg c944adfcc6 mypy: Enable redundant-expr errors.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-23 19:22:12 -07:00
Alex Vandiver 41deef40cf nagios: Switch to generic check_cron_file for queues and consumers.
These share a common root; 91da4bd59b duplicated the code, but
didn't move the existing uses to the new utility.
2022-06-22 12:07:38 -07:00
Alex Vandiver 91379fd67e ci: Update upgrade test to 5.3, from 5.2. 2022-06-21 17:40:33 -07:00
Alex Vandiver a35af3f38b install/upgrade: Allow new packages during `apt-get upgrade`.
`postgresql-14.4` is a notable upgrade in the PostgreSQL series, as it
fixes potential database corruption from `CREATE INDEX CONCURRENTLY`
statements which are run while rows are modified[1].  However, it also
requires an upgrade from `libllvm9` to `libllvm10`, which means it is
not installed by a mere `apt-get upgrade`.

Add the `--with-new-pkgs` flag to all of the potentially relevant
`apt-get upgrade` calls, so that this (and similar) packages are
upgraded successfully.

[1]: https://www.postgresql.org/docs/release/14.4/
2022-06-21 11:21:49 -07:00
Anders Kaseorg a7f9c4f958 logging: Pass more format arguments to logging.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-03 12:27:23 -07:00
Anders Kaseorg 79842fd228 install-shfmt: Upgrade shfmt from 3.4.3 to 3.5.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-02 12:03:49 -07:00