Commit Graph

5305 Commits

Author SHA1 Message Date
Anders Kaseorg ba1895cb1c notifications: Rename to desktop_notifications.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-06 16:22:40 -07:00
Anders Kaseorg fcaf1fe7d5 notifications: Split out audible_notifications module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-06 16:22:40 -07:00
Anders Kaseorg 6ef08873d8 notifications: Split out compose_notifications module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-06 16:22:40 -07:00
Anders Kaseorg db20fd12e0 notifications: Split out message_notifications module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-06 16:22:40 -07:00
Anders Kaseorg 971a003180 compose_actions: Extract compose_reply module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-06 09:07:43 -07:00
Anders Kaseorg 9c70f92514 reload: Move initialize to new reload_setup module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-06 08:34:39 -07:00
Anders Kaseorg c4d338a6ab drafts: Extract drafts_overlay_ui module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-06 08:30:31 -07:00
Anders Kaseorg d5064fc7b2 narrow: Extract narrow_title module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-05 16:26:45 -07:00
Anders Kaseorg e8a2a5ce36 compose: Move initialize to new compose_setup module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-05 14:42:19 -07:00
Anders Kaseorg 6e725f0d17 compose: Move call link insertion to new compose_call_ui module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-05 14:42:19 -07:00
Anders Kaseorg ac8436d46c resize: Move handler to new resize_handler module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-05 09:26:59 -07:00
Alex Vandiver 89f824c769 update-prod-static: Only run generate_landing_page_images if necessary.
The vast majority of deployments do not need landing page assets
generated every deploy, which takes more than 15s.  This also removes
them from built tarballs, which also do not need them.
2023-10-05 09:04:33 -07:00
Alex Vandiver 369eea27ba provision: Tweak comment to not just mention static assets. 2023-10-05 09:04:33 -07:00
Aman Agrawal 75826d8409 handlebars: Move popover handlebars to a new popovers folder.
Organize related handlebars in a new folder together.
2023-10-05 08:19:49 -07:00
Aman Agrawal 3415a6898c user_card_popover: Rename `_title` suffix to `_avatar`. 2023-10-05 08:19:49 -07:00
Mateusz Mandera f71e2c8247 support: Add basic support endpoint for remote servers. 2023-10-04 16:55:47 -07:00
Aman Agrawal 9388db83dd update-prod-static: Generate landing page when updating prod instance. 2023-10-04 15:54:57 -07:00
Aman Agrawal a6fbf570cd provision_inner: Add a reminder to check for production needs.
This script is not run in production. Help developers remind
that.
2023-10-04 15:54:57 -07:00
Anders Kaseorg b4878d4f0b stream_edit: Move set_stream_property to new stream_settings_api module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-04 13:04:50 -07:00
Prakhar Pratyush 28fb5ea24d settings: Add UI for automatically follow and unmute topics settings.
This commit adds two drop-down settings in 'SETTINGS / NOTIFICATIONS'
and 'SETTINGS / DEFAULT USER SETTINGS'.

The new settings lie in a new section named "Topic notifications",
just below the "Noification triggers" section.

Label: "Automatically follow topics"
Options: "Topics I participate in", "Topics I send a message to",
"Topics I start", and "Never".

Label: "Automatically unmute topics in muted streams"
Options: "Topics I participate in", "Topics I send a message to",
"Topics I start", and "Never".

Fixes #25914.
2023-10-04 13:04:29 -07:00
Prakhar Pratyush 58568a60d6 settings: Add automatically follow and unmute topics policy settings.
This commit adds two user settings, named
* `automatically_follow_topics_policy`
* `automatically_unmute_topics_in_muted_streams_policy`

The settings control the user's preference on which topics they
will automatically 'follow' or 'unmute in muted streams'.

The policies offer four options:
1. Topics I participate in
2. Topics I send a message to
3. Topics I start
4. Never (default)

There is no support for configuring the settings through the UI yet.
2023-10-04 13:04:29 -07:00
Anders Kaseorg ac38aaad21 scheduled_messages: Move UI to new scheduled_messages_ui module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 17:56:43 -07:00
Anders Kaseorg d4a93aa05a popover_menus: Extract compose_popovers module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 11:58:29 -07:00
Anders Kaseorg d01db0f5a0 popover_menus: Extract left_sidebar_navigation_area_popovers module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 11:58:29 -07:00
Anders Kaseorg dd6676f16e popover_menus: Extract user_topic_popover menus.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 11:58:29 -07:00
Anders Kaseorg 825fb8cef5 popover_menus: Extract topic_popover module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 11:58:29 -07:00
Anders Kaseorg 16e2a32c4c popover_menus: Extract message_actions_popover module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 11:58:29 -07:00
Anders Kaseorg 26e34a163e popover_menus: Extract scheduled_messages_popover module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 11:58:22 -07:00
Eeshan Garg f3465dea08 demo-orgs: Add UI for converting to permanent organization.
Adds warning banner to the organization settings overlay/tabs
for demo organizations. For owners, clicking on the link in the
banner opens a modal to convert the demo organization into a
permanent organization.

For admins that are not owners, clicking on the link will go
to the help center article on demo organizations.

Non-admin users will not see the warning banner.

The modal for converting a demo organization requires the owner
to have set their email address. Once the owner's email address
is set, then the organization can be made permanent by changing
the subdomain of the organization. The deletion date for the
demo organization will be removed as part of updating the
subdomain.

The organization owner must also have updated the organization
type to be any value other than unsepecified to convert the
demo organization to a permanent organization. The modal's
submit button will be disabled if that is the value for the
organization type in the form. The demo organizations created
in the dev environment now have their organization type set to
unspecified on creation.

This is a part of #19523.

Co-authored by: Lauryn Menard <lauryn@zulip.com>
2023-10-03 09:11:41 -07:00
Lauryn Menard 6ad3ec0891 check-schemas: Add list of deprecated events in OpenAPI documentation.
In commit 268f858f3, we removed the "realm_filters" event from the
schemas that we test in `zerver/lib/event_schemas.py`, but the event
is still documented (as deprecated) in the api/get-events doc.

Updates `tools/check_schemas` to not print a warning for an event
schema in the OpenAPI documentation if it's include in the list of
deprecated events list.
2023-10-03 09:05:22 -07:00
Anders Kaseorg 5efbece30a stream_color: Extract update_stream_color to new module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 08:15:21 -07:00
Anders Kaseorg 0efd9f7bdd settings_users: Extract user sort comparators to new user_sort module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 08:14:51 -07:00
Anders Kaseorg 30ae936793 settings_invites: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-03 08:14:11 -07:00
Aman Agrawal 75a1a74adf hello: Redesign page.
Fixes #24082
2023-10-02 22:30:03 -07:00
Tim Abbott 7a97ceab86 popovers: Extract add_stream_options_popover.js. 2023-10-02 19:17:08 -07:00
Anders Kaseorg 91e656398d settings_muted_users: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-02 19:15:03 -07:00
Anders Kaseorg c5c5b18ab4 settings_exports: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-02 17:49:36 -07:00
Anders Kaseorg e459b83c06 pm_list_dom: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-02 17:48:39 -07:00
Anders Kaseorg 819a91aec7 alert_words_ui: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-02 17:46:22 -07:00
Anders Kaseorg 457544e0d2 message_scroll: Move state flags to new module message_scroll_state.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-02 17:37:56 -07:00
Anders Kaseorg 919381e5e7 copied_tooltip: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-02 15:36:23 -07:00
Anders Kaseorg 60f324518b Revert "provision: enabling automatic activation of zulip-py3-venv at the lunch of shell only for dedicated containers."
This reverts commit ba4c45aa90.

The tests it used don’t make sense.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-01 12:10:56 -07:00
Mateusz Mandera 0e4532f8e7 semgrep: Add rule to specify tick kwarg on time_machine.travel() calls.
Letting the clock tick without a reason introduces the
possibility of nondeterministic test failures depending on the execution
time. The default value when not specified is tick=True, which makes it
easy to miss.
The rule doesn't prohibit setting tick=True, as perhaps there will be
tests wanting to use that feature on purpose, but such a test should
explicitly set it to make the intent clear.
2023-10-01 09:28:42 -07:00
Sudip Bose ba4c45aa90 provision: enabling automatic activation of zulip-py3-venv at the lunch of shell only for dedicated containers.
Refactor tools/lib/provision_inner to conditionally write activation commands to user's bash profile based on the host machine type. Automatic activation now skipped for native linux containers.
Fixes #15029
2023-10-01 08:43:14 -07:00
Hardik Dharmani bae28dfe64 message_list_hover: Extract module for hover/unhover logic.
Remove the various message-hover related functions from ui_init.js and
put them in a new module.
2023-09-28 18:07:26 -07:00
Anders Kaseorg f4768b0030 lint: Fix semgrep scanning of extensionless Python scripts.
Semgrep 0.118.0 changed the default of --scan-unknown-extensions to
false.  It also seems that it no longer respects --lang (or never
did), so rename the config file to reflect that it only includes
Python rules, to make it clear that additional languages will require
separate config files.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-28 14:19:50 -07:00
Aman Agrawal 820b564072 right_sidebar_ui: Rename to sidebar_ui. 2023-09-28 08:41:03 -07:00
Sahil Batra 1de3a9f4a3 bots: Extract code for showing integration url modal in new file.
This commit creates a new file integration_url_modal.js which
now contains the code for showing integration url modal.

Since show_generate_integration_url_modal is used in multiple
places, this change helps us in avoiding import cycles.
2023-09-26 09:15:20 -07:00
Sahil Batra f3c17b8d4f users: Extract code for showing deactivation modal in new file.
This commit creates a new file user_deactivation_ui.js which
now contains the code for showing deactivation modal for user
and bots.

Since confirm_deactivation and confirm_bot_deactivation functions
are used in multiple places, this change helps us in avoiding
import cycles.
2023-09-26 09:15:20 -07:00
Aman Agrawal e0ff125116 views: Write a common show and hide function.
Fixes #26688
2023-09-25 13:16:25 -07:00
Sahil Batra c7cbdd3741 custom_fields: Extract some code in custom_profile_fields_ui.js.
This commit moves initialize_custom_pronouns_type_fields,
initialize_custom_user_type_fields, initialize_custom_date_type_fields
and append_custom_profile_fields functions to the new file
custom_profile_fields_ui.js from settings_account.js since
they are used for both showing custom profile fields in
"Profile" settings panel and "Edit user" form shown in
user profile modal. This change also helps us in avoiding
import cycles.
2023-09-25 13:14:49 -07:00
Tim Abbott c379d0bad6 right_sidebar_ui: Extract new module.
This UI is not actually a popover, and so there is no need for it to
live in popovers.js.
2023-09-25 06:29:47 -07:00
Aman Agrawal bdf6b210b8 copied_tooltip: Remove tippyjs import cycle by extracting it.
Moving show_copied_confirmation to a new library breaks the
dependency cycle of popover_menus and tippyjs.
2023-09-24 15:16:27 -07:00
Anders Kaseorg d6884399b2 invite: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-22 17:10:51 -07:00
Anders Kaseorg 35568ec4d6 flatpickr: Convert module to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-22 17:08:17 -07:00
YashRE42 cb04ae1f95 search: Redesign search box.
Fixes #21798.
2023-09-21 08:58:31 -07:00
Anders Kaseorg acd0c5568a corporate: Remove Twitter third-party JavaScript and branding.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-21 08:53:37 -07:00
Daniil Fadeev d96110590b playground_links: Migrate popover to Tippy.
Fixes part of #23632.
2023-09-15 12:25:26 -07:00
Daniil Fadeev d89cd1e241 user_card_popover: Rename Handlebars templates.
List of renamed templates:
`user_info_popover_content` -> `user_card_popover_content`
`user_info_popover_manage_menu` -> `user_card_popover_manage_menu`
`user_info_popover_title` -> `user_card_popover_title`
2023-09-15 12:24:00 -07:00
Anders Kaseorg 28597365da python: Delete superfluous parens.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-13 13:40:19 -07:00
evykassirer a0df603aa7 puppeteer: Allow reruns in interactive mode when tests succeed.
We already allowed reruns for failing tests, and this adds
the ability to rerun tests that succeeded as well, which is
helpful for debugging flaky tests.
2023-09-13 12:46:22 -07:00
Anders Kaseorg 2665a3ce2b python: Elide unnecessary list wrappers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-13 12:41:23 -07:00
Daniil Fadeev c5f6c00a81 popovers: Extract user info popovers into their own module.
The intent behind this commit is to tidy up how we handle user info
popovers. The first step is to move everything related to them into
its own module. This commit should not have any functional changes.
2023-09-13 11:49:08 -07:00
Aman Agrawal 6ef0753a51 inbox: Add new narrow. 2023-09-12 09:20:33 -07:00
Anders Kaseorg 6988622fe8 ruff: Enable B023 Function definition does not bind loop variable.
Python’s loop scoping is misdesigned, resulting in a very common
gotcha for functions that close over loop variables [1].  The general
problem is so bad that even the Go developers plan to break
compatibility in order to fix the same design mistake in their
language [2].

Enable the Ruff rule function-uses-loop-variable (B023) [3], which
conservatively prohibits functions from binding loop variables at all.

[1] https://docs.python-guide.org/writing/gotchas/#late-binding-closures
[2] https://go.dev/s/loopvar-design
[3] https://beta.ruff.rs/docs/rules/function-uses-loop-variable/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-11 18:03:45 -07:00
Anders Kaseorg cf4791264c python: Replace functools.partial with type-safe returns.curry.partial.
The type annotation for functools.partial uses unchecked Any for all
the function parameters (both early and late).  returns.curry.partial
uses a mypy plugin to check the parameters safely.

https://returns.readthedocs.io/en/latest/pages/curry.html

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-11 18:03:45 -07:00
Alex Vandiver b94402152d models: Always search Messages with a realm_id or id limit.
Unless there is a limit on `id`, always provide a `realm_id` limit as
well.  We also notate which index is expected to be used in each
query.
2023-09-11 15:00:37 -07:00
Alex Vandiver 067de6f948 coverage: Skip zerver.lib.migrate coverage.
It is only covered when we run migration tests, which we are not
guaranteed to always be able to do.
2023-09-11 15:00:37 -07:00
Alex Vandiver d6745209f2 django: Use .exists() instead of .count() when possible. 2023-09-11 15:00:37 -07:00
Lalit Kumar Singh 5f74b9051e ts: Migrate `user_topics.js` to typescript. 2023-09-11 10:57:38 -07:00
Daniil Fadeev 7777c55b22 popovers: Extract `user_group_popover` into separate module.
This is a preparatory commit before we migrate `user_group_popover`
from Bootstrap to Tippy library.

The previous implementation was weirdly sharing the logic around
`current_message_info_popover_elem` with the user info popovers based
on a message; very likely an unfortunate latent bug caused by
copy/paste.

To address that, we need to add dedicated functions like
get_user_group_popover_items to avoid breaking keyboard navigation
with this extraction.
2023-09-11 10:26:50 -07:00
Lalit Kumar Singh c47a18eb2b ts: Migrate `realm_logo` to TypeScript. 2023-09-09 18:30:43 -07:00
Lalit Kumar Singh b8e0850aa1 ts: Migrate `realm_icon` to TypeScript. 2023-09-09 18:30:43 -07:00
Aman Agrawal 1305fe74b6 test-js-with-node: Remove deleted dropdown-list-widget module.
Fixes #25741
2023-09-09 18:22:25 -07:00
Anders Kaseorg 31b5879242 docs: Remove obsolete mentions of reStructuredText.
Commit b53e676860 (#19600) removed the
last of our reST.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-09 01:56:16 -07:00
Anders Kaseorg 964cc182a4 sync-translations: Normalize translations to NFC.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-08 17:51:47 -07:00
Lalit Kumar Singh affdffdda5 ts: Migrate `settings_emoji.js` to TypeScript. 2023-09-08 13:09:34 -07:00
Lalit Kumar Singh 5c2ba6c8fa ts: Migrate `avatar.js` module to TypeScript. 2023-09-08 13:09:12 -07:00
Zixuan James Li c336bf0398 api: Avoid programming errors due to nested Annotated types.
We want to reject ambiguous type annotations that set ApiParamConfig
inside a Union. If a parameter is Optional and has a default of None, we
prefer Annotated[Optional[T], ...] over Optional[Annotated[T, ...]].

This implements a check that detects Optional[Annotated[T, ...]] and
raise an assertion error if ApiParamConfig is in the annotation. It also
checks if the type annotation contains any ApiParamConfig objects that
are ignored, which can happen if the Annotated type is nested inside
another type like List, Union, etc.

Note that because
param: Annotated[Optional[T], ...] = None
and
param: Optional[Annotated[Optional[T], ...]] = None
are equivalent in runtime prior to Python 3.11, there is no way for us
to distinguish the two. So we cannot detect that in runtime.
See also: https://github.com/python/cpython/issues/90353
2023-09-08 08:20:17 -07:00
Zixuan James Li f4caf9dd79 api: Add new typed_endpoint decorators.
The goal of typed_endpoint is to replicate most features supported by
has_request_variables, and to improve on top of it. There are some
unresolved issues that we don't plan to work on currently. For example,
typed_endpoint does not support ignored_parameters_supported for 400
responses, and it does not run validators on path-only arguments.

Unlike has_request_variables, typed_endpoint supports error handling by
processing validation errors from Pydantic.

Most features supported by has_request_variables are supported by
typed_endpoint in various ways.

To define a function, use a syntax like this with Annotated if there is
any metadata you want to associate with a parameter, do note that
parameters that are not keyword-only are ignored from the request:
```
@typed_endpoint
def view(
    request: HttpRequest,
    user_profile: UserProfile,
    *,
    foo: Annotated[int, ApiParamConfig(path_only=True)],
    bar: Json[int],
    other: Annotated[
        Json[int],
        ApiParamConfig(
            whence="lorem",
            documentation_status=NTENTIONALLY_UNDOCUMENTED
        )
    ] = 10,
) -> HttpResponse:
    ....
```

There are also some shorthands for the commonly used annotated types,
which are encouraged when applicable for better readability and less
typing:
```
WebhookPayload = Annotated[Json[T], ApiParamConfig(argument_type_is_body=True)]
PathOnly = Annotated[T, ApiParamConfig(path_only=True)]
```

Then the view function above can be rewritten as:
```
@typed_endpoint
def view(
    request: HttpRequest,
    user_profile: UserProfile,
    *,
    foo: PathOnly[int],
    bar: Json[int],
    other: Annotated[
        Json[int],
        ApiParamConfig(
            whence="lorem",
            documentation_status=INTENTIONALLY_UNDOCUMENTED
        )
    ] = 10,
) -> HttpResponse:
    ....
```

There are some intentional restrictions:
- A single parameter cannot have more than one ApiParamConfig
- Path-only parameters cannot have default values
- argument_type_is_body is incompatible with whence
- Arguments of name "request", "user_profile", "args", and "kwargs" and
  etc. are ignored by typed_endpoint.
- positional-only arguments are not supported by typed_endpoint. Only
  keyword-only parameters are expected to be parsed from the request.
- Pydantic's strict mode is always enabled, because we don't want to
  coerce input parsed from JSON into other types unnecessarily.
- Using strict mode all the time also means that we should always use
  Json[int] instead of int, because it is only possible for the request
  to have data of type str, and a type annotation of int will always
  reject such data.

typed_endpoint's handling of ignored_parameters_unsupported is mostly
identical to that of has_request_variables.
2023-09-08 08:20:17 -07:00
evykassirer 3c16541eb4 recent: Rename recent_topics_util. 2023-09-08 07:36:33 -07:00
evykassirer f8db06569f recent: Rename recent_topics_ui. 2023-09-08 07:36:33 -07:00
Lalit Kumar Singh 9a6f7f0ead ts: Migrate `attachments_ui.js` to typescript. 2023-09-07 17:54:48 -07:00
Alex Vandiver 5d3ce8b2d4 puppet: Update dependencies. 2023-09-06 09:20:06 -07:00
Anders Kaseorg 81bd63cb46 ruff: Fix PIE808 Unnecessary `start` argument in `range`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-01 14:57:01 -07:00
Lalit Kumar Singh 4f3a971e64 ts: Migrate `channel.js` module to TypeScript. 2023-08-28 09:19:07 -07:00
Zixuan James Li a081428ad2 user_groups: Make locks required for updating user group memberships.
**Background**

User groups are expected to comply with the DAG constraint for the
many-to-many inter-group membership. The check for this constraint has
to be performed recursively so that we can find all direct and indirect
subgroups of the user group to be added.

This kind of check is vulnerable to phantom reads which is possible at
the default read committed isolation level because we cannot guarantee
that the check is still valid when we are adding the subgroups to the
user group.

**Solution**

To avoid having another transaction concurrently update one of the
to-be-subgroup after the recursive check is done, and before the subgroup
is added, we use SELECT FOR UPDATE to lock the user group rows.

The lock needs to be acquired before a group membership change is about
to occur before any check has been conducted.

Suppose that we are adding subgroup B to supergroup A, the locking protocol
is specified as follows:

1. Acquire a lock for B and all its direct and indirect subgroups.
2. Acquire a lock for A.

For the removal of user groups, we acquire a lock for the user group to
be removed with all its direct and indirect subgroups. This is the special
case A=B, which is still complaint with the protocol.

**Error handling**

We currently rely on Postgres' deadlock detection to abort transactions
and show an error for the users. In the future, we might need some
recovery mechanism or at least better error handling.

**Notes**

An important note is that we need to reuse the recursive CTE query that
finds the direct and indirect subgroups when applying the lock on the
rows. And the lock needs to be acquired the same way for the addition and
removal of direct subgroups.

User membership change (as opposed to user group membership) is not
affected. Read-only queries aren't either. The locks only protect
critical regions where the user group dependency graph might violate
the DAG constraint, where users are not participating.

**Testing**

We implement a transaction test case targeting some typical scenarios
when an internal server error is expected to happen (this means that the
user group view makes the correct decision to abort the transaction when
something goes wrong with locks).

To achieve this, we add a development view intended only for unit tests.
It has a global BARRIER that can be shared across threads, so that we
can synchronize them to consistently reproduce certain potential race
conditions prevented by the database locks.

The transaction test case lanuches pairs of threads initiating possibly
conflicting requests at the same time. The tests are set up such that exactly N
of them are expected to succeed with a certain error message (while we don't
know each one).

**Security notes**

get_recursive_subgroups_for_groups will no longer fetch user groups from
other realms. As a result, trying to add/remove a subgroup from another
realm results in a UserGroup not found error response.

We also implement subgroup-specific checks in has_user_group_access to
keep permission managing in a single place. Do note that the API
currently don't have a way to violate that check because we are only
checking the realm ID now.
2023-08-24 17:21:08 -07:00
evykassirer a8d2481d77 generate emoji: Use hex_codepoint_to_emoji.
This new util function helps make the
code here a little more readable.
2023-08-23 16:18:15 -07:00
evykassirer 0289beb784 emoji: Match emoji sequences in markdown.
Fixes #11767.

Previously multi-character emoji sequences weren't matched in the
emoji regex, so we'd convert the characters to separate images,
breaking the intended display.

This change allows us to match the full emoji sequence, and
therefore show the correct image.
2023-08-23 16:18:15 -07:00
evykassirer 78f0dca269 narrow: Save blue box position and restore on navigation.
This saves the blue box position as state on the location
in brower history, with `history.replaceState`.

The position is restored when a narrow is activated.

Fixes #20066.
2023-08-23 13:40:26 -07:00
shu.chen 321776ac49
provision: Add provision support for Fedora 38.
Tweaked provision script to run successfully in Fedora 38 and
included a script to build the groonga libs from source because
the packages in Fedora repos are outdated.

There is a major version jump from the last supported version (F34)
which is EOL so references and support for older versions were
removed.

Fixes: #20635
2023-08-22 11:26:29 -07:00
Anders Kaseorg 328cdde243 documentation: Remove duplicate heading IDs on server side.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-21 17:25:14 -07:00
Anders Kaseorg c5dce72ff4 install-transifex-cli: Upgrade Transifex CLI from 1.6.8 to 1.6.10.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 18:16:47 -07:00
Anders Kaseorg 041dcdfbad tools: Use subprocess.check_call where appropriate.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Anders Kaseorg c43629a222 ruff: Fix PLW1510 `subprocess.run` without explicit `check` argument.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
Junyao Chen 983c6b74ab ts: Migrate `user_group_create_members_data.js` to TypeScript. 2023-08-17 11:38:13 -07:00
Junyao Chen 6757c5a9f8 stats: Install @types/plotly.js for type check.
This is a prep commit for typescript migration for `stats.js`,
which used a third party module `plotly.js` that doesn't come
with types.
2023-08-17 11:35:00 -07:00
Sahil Batra ba00907946 bots: Re-parent bot while reactivating if original owner is deactivated.
Since the original owner is deactivated, we change the owner to the
admin who is reactivating the bot.
2023-08-16 15:37:37 -07:00
Junyao Chen 446aea41b3 ts: Migrate `settings_ui.js` to TypeScript. 2023-08-15 17:17:40 -07:00
Lalit Kumar Singh a0cb034731 ts: Migrate `spectators.js` to typescript. 2023-08-15 09:58:17 -07:00
Lalit Kumar Singh 9b01872e8c ts: Migrate `hash_util.js` module to TypeScript. 2023-08-15 09:58:17 -07:00
Zixuan James Li c9a299a8f8 validators: Split out test_validators. 2023-08-11 16:43:12 -07:00
Steve Howell 27b5b0dc26 tests: Defer writing coverage report.
It's relatively rare that you want to read the coverage report
if the coverage tests pass. This allows devs to get quicker
feedback. Particularly on tricky rebases this can be a real
time saver. It takes about a minute to write the coverage
report on my box.
2023-08-10 14:02:17 -07:00
Anders Kaseorg 4fe93d2523 generate_emoji_names: Fix use of stale aliases variable.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-09 21:51:15 -07:00
Alex Vandiver def6bf17eb tests: Split out missed_message email tests. 2023-08-09 15:49:49 -07:00
Anders Kaseorg 562a79ab76 ruff: Fix PERF401 Use a list comprehension to create a transformed list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg 0b95d83f09 ruff: Fix PERF402 Use `list` or `list.copy` to create a copy of a list.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg c4748298bb ruff: Fix PERF102 Using only the keys/values of a dict.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:23:55 -07:00
Anders Kaseorg 55aa29bef4 ruff: Fix FLY002 Consider f"…" instead of string join.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-07 17:12:41 -07:00
evykassirer 0930e5a78f emoji: Run generate_emoji_names with up-to-date data. 2023-08-07 15:01:36 -07:00
Anders Kaseorg bfd52ccd68 mailmap: Put sorted list back in order.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-02 17:19:05 -07:00
Anders Kaseorg e932e2ce52 ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-02 15:58:55 -07:00
Anders Kaseorg 5ccb408f19 build-pgroonga: Upgrade PGroonga from 3.0.3 to 3.1.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-25 17:33:42 -07:00
Zixuan James Li 000761ac0c realm_playgrounds: Replace url_prefix with url_template.
Dropping support for url_prefix for RealmPlayground, the server now uses
url_template instead only for playground creation, retrieval and audit
logging upon removal.

This does the necessary handling so that url_template is expanded with
the extracted code.

Fixes #25723.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-07-24 17:40:59 -07:00
Lalit 829caae189 ts: Migrate `realm_playground` to typescript. 2023-07-24 12:49:53 -07:00
Anders Kaseorg 2ae285af7c ruff: Fix PLR1714 Consider merging multiple comparisons.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-23 15:21:33 -07:00
Anders Kaseorg 3b09197fdf ruff: Fix RUF015 Prefer `next(...)` over single element slice.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-23 15:20:53 -07:00
Anders Kaseorg 4e645c8ff9 tests: Remove compatibility code for Python < 3.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-23 15:20:53 -07:00
Anders Kaseorg bca5564c1e ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-23 15:18:18 -07:00
Anders Kaseorg 7746e11486 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-21 15:58:42 -07:00
Anders Kaseorg 3b12ae18c8 install-transifex-cli: Upgrade transifex-cli from 1.6.7 to 1.6.8.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-21 15:58:42 -07:00
Anders Kaseorg b9b4711977 install-shfmt: Upgrade shfmt from 3.6.0 to 3.7.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-21 15:58:42 -07:00
Anders Kaseorg b285813beb error_notify: Remove custom email error reporting handler.
Restore the default django.utils.log.AdminEmailHandler when
ERROR_REPORTING is enabled.  Those with more sophisticated needs can
turn it off and use Sentry or a Sentry-compatible system.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-20 11:00:09 -07:00
Anders Kaseorg 143baa4243 python: Convert translated positional {} fields to {named} fields.
Translators benefit from the extra information in the field names, and
need the reordering freedom that isn’t available with multiple
positional fields.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-18 15:19:07 -07:00
Prakhar Pratyush 32b201d5e0 user_topics: Rename 'muted_topics_ui' to 'user_topics_ui'.
This commit renames the file 'muted_topics_ui.js' to
'user_topics_ui.js', as we also support visibility policies
other than MUTED.
2023-07-17 17:39:17 -07:00
AcKindle3 1aeab40779 ts: Convert `stream_settings_containers.js` to TypeScript.
Added assertion to enforce `sub` is not `undefined` and thus type
safe for the rest.
Added type annotation to function parameter and return value.
2023-07-17 17:24:15 -07:00
Lalit c0c4c2e1e7 ts: Migrate `message_overlay_ui` to TypeScript. 2023-07-17 16:24:16 -07:00
Lalit b51bdd33bb ts: Migrate `list_widget` to TypeScript. 2023-07-17 11:31:51 -07:00
Zixuan James Li 604ffebf5f tools: Make screenshot tool work with the modern web client.
This fixes some deprecation use of the APIs, including using "new"
instead of "true" for the "headless" flag when launching puppeteer and
using $$ instead of $x when using XPath selectors.

We also use {waitUntil: "networkidle2"} to fix the issue of webhook bot
avatar not being loaded consistently when generating the screenshots
(this happened to webhooks like Slack and Harbor).

Positioning of the clip area is adjusted to take the new grid layout
into account.
2023-07-17 09:23:01 -07:00
Zixuan James Li d1455d81f2 integrations: Extend screenshot tool for more flexibility.
This allows the user to skip generating screenshots up to certain
integration when working through generating screenshots for all
integrations.

This also allow the user to select a number of integrations to
selectively generate screenshots for.
2023-07-17 09:23:01 -07:00
xoldyckk a062a82f06 ts: Migrate setup.js to typescript.
Also added global type definition for `get_offset_to_window`
helper function.
2023-07-10 13:49:26 -07:00
Lalit ae6063807b ts: Convert `poll_data` to typescript.
Used zod schemas to validate inbound data types and removed some over
defensive code.
2023-07-06 16:21:52 -07:00
Alex Vandiver 23fd05494d release: Release 8.0-prereleases from `main`. 2023-07-06 16:16:11 -07:00
Alex Vandiver c1008b5b12 release: Link to the changelog in the Github Release.
Release notes in GitHub have significant-newlines, which makes our
canonical `changelog.md` content look not-great -- and the relative
links are also broken, as they have the wrong relative path.

Switch to linking to the canonical release notes on ReadTheDocs, now
that those have proper anchors.
2023-07-06 16:16:11 -07:00
Alex Vandiver 9bd340957f changelog: Retitle sections, to allow section anchors. 2023-07-06 16:16:11 -07:00
Zixuan James Li eebe46ad1c test_classes: Do not necessary wrap test cases in a transaction.
By relocating helper methods into a mixin class, we can be more flexible
with managing transactions in test cases, without always forcing the
django.test.TestCase behavior of always putting the test case into an
atomic transaction.

We include a check for side effects in ZulipTransactionTestCase. It only
checks for the set of row ids in all tables before and after each test.
It is not a comprehensive check for side effects, but should be
sufficient for the basics without much performance overhead.
2023-07-06 11:44:50 -07:00
Satyam Bansal 605e9065a2 tools: Copy "assets" subdirectory of bots to "static/generated/bots/".
Previously, the "assets" subdirectory of various bots was not
being copied to the "static/generated/bots/" folder, which
resulted in the documentation on some pages not loading at all.
2023-07-05 14:59:06 -07:00
Akshat e7f4700d76 test-backend: Enable single-threaded profiling when thread count is unspecified.
If the number of threads is not specified
while profiling then use a single thread. This is
because profiling across multiple threads (earlier
default behaviour) may obscure the accurate
measurement of which functions are the most costly
due to thread blocking.

Signed-off-by: Akshat <akshat25iiit@gmail.com>
2023-07-02 16:18:59 -07:00
Anders Kaseorg 407c16fc77 eslint: Expand no-unused-vars check to all function parameters.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-02 16:16:38 -07:00
Alex Vandiver dc2726c814 pgroonga: Remove now-unnecessary 'GRANT USAGE' statement.
This was only necessary for PGroonga 1.x, and the `pgroonga` schema
will most likely be removed at some point inthe future, which will
make this statement error out.

Drop the unnecessary statement.
2023-06-23 14:40:27 -07:00
Lalit 13187ff8f6 ts: Convert `web/src/billing` module to TypeScript.
Converted all files inside `web/src/billing` to TypeScript.
2023-06-23 11:42:29 -07:00
Anders Kaseorg 639efc5ce1 docs: Update Transifex URLs.
https://help.transifex.com/en/articles/7171815-web-application-s-domain-change

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-12 17:38:41 -07:00
Zixuan James Li 4a18552ff8 docs: Document and advocate systemd for WSL 2.
systemd makes it a lot convenient to run services required for Zulip's
development environment on WSL 2. Once enabled, `tools/start_service`
becomes unnecessary.

More info:
https://devblogs.microsoft.com/commandline/systemd-support-is-now-available-in-wsl/

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-06-12 10:58:11 -07:00
Anders Kaseorg 7e811ea1bd custom_check: Remove checks for deleted ui_report aliases.
They were deleted in commit 2d52463b61.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-06-09 16:46:57 -07:00
Lalit 5744ed32e2 tippyjs: Extract compose_tooltips.js module. 2023-06-08 16:20:59 -07:00
Tim Abbott e7c012c850 tippyjs: Extract message_list_tooltips module. 2023-06-08 16:20:59 -07:00
Zixuan James Li 119cf9100c semgrep: Check if a batched migration is made atomic.
This raises an error when we forget to make migration non-atomic when
doing it in batches. BATCH_SIZE seems to be a good indicator for when
we do batching with few early exceptions.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-06-07 16:49:11 -07:00
evykassirer aa270bcef0 search: Remove support for experimental search pills.
This in-progress feature was started in 2018 and hasn't
been worked on much since. It's already in a broken state,
which makes it hard to iterate on the existing search bar
since it's hard to know how those changes will affect search
pills.

We do still want to add search pills eventually, and when
we work on that, we can refer to this diff to readd the
changes back.
2023-06-06 18:36:02 -07:00
Tim Abbott 713a64d415 message_scroll: Extract module message_feed_top_notices. 2023-06-05 21:57:35 -07:00
Anders Kaseorg b5e5728112 coverage: Clean up coverage configuration.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-31 13:53:04 -07:00
Lalit 38dd1de5b2 ts: Convert `compose_textarea.js` to TypeScript.
Added type definitions for `jquery-caret-plugin` third party package and
converted `compose_textarea.js` to TypeScript.
2023-05-25 17:33:18 -07:00
Anders Kaseorg 13d92acdb3 run-dev: Clarify magic number for errno.EADDRINUSE.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-25 15:49:06 -07:00
Anders Kaseorg 8a128ba81e commit-msg: Don’t change the current directory for non-Vagrant.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-23 12:24:44 -07:00
Greg Price 2baa4fc0ca total-contributions: Add zulip-flutter to the list of repos. 2023-05-22 16:04:26 -07:00
Greg Price c4cc27cd20 total-contributions: Fix bug that replaced a repo's whole history with nothing.
When using a start date before the first commit to a repo, we should
include the repo's entire history (up to the end date) in our totals.

Instead, we were using a range like "..{upper_version}", which in Git
revision-range syntax means the start of the range is HEAD -- so the
range was empty.

Fix that by leaving out the ".." when we want no left endpoint.
2023-05-22 16:04:26 -07:00
Greg Price 0c8fef2fc8 total-contributions: Factor out logic common to all repos. 2023-05-22 16:04:26 -07:00
Lalit 7346ce98f3 ts: Convert `emojisets` to TypeScript.
Declared types for '*.png' modules and '!style-loader?*' modules in
`assets.d.ts` and used them in `emojisets.ts`.
2023-05-22 09:02:42 -07:00
Alex Vandiver c978bfaa32 models: Add a unique index on UserProfile.api_key.
This prevents `get_user_profile_by_api_key` from doing a sequential
scan.

Doing this requires moving the generation of initial api_key values
into the column definition, so that even bare calls to
`UserProfile.objects.create` (e.g. from tests) call appropriately
generate a random initial value.
2023-05-19 11:11:04 -07:00
Anders Kaseorg 12310189ed install: Support Debian 12.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Anders Kaseorg 480a2e2d64 postgresql-init-dev-db: Work around PGroonga WAL issue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-18 11:52:22 -07:00
Lalit a2db6df826 ts: Convert `deprecated_feature_notice` module to TypeScript. 2023-05-18 08:52:26 -07:00
Lalit a3f46957bc ts: Convert `confirm_dialog` module to TypeScript. 2023-05-18 08:44:30 -07:00
Lauryn Menard 4fcb243769 api-docs: Standardize on lowercase true, false and null.
Creates a custom linter rule for `zerver/openapi/zulip.yaml` to
only allow lowercase versions of "true", "false" and "null".

Updates existing documentation for new rules.
2023-05-16 12:31:03 -07:00
Daniil Fadeev f2e627ba51 drafts: Move code handling keyboard navigation to a separate module.
The keyboard navigation that used to only work in drafts can now be
reused. This commit has moved the related functions to a separate
module.
2023-05-15 14:46:21 -07:00
Lalit cab3a992c0 refactor: Extract `message_feed_loading` module from `message_scroll`.
This new module allows us to remove dependency on `message_scroll` in `fetch_status`
and hence allowing us to migrate it to TypeScript.
2023-05-14 11:24:31 -07:00
Anders Kaseorg 54f90e41c0 eslint: Fix unicorn/prefer-string-replace-all.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-12 11:12:20 -07:00
Anders Kaseorg 5f1a07aaad dev-vagrant-docker: Upgrade docker-systemctl-replacement.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-11 14:50:53 -07:00
Sahil Batra ddb0bb58ed tests: Add tests to update visibility policy when target topic is empty.
This commit adds a new test to check how the visibility policy updates
when moving messages to a topic that didn't exist previously.

This test also helps us adding coverage for the code which just
skips setting visibility_policy if there is no need to update the
value because both previous and new value of visibility policy
is INHERIT. The "actions/message_edit.py" file has 100% coverage
now and thus is removed from "not_yet_fully_covered" list.
2023-05-11 12:13:50 -07:00
Sahil Batra 9fa67f0fa9 tests: Add coverage to actions/create_realm.py.
This commit adds test coverage to actions/create_realm.py.
The file is also removed from not_yet_fully_covered list
since it has 100% coverage now.
2023-05-11 12:13:50 -07:00
Lauryn Menard 1d209220dd tests: Add coverage for error when editing a sent scheduled message.
Adds test coverage for the error sent for editing a scheduled
message that was successfully sent.

`zerver/actions/scheduled_messages.py` now has 100% test coverage
again.
2023-05-11 10:52:01 -07:00
Lalit ebaabc6b68 ts: Migrate `feedback_widget` to TypeScript.
This commit migrates `feedback_widget` module to TypeScript.
2023-05-10 14:31:42 -07:00
Anders Kaseorg 3bc1acba4a install: Use set -x when installing node, shellcheck, shfmt, tx.
This makes it clearer which step failed if there’s an error.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-10 12:15:34 -07:00
Aman Agrawal a196b949f7 dropdown_widget: Implement dropdown widget using tippy.
This will soon replace DropdownListWidget.
2023-05-09 14:11:26 -07:00
Lauryn Menard df17a1bf28 test-events: Test do_delete_message with no messages specified.
Test coverage for `zerver/actions/message_delete.py`.

Both callers of this function would already return if there were
no Messages specified to delete, which is why existing tests did
not cover this.
2023-05-09 09:58:33 -07:00
Tim Abbott cce0c57a04 test-backend: Actions files should target full coverage.
We neglected to add this pattern when we split actions.py into a
directory.
2023-05-09 08:09:17 -07:00
Alex Vandiver dffbd91452 ci: Test that the installed PostgreSQL was what was asked for. 2023-05-05 13:35:32 -07:00
Alex Vandiver 787c74d0b8 ci: Reduce number of places that hardcode Ubuntu and PostgreSQL versions. 2023-05-05 13:35:32 -07:00
Alex Vandiver f32e6543be ci: Install PostgreSQL 12 on focal.
The previous commit removed support for PostgreSQL 11, and PostgreSQL
10 support was removed in 11a86ec328.
2023-05-05 13:35:32 -07:00
Anders Kaseorg 8f27a6fb73 provision: Disable compilemessages searching for locale directories.
We explicitly configure LOCALE_PATHS, so we can safely disable this
search of the entire tree for other locale directories.

https://github.com/django/django/blob/4.1/django/core/management/commands/compilemessages.py#L92-L100

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-03 17:20:16 -07:00
evykassirer 9ea6eca253 compose: Move cursor to end of topic input after stream selection.
Fixes #25321.
2023-05-02 12:49:31 -07:00
Alex Vandiver 47e0a89739 webpack: Silence out-of-date warnings from caniuse browserslist. 2023-05-02 09:12:22 -07:00
Aman Agrawal f40855bad2 reminder: Remove feature from zulip.
This is being removed to make the code simpler. We have plans
to add it as a feature in the future , but it will most likely
not use the same code.
2023-04-28 17:25:00 -07:00
Lalit c7b8f0658b ts: Migrate `scroll_util.js` to TypeScript.
This commit migrates `scroll_util.js` to typescript. I made a helper
type `JQueryOrZJQuery` for the argument of `get_scroll_element` instead
of adding a `__zjquery` property to the global JQuery object because it
isn't being used anywhere outside of this function and hence it makes
sense to have a little helper type than to add the property to the global
object.
2023-04-27 09:00:13 -07:00
AcKindle3 de420f68a4 ts migration: Convert `dark_theme.js` to `dark_theme.ts`.
Add `void` type annotation to the return type of functions. The
rest of the file is not modified.
2023-04-26 15:40:23 -07:00
Lalit b9223088f3 refactor: Move simplebar helper functions from `ui` to `scroll_util`.
This commit eliminates the `ui.js` module from the codebase by moving
all simplebar helper functions to the `scroll_util` module.
2023-04-26 12:59:08 -07:00
Lalit e132a68193 refactor: Create `compose_textarea` and extracted related handlers from `ui.js`.
Created the new `compose_textarea` and moved event handlers for `#compose_textarea`
from `ui` to this new module so that it is now responsible for initializing event handlers
for compose_textarea instead of `ui` module.
2023-04-26 12:59:08 -07:00
sbansal1999 aad2f7c7c5 compose: Remove empty message warning border after closing.
Adding this made the line coverage of
"web/tests/lib/zjquery_element.js" equal to 100%, hence it
was required to remove it from the exempt files list.
2023-04-26 10:58:08 -07:00
Anders Kaseorg b8023da17a dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 22:18:48 -07:00
Anders Kaseorg 03b3c8522d requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 21:20:33 -07:00
Anders Kaseorg 9db3451333 Remove statsd support.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-25 19:58:16 -07:00
Prakhar Pratyush a079bb811b user_topics: Rename 'settings_muted_topics.js'.
This commit renames 'settings_muted_topics.js'
to 'settings_user_topics.js' because the file now
supports the settings for topics with any visibility_policy,
not just MUTED.

It also renames the corresponding test file.
2023-04-25 12:46:05 -07:00
AcKindle3 70b8eb3423 TS migration: Convert `localstorage.js` to `localstorage.ts`.
Added function parameter types, return type, and types of local
varaibles. Added a `null` check for `raw_data` before `JSON.parse`.
Created a type `FormData` and an export type `LocalStorage` to
imporve conciseness and clearity.

Type `LocalStorage` is exported because it might be used in other
files based on an observation that many files have imported `localstorage`.
2023-04-24 11:59:02 -07:00
evykassirer 0c706aeafc compose: Extract a compose_recipient module.
This helps reduce the amount of import cycles we have in the compose
code path following the migration to a fancier stream input.

`compose_closed_ui.initialize()` was moved further down in the
initialization order because it relies on the dropdown widget
to be defined.
2023-04-20 16:09:55 -07:00
Zixuan James Li 6f42542320 migrations: Remove filter_format_validator.
This removes the validator argument for 0423_realmfilter_url_template,
which do not really alter the database schema. It otherwise fails
the migration because the filter_format_validator function is removed.

Migration 0094_realm_filter_url_validator is modified because we can no
longer refer to filter_format_validator.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
Zixuan James Li 268f858f39 linkifier: Support URL templates for linkifiers.
This swaps out url_format_string from all of our APIs and replaces it
with url_template. Note that the documentation changes in the following
commits  will be squashed with this commit.

We change the "url_format" key to "url_template" for the
realm_linkifiers events in event_schema, along with updating
LinkifierDict. "url_template" is the name chosen to normalize
mixed usages of "url_format_string" and "url_format" throughout
the backend.

The markdown processor is updated to stop handling the format string
interpolation and delegate the task template expansion to the uri_template
library instead.

This change affects many test cases. We mostly just replace "%(name)s"
with "{name}", "url_format_string" with "url_template" to make sure that
they still pass. There are some test cases dedicated for testing "%"
escaping, which aren't relevant anymore and are subject to removal.
But for now we keep most of them as-is, and make sure that "%" is always
escaped since we do not use it for variable substitution any more.

Since url_format_string is not populated anymore, a migration is created
to remove this field entirely, and make url_template non-nullable since
we will always populate it. Note that it is possible to have
url_template being null after migration 0422 and before 0424, but
in practice, url_template will not be None after backfilling and the
backend now is always setting url_template.

With the removal of url_format_string, RealmFilter model will now be cleaned
with URL template checks, and the old checks for escapes are removed.

We also modified RealmFilter.clean to skip the validation when the
url_template is invalid. This avoids raising mulitple ValidationError's
when calling full_clean on a linkifier. But we might eventually want to
have a more centric approach to data validation instead of having
the same validation in both the clean method and the validator.

Fixes #23124.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
Zixuan James Li a19c80df43 linkifiers: Add url_template field.
This will later be used to expand matching linkifier patterns.
Making it nullable for now, but we will make it required in
the APIs.

As a part of this transition, we temporarily make url_format_string
nullable as well, which will be later removed. This allows us to
switch to populating url_template without caring about passing
url_format_string.

Note that the validators are imported in the migration because Django
otherwise diffs it and considers the schema to be different, generating
a migration, failing the "tools/test-migrations" test.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
Aman Agrawal 043d54d170 scheduled_messages: Add overlay to display and edit them.
Fixes #20971
2023-04-17 16:59:25 -07:00
Aman Agrawal c0ef1c360a message_send: Edit scheduled message if its ID is present.
If the ID of the scheduled message is passed by the client, we
edit the existing scheduled message instead of creating a new one.

However, this will soon be moved into its own API endpoint.
2023-04-14 17:38:37 -07:00
Tim Abbott 1d2cf734f3 lint: Limit JS assert rule to web/tests. 2023-04-14 12:26:33 -07:00
Alex Vandiver e536a14b61 report_error: Remove API endpoint for client error reporting. 2023-04-13 14:59:58 -07:00
Anders Kaseorg b3a185103d ruff: Enable new lints INT, PYI.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-12 17:32:38 -07:00
Hardik Dharmani 46cabdd59e user_topics: Use new API endpoint to update topics_visibility_policy.
Removed functions mute_topic and unmute_topic in muted_topics_ui.js
which were using old API endpoints to mute/unmute topics.

In user_topics.js added new function set_user_topic_visibility_policy
which uses new API endpoint `/user_topics` to update the topic's
visibility_policy to visibility_policy passed to
set_user_topic_visibility_policy function as a parameter.

In functions toggle_topic_mute and mute_or_unmute_topic, replaced
the calls of mute_topic and unmute_topic with the new
user_topics.set_user_topic_visibility_policy function.

Added "web/src/user_topics.js" in EXEMPT_FILES in test-js-with-node
as adding function `set_user_topic_visibility_policy` resulted in
user_topics.js losing 100% test coverage.

Fixes #24244
2023-04-11 09:51:01 -07:00
Sahil Batra 9278a219ea user_groups: Refactor get_realm_user_groups_for_dropdown_list_widget.
This commit refactors get_realm_user_groups_for_dropdown_list_widget
function to use "group_permission_settings" module instead of
passing variables like "allow_internet_group", etc. individually
for each settings.

We lose some test coverage due to this commit, but that would be
fixed when we add some more group-based settings and allow groups
other than system groups in the group based settings.
2023-04-11 09:02:09 -07:00
Sahil Batra a5cbf95c12 settings: Add group_permission_settings.ts.
This commit adds a new module group_permission_settings.ts
which is used to define configuration options like
allow_owners_group, allow_nobody_group, etc. for group-based
settings similar to GroupPermissionSetting dataclass we have
in backend.
2023-04-11 09:02:09 -07:00
Aman Agrawal 8c744752ed stream_privacy: Use new lock and hashtag icons.
This commit doesn't modify the lock and hashtag icon in settings.
2023-04-10 23:41:30 -07:00
AcKindle3 ac4ec8888a ts migration: Convert `favicon.js` to `favicon.ts`.
Added type annotations to function parameters, function return
values and local variables. Added neccessary `if` statements to
enforce the objects having proper type before executing the later
operations.

An error type check in the catch block in `update_favicon`
function is added because the statement in the catch block requires
error to be a type `Error` in order to access `error.stack`.
2023-04-06 13:56:59 -07:00
Daniil Fadeev 2f203f4de1 emails: Inline CSS in emails in build_email.
Previously, we had an architecture where CSS inlining for emails was
done at provision time in inline_email_css.py. This was necessary
because the library we were using for this, Premailer, was extremely
slow, and doing the inlining for every outgoing email would have been
prohibitively expensive.

Now that we've migrated to a more modern library that inlines the
small amount of CSS we have into emails nearly instantly, we are able
to remove the complex architecture built to work around Premailer
being slow and just do the CSS inlining as the final step in sending
each individual email.

This has several significant benefits:

* Removes a fiddly provisioning step that made the edit/refresh cycle
  for modifying email templates confusing; there's no longer a CSS
  inlining step that, if you forget to do it, results in your testing a
  stale variant of the email templates.
* Fixes internationalization problems related to translators working
  with pre-CSS-inlined emails, and then Django trying to apply the
  translators to the post-CSS-inlined version.
* Makes the send_custom_email pipeline simpler and easier to improve.

Signed-off-by: Daniil Fadeev <fadeevd@zulip.com>
2023-04-05 12:22:29 -07:00
Anders Kaseorg 620b1aa1bd Revert "ts: Convert css_variables.js to TypeScript."
This reverts commit 5f4bdfe6bc (#24630).

This was causing Webpack to trigger type checking with the TypeScript
compiler, which is very expensive.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-04 10:54:01 -07:00
Anders Kaseorg 0d4dcc8583 install-transifex-cli: Upgrade Transifex CLI from 1.6.5 to 1.6.7.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-04-04 00:16:04 -07:00
Daniil Fadeev 5c6f842b28 emails: Replace Premailer with CSS-inline.
Primary goal of library replacement is improving execution speed.
This commit should not affect the functionality of the system
or make any changes to it.
2023-04-03 15:15:05 -07:00
Hardik Dharmani ebc07c0dd7 ts: Covert `dialog_widget.js` to typescript.
Created `WidgetConfig` type which has all the parameter that can be
passed launch(). Except html_heading, html_body, and on_click all
other fields are optional. Therefore, removed checking of undefined
mandatory fields as it will be done by typescript.

Co-authored-by: Hardik Dharmani <Ddharmani99@gmail.com>
2023-04-01 11:54:53 -07:00
Lalit fcbc6dcae4 ts: Convert `overlays.js` to TypeScript.
Refactored some code as well to avoid unneccesaary `undefined`
checks in `overlays`. To be exact created an action_overlay
object of type `Overlay` such that the `reset_state` is just
`action_overlay = undefined` and having attributes like `$element`
and `close_handler`.

This would ensure that if you have an `active_overlay`, there will
be a registered `close handler` attached to it without needing to
write additional checks for `close_handler` just to satisfy type
checker.
2023-04-01 11:54:53 -07:00
human 0de0743e91
ts: Migrate debug.js to TypeScript.
TypeScript doesn't understand Array.prototype.slice.call; rather than dealing with the
type-checking issues around it, we remove it; it was only necessary for ancient browsers where
NodeList isn’t directly iterable.
2023-04-01 11:50:34 -07:00
Adarsh Tiwari eed6f67fca
ts: Migrate scroll_bar.js to typescript.
Remove the winjs specific `getScrollbarWidth` property while we're at it; 
WinJS isn't relevant anymore.

Reorders methods due to TypeScript files enforcing define-before-use when
this commit was drafted.
2023-04-01 11:43:58 -07:00
Adarsh Tiwari 78fdb6135a
ts: Migrate gear_menu.js to TypeScript.
Includes adding various missing values to page_params.
2023-03-29 09:36:52 -07:00
Sahil Batra 1874491178 templates: Extract realm creation form html in new template file.
This commit extracts realm creation form html in a new file
realm_creation_form.html as we are reworking the organization
flow as per #24307 and this change would allow us to avoid
duplicating the code.
2023-03-27 15:44:42 -07:00
Alex Vandiver 93bfd3fb5f sentry: Provide a server-side tunnel.
Some well-intentioned adblockers also block Sentry client-side error
reporting.  Provide an endpoint on the Zulip server which forwards to
the Sentry server, so that these requests are not blocked.
2023-03-24 15:16:38 -07:00
Alex Vandiver 164d58bec9 web: Save a 301 redirect from /help to /help/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 8da1c51a06 web: Save a 301 redirect from /security to /security/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 3e307781dd web: Save a 301 redirect from /for/x to /for/x/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 4b21852e72 web: Save a 301 redirect from /why-zulip to /why-zulip/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 14d19c00d7 web: Save a 301 redirect from /features to /features/. 2023-03-24 14:51:01 -07:00
Alex Vandiver b5731c55b7 web: Save two 301 redirects from /privacy to /privacy/ to /policies/privacy. 2023-03-24 14:51:01 -07:00
Alex Vandiver ce847fd09c web: Save two 301 redirects from /terms to /terms/ to /policies/terms. 2023-03-24 14:51:01 -07:00
Alex Vandiver d9daeb13e0 web: Save a needless 301 redirect from /apps to /apps/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 61adc9cfcc web: Save a needless 301 redirect from /team to /team/. 2023-03-24 14:51:01 -07:00
Alex Vandiver c686c5ed0f web: Save a needless 301 redirect from /plans to /plans/. 2023-03-24 14:51:01 -07:00
Alex Vandiver bf06d493ee web: Save a needless 301 redirect from /history to /history/. 2023-03-24 14:51:01 -07:00
Alex Vandiver eb81e62eaa web: Save a needless 301 redirect from /hello to /hello/. 2023-03-24 14:51:01 -07:00
Alex Vandiver a1570ff3dd web: Save a needless 301 redirect from /integrations to /integrations/. 2023-03-24 14:51:01 -07:00
Alex Vandiver 7448ab6234 bootstrap-aws-installer: Update the github.com SSH key.
Per https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/.
2023-03-24 12:01:55 -07:00
Anders Kaseorg 870238206c create-test-api-docs: Remove.
This tool plays fast and loose with HTML generation, and nobody seems
to use it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-24 11:55:12 -07:00
evykassirer 12a61fcec6 emoji: Remove parens and apostrophes from emoji names.
This was breaking rendering of the emoji with these names.
More context on CZO: https://chat.zulip.org/#narrow/stream/9-issues/topic/issue.20with.20built-in.20emoji.20name/near/1517735
2023-03-23 15:58:43 -07:00
Anders Kaseorg afa218fa2a semgrep: Detect some unsafe uses of markupsafe.Markup.
Use the built-in HTML escaping of Markup("…{var}…").format(), in order
to allow Semgrep to detect mistakes like Markup("…{var}…".format())
and Markup(f"…{var}…").

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-22 11:23:27 -07:00
Anders Kaseorg 3a27b12a7d dependencies: Switch to pnpm.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-20 15:48:29 -07:00
Anders Kaseorg b3f9fdf99a dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-20 15:48:29 -07:00
Anders Kaseorg 14352547cd install-transifex-cli: Upgrade Transifex CLI from 1.6.4 to 1.6.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-17 13:11:02 -07:00
Prakhar Pratyush ab8222c4ae custom_check: Update FILES_WITH_LEGACY_SUBJECT set.
This commit adds 'zerver/lib/email_notifications.py'
to the FILES_WITH_LEGACY_SUBJECT set.

Because the file can have 'subject' in the email sense,
it should be exempted from the 'avoid subject as a var' lint rule.
2023-03-14 17:48:57 -07:00
sbansal1999 5f4bdfe6bc ts: Convert css_variables.js to TypeScript. 2023-03-10 14:35:23 -08:00
palashb01 dee63680df docs: Update time format and Emoticon help page.
Updated the title and description in the 'enable-emoticon-translation'
file and renamed the file accordingly. Added a new bullet point for
'time format' in the 'configure-new-user-settings.md' file and updated
the sidebar index by replacing the title 'Use 24-hour time' with
'Change the time format'.
2023-03-09 16:12:45 -08:00
Anders Kaseorg 2d9b2a2a05 models: Remove type prefixes from __str__ values.
The Django convention is for __repr__ to include the type and __str__
to omit it.  In fact its default __repr__ implementation for models
automatically adds a type prefix to __str__, which has resulted in the
type being duplicated:

    >>> UserProfile.objects.first()
    <UserProfile: <UserProfile: emailgateway@zulip.com <Realm: zulipinternal 1>>>

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-08 22:56:55 -08:00
Alex Vandiver 8f8a9f6f04 sentry: Add frontend event monitoring.
Zulip already has integrations for server-side Sentry integration;
however, it has historically used the Zulip-specific `blueslip`
library for monitoring browser-side errors.  However, the latter sends
errors to email, as well optionally to an internal `#errors` stream.
While this is sufficient for low volumes of users, and useful in that
it does not rely on outside services, at higher volumes it is very
difficult to do any analysis or filtering of the errors.  Client-side
errors are exceptionally noisy, with many false positives due to
browser extensions or similar, so determining real real errors from a
stream of un-grouped emails or messages in a stream is quite
difficult.

Add a client-side Javascript sentry integration.  To provide useful
backtraces, this requires extending the pre-deploy hooks to upload the
source-maps to Sentry.  Additional keys are added to the non-public
API of `page_params` to control the DSN, realm identifier, and sample
rates.
2023-03-07 10:51:45 -08:00
Alex Vandiver a8181152b7 webpack: Build a ZULIP_VERSION global constant into the built product. 2023-03-07 10:51:45 -08:00
Daniil Fadeev d606f5ba42 i18n: Add new markup for lazy translation of giphy ratings.
Also, it adds phrases such as 'rated Y' to the ignored list.
Otherwise, the linter would require to write it as 'rated y'.
2023-03-06 10:52:52 -08:00
Anders Kaseorg 7b47b6a311 docs: Move homepage link to TOC tree.
This lets us avoid maintaining a forked copy of sphinx_rtd_theme’s
sidebartitle block.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 23:06:17 -08:00
Anders Kaseorg 821584b8fc gitlint-rules: Rename to gitlint_rules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:38 -08:00
Anders Kaseorg 43b4f10578 run-dev: Drop .py from script name.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-03-03 18:02:37 -08:00
Sahil Batra 3950a8e19d test_helpers: Rename reset_emails_in_zulip_realm.
This commit renames reset_emails_in_zulip_realm function to
reset_email_visibility_to_everyone_in_zulip_realm which makes
it more clear to understand what the function actually does.

This commit also adds a comment explaining what this function
does.
2023-03-01 12:17:11 -08:00
Lauryn Menard 42775219b3 private-messages: Update frontend strings to use "direct message" or "DM".
Updates frontend user-facing strings with "private message" or "PM" to
use "direct message" or "DM" respectively instead.

Note that this updates translated strings as well as a few that
are not translated like search suggestions.

Updates `tools/lib/capitalization.py` for some specific strings
that are impacted by these changes, and removes "PM" and "PMs"
from checked strings.
2023-02-24 11:47:26 -08:00
Lauryn Menard e39c689f36 email-notifications: Update email subjects with "PM" to "DM".
Updates email notification subjects with "PM" to use "DM" instead.

Also, updates `tools/lib/capitalization.py` to include both "DM"
and "DMs".
2023-02-24 11:47:26 -08:00