Commit Graph

49991 Commits

Author SHA1 Message Date
Anders Kaseorg 9a7f33ab98 migrations: Fix ‘continue’ logic error in 0037.
The intention was to continue the outer ‘for’ loop, not the inner one
(but Python doesn’t have labelled ‘continue’).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-18 13:34:52 -08:00
Greg Price f38b5c41da shared: Bump version to 0.0.18 2023-01-18 13:25:44 -08:00
Anders Kaseorg a5fefa5932 shared: Remove js source from distributed package.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-18 13:25:08 -08:00
Greg Price 5407d52cb4 shared: Bump version to 0.0.17 2023-01-18 13:22:47 -08:00
Greg Price 52de17e08f shared: Specify files for NPM package, so lib/ gets included
Otherwise `npm publish` and friends leave it out because it's
ignored in Git.

This also cuts out `tools/` and a few other such files.
2023-01-18 13:20:56 -08:00
Anders Kaseorg d41917ecd0 shared: Add TypeScript build step.
This transpiles the JavaScript and (future) TypeScript in
`static/shared/js` to `static/shared/lib`. It also compiles away
ES2020 syntax that’s not supported by the oldest JS engines targeted
by zulip-mobile.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-18 13:10:13 -08:00
Trident Pancake c6ea673cc9 markdown: Update max inline preview from 10 to 24.
The max inline preview limit was previously increased to 10 by #20789.
However, as issue #23624 shows, it's still causing confusion for users
when they include more than 10 links.

Bump this limit up to 24, which is a multiple of the 4 image preview
per line logic.
2023-01-18 14:58:00 -05:00
N-Shar-ma 25eff08324 typeahead: Show the time typeahead irrespective of text before syntax.
Uptil now, on typing "<time" after some other autocompleteable token
like a mention or emoji, the timezone aware time typeahead would not
get triggered since the time syntax was checked after the earlier syntax
had been mistakenly used to (wrongly) tokenize the precursor text.

Now the code has been fixed to detect the time syntax the same way as
the rest: checking each character in the precursor text from end to
start, and tokenize it correctly.

Fixes: #23998.
2023-01-17 17:10:14 -08:00
Lauryn Menard ba443cac03 templates: Set robots noindex for attribution corporate template.
Overrides the default context `allow_search_engine_indexing` to
always be `False` for `templates/corporate/attribution.html` so
that it does not appear in Google / search engine indexes.

Updates test of documentation pages in `test_docs.py` to have an
option for corporate pages to set this value in the template and
verifies that the meta tag for robots noindex, nofollow is
always in the response.
2023-01-17 15:00:39 -08:00
Lauryn Menard 53ed7d5ce2 docs: Remove paragraph about getting help finding issues in contributing.
Removes a paragraph about posting in the new members stream to get
help finding issues to work on from the first contribution part of
the contributing guide.

Also updates the remaining section about posting to new members as
a way to introduce yourself with a link to the community norms and
offering positive feedback as well as constructive criticism.
2023-01-17 14:51:53 -08:00
Lauryn Menard c2bcfb52aa api-tests: Reduce error output for `/register` openapi validation.
For descriptive endpoints, such as `/register`, that might raise
Schema Validation errors via `validate_against_openapi_schema`,
omits the OpenAPI schema definition in the error output.

Also omits the error instance definition in the error output
when it is a jsonschema object with over 100 properties. This
means that the test instance for objects, like user settings,
will be printed in the error output, but the test instance for
the entire endpoint will not be printed to the console.

The omitted output can be thousands of lines long making it
difficult to find the initial console output that actually helps
the contributor with debugging.

Adds a section in "Documenting REST API endpoints" about
debugging and understanding these errors that is linked to
in the error console output.
2023-01-17 14:50:42 -08:00
Sahil Batra e80e3b07c4 settings: Fix flickering of icon when hovering on discard button.
The color of "x" icon in "Discard" button flickers when hovering
over the button. This commit fixes it by just adding the hover
color to the text and not icon which was anyways set to original
color using different selector which was the cause of flickering.
2023-01-17 14:45:48 -08:00
Sahil Batra c0e2c9b6d6 settings: Change hover behavior of save discard buttons.
This commit fixes the hover behavior of save-discard button
in dark theme. We change the text to be slightly brighter
on hover and keep the icon color same.

The background-color property is removed from hover CSS. This
change is safe because for save-button, we already define
different hover behavior below which takes precedence. And
for light-theme, the discard button already has the same
background-color without hovering, so this property was only
affecting discard button in dark-theme.
2023-01-17 14:45:48 -08:00
Sahil Batra e804795fa6 settings: Fix text color for discard button in dark theme.
We change the color of text and "x" icon in the discard
button in dark theme to be same as that of other text
in the settings page.
2023-01-17 14:45:48 -08:00
Sahil Batra 4e63794ddd settings: Fix background-color of save-discard widget in dark theme.
The correct background-color for buttons of save-discard widget
was not being applied and instead almost transparent color was
applied to dark-theme CSS rules. This commit adds ID to the
selector such that CSS in app_components.css is preferred over
dark-theme CSS.
2023-01-17 14:45:48 -08:00
Josh Klar 111f395eb6 i18n: Sync latest unbranched translations from Transifex.
Additionally, run unescape-html-in-json-translations over the resulting
JSON files to ensure HTML escape sequences are removed.
2023-01-17 13:20:49 -08:00
Josh Klar 70b30e7792 i18n: Unescape Unicode sequences in JSON.
This greatly improves the readability of the diffs and in-codebase
translation strings over using ASCII sequences for unicode in the JSON.

We've previously noticed [^1] some JSON translation files ending up with
escaped Unicode sequences on disk, which Transifex indicates is expected
behavior [^2], though it is sometimes fixed by `manage.py
compilemessages` [^3]. Further, as noted in #23932 [^4], some JSON
translation files include HTML-escaped entities like quotation marks.

This script will ingest valid JSON files and output them as proper UTF-8
files with appropriately unescaped (unless otherwise necessary, like
double quotes being backslash-escaped) sequences, except when the key
itself contains HTML escape sequences (as it's presumed the value of
such entries must be pre-escaped before being passed to consumers).

[^1]: https://chat.zulip.org/#narrow/stream/58-translation/topic/Transifex.20client/near/1479205

[^2]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481287

[^3]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481908

[^4]: Which is not end-to-end fixed yet by this commit: that will
require a new release of Zulip Server.

gitlint-ignore: B1, title-trailing-punctuation, body-min-length, body-is-missing
2023-01-17 13:19:45 -08:00
Sahil Batra 0718043283 bot_settings: Remove "#bot_table_error" element.
We do not use "#bot_table_error" element to show any errors
anymore. It was previously used to show the error if bot
creation failed but since 6db88f0d39 moved bot creation
to a modal, we now show error, if any, inside the modal
itself. This commit also removes the hide_errors function
since the error element itself is removed.
2023-01-17 09:45:40 -08:00
Sahil Batra 316248308a bot_avatar_row: Remove ".bot_error" element.
We do not use ".bot_error" element to show errors
anymore after a9893fb654 added confirmation modal
for reactivating bot.
2023-01-17 09:45:40 -08:00
Sahil Batra fe3e58b497 settings_bots: Remove unused functions.
This commit removes bot_error function in settings_bots.js
since it is not used now. It was added in d90d1a04 and was
used in error handling, but since we changed the design to
show dialog for reactivation and deactivation of bots and
errors are shown inside the dialog and errors are handled
differently and thus the function's usage was removed in
a9893fb654.

This commit also removes get_bot_info_div which was only
used by bot_error function.
2023-01-17 09:45:40 -08:00
Sahil Batra 2aeda1b0c4 profile_settings: Remove "hide" class from h3 element.
We already have "inline-block" class for the h3 element
in profile_settings.hbs, so the display property is set
as "inline-block" and the CSS set by "hide" class is
overridden. We should not have "hide" class for this
element, since we want to show that element and not
hide it.

This was probably added while picking code of some
other element in bb816e199.
2023-01-17 09:45:40 -08:00
Lauryn Menard 2a60e4c1e6 portico-signin: Clean up CSS for buttons on dev login page.
Removes the `btn-direct` class in `portico.css` that was only
being used for dev login buttons.

Adds `dev-button` class for general CSS rules for buttons on the
dev login page. Adds `dev-login-button` and `dev-create-button`
classes for CSS rules specific to the two types of buttons on the
page.
2023-01-17 09:44:26 -08:00
Lauryn Menard 49daf6743a portico-css: Remove unused `find_account` CSS rules.
Removes `#find_account .btn` and `#find_account .form-control`
rules in `static/styles/portico/portico.css`.

The last use of these rules was removed in commit 7afbc9ddd6
when the login and registration pages were redesigned.
2023-01-17 09:42:56 -08:00
Sahil Batra 42c28f008b stream_settings: Move upgrade-text below the label.
This commit moves the upgrade text for message retention
setting below the label so that it is clear that the text
is present for message retention setting. This change is
done for both stream creation form and stream edit panel.
2023-01-15 13:36:33 -08:00
Sahil Batra 0bc61281df stream_settings: Fix upgrade text in stream edit panel.
The upgrade text shown for message retention setting for realms
on limited plans should mention about sponsorship only for
non-business orgs. In the stream edit panel, the sponsorship
text was present even for non-business orgs because the
is_business_type_org parameter was not being passed to the
template, so this commit fixes the code to pass the
is_busines_type_org parameter correctly to the template.
2023-01-15 13:36:33 -08:00
Sahil Batra 90566ab772 stream_create: Disable the message retention setting for limited plans.
We should disable the message retention setting in stream creation
form for limited plans.
2023-01-15 13:36:33 -08:00
Sahil Batra 6f20d7c9ab stream_create: Refactor code to set default value for retention setting.
The message retention setting is only visible to owners in the
stream creation form, so the jquery code to hide the custom
input, set the default value for dropdown and listener to
show and hide the custom input should be called only for owners.
2023-01-15 13:36:33 -08:00
Sahil Batra 1c1b911a42 stream_create: Fix comments in show_new_stream_modal.
We fix the comment stating that announce_stream setting
is set on "on", as we handle it differently now in
update_announce_stream_state.
2023-01-15 13:36:33 -08:00
Aman Agrawal 1f3367abfb message_list_view: Remove dead next_is_same_sender code.
The `next_is_same_sender` has no effect on the CSS of the message
displayed and the JS changes seem to have no effect too.

See cc8021a742 for more details.
2023-01-14 14:49:28 -08:00
Josh Klar 465d0b2710 message_list_view: Use translated form of "at" in timestamp tooltip.
The English word "at" was manually appended to the string output of
datetime-related functions to generate the string shown in the tooltip
when hovering over the timestamp of a message. Use the translated form
"{date} at {time}" instead, as found elsewhere in the codebase.
2023-01-12 20:47:01 -08:00
Lauryn Menard 1a3b0edf4b account-settings: Disable deactivate account button when only owner.
Disables the deactivate account button in the user's account and
privacy settings tab if they are the only active organization owner.

Adds a tooltip when hovering on the deactivated button to let the
user know why the button is disabled.

The backend already returns an error for self account deactivation
requests if the user is the only organization owner.
2023-01-11 13:30:31 -08:00
Lauryn Menard 7abf476443 settings_account: Update avatar widget if user/org permissions changed.
Adds `avatar.build_user_avatar_widget` to the updates done in
`settings_account.update_avatar_change_display` when the user
has permission to change/delete their avatar.

For the case where a user has their personal account profile
settings open while the organization or user's permissions are
changed, `user_avatar_widget_created` boolean is added to
`settings_account.js` to track whether the widget and handlers
have been created so that live updates don't rebuild the
existing widget and handlers.

Also, updates `avatar.build_user_avatar_widget` to return early if
the user cannot change their avatar. This pattern better matches
the other instances where we use the direct upload widget for the
organization icon and logos.
2023-01-11 13:12:59 -08:00
Lauryn Menard fd4e071f32 organization-logo: Update the titles for the logo color schemes.
The "day" and "night" color scheme names were updated in #20371,
but the organization profile settings tab still has had the old
names for the two types of wide logos.

Updates "Day logo" to be "Light theme logo" and "Night logo" to
be "Dark theme logo".
2023-01-11 13:12:59 -08:00
Lauryn Menard 55f34c3086 settings-org: Hide image upload button if user no longer admin.
Adds hiding any elements with the `image_upload_button` class in
the organization profile (icon and logos) to the live updates in
`settings_org.maybe_disable_widgets`, so that any role changes
that remove admin permissions will update these images so that
they no longer appear able to be updated / deleted.
2023-01-11 13:12:59 -08:00
Lauryn Menard eef22e4e04 settings-account: Make live update for avatar changes consistent.
The `settings-info-icon` with the tooltip for the user avatar
section was removed in commit 103db2afaf. This removes the changes
to that tooltip in `settings_account.update_avatar_change_display`.

Instead of hiding or showing the `image_upload_button` in that same
live update function, we now add or remove the `hide` class, which
in the previous commit has a specific CSS hover rule for the image
upload widget.

Also adds test coverage for the `is_admin` case for this setting
because administrators can always change their own avatar even if
the organization has disabled avatar changes.
2023-01-11 13:12:59 -08:00
Lauryn Menard d3a513f4bd image-upload-widget: Render `image_disabled` div if text to display.
If there is no `disabled_text` provided for the image upload widget,
then there is no need to have the `image_disabled` div rendered in
the template. This allows the hover CSS rules to be more general for
the image upload widget in general.

Adds a check for `disabled_text` around the `image_disabled` div
element in `image_upload_widget.hbs`.

Also, changes `image_upload_background` class to
`image_hover_background` so that it more accurately describes what
the CSS rule is for and why it's used in both the `image_disabled`
div and the `image_upload_button` div.
2023-01-11 13:12:59 -08:00
Manas Jayaswal 3cbec56e81 image-upload-widget: Fix hover behavior when image cannot be changed.
The `hide` CSS rule in `app_components.css` was being overwritten
by the more specific rules in `image_upload_widget.css`, which
meant that when changing the inage was disabled the hover text for
updating or deleting the image was still visible.

Adds `hide` class (and therefore more specific rule) to
`image_upload_widget.css` for when this text should be display
none.

Fixes #23844.
2023-01-11 13:12:59 -08:00
Tim Abbott e50ab2dad3 message_list_view: Remove include_footer parameter.
This only set the last_message CSS class, which is no longer used for
styling or JavaScript code.

(The calculation was also wrong, in that new messages arriving would
not cause it to be removed from the previous message with the
last_message class).
2023-01-10 17:03:37 -08:00
Tim Abbott b1155516d1 lightbox: Rewrite logic depending on last_message CSS class.
The last_message CSS class didn't mean what it said it did, due to
issues with live update.

Further, this logic was poorly written, with `$message` changing types
from a .message_row to a .recipient_row for now apparent reason.

I was able to reproduce at least one bug where the `v` shortcut would
not correctly open the lightbox that is fixed by this rewrite.
2023-01-10 17:03:37 -08:00
Aman Agrawal 2fa88362bc message: Hide first action button on narrow widths.
Fixes zulip#19759

Before grid, there was overlap of action buttons and text (photo 1).
With grid, there was just very little space for messages (photo 2).
With this commit, the message gets a bit more space (photo 3).

This helps us add more width to the content column of message
on narrow widths. Note that it will still always be less than
before grid, since we are explicitly preventing overlap.
2023-01-10 17:03:37 -08:00
Aman Agrawal 26011c98fd css: Use grid layout for messages.
Using grid layout moves us away from fixed widths
and position for items where possible, which will
make it easier to make messagebox changes without
breaking formatting.

Visual changes expected in this commit:
- When the action buttons overlap the message content,
it will no longer visually overlap but completely cut off
the content — which is good, but means the messages can’t
be as wide on narrow width views. There’s a slight improvement
to this in an upcoming commit.

Also removes the `last_message` CSS, which was busted due to the
`last_message` calculation not correctly being updated for new
messages arriving, and didn't improve styling.
2023-01-10 17:03:37 -08:00
Aman Agrawal 953277bdae message_list: Add `content_edit_mode` when editing a message.
Having the class at the top of message DOM structure when the
user is editing a message, helps apply css when user is editing
a message.

Short prep commit for the next commit; no visible changes
2023-01-10 17:03:37 -08:00
Lauryn Menard fe03d2a533 api-docs: Clarify only API doc paths check for endpoint info.
Previously, we got the directory path for all documentation pages
before checking for API method and path information in the OpenAPI
documentation. Instead, we now check the `path_template` is the
API documentation view template before getting the directory path.

Also, changes the confusingly named `article_path` variable, which
overlapped with the DocumentationArticle dataclass `article_path`
field, to now be `api_documentation_path`.

Prep commit for moving the help center documentation to a top level
directory.
2023-01-10 15:32:47 -08:00
Alex Vandiver e351df4095 changelog: Add entry for S3 nginx proxying. 2023-01-10 15:30:57 -08:00
Mateusz Mandera 89d1f1f385 messages: Eliminate redundant realm fetch in has_message_access.
Accessing .realm will cause a fetch query from the database if the
attribute hasn't been fetched already earlier in the codepath. That's
completely redundant if we're just comparing realms, and we should only
access .realm_id attribute. This seems to eliminate a query in some
codepaths, which is nice in this performance-sensitive function.
2023-01-10 15:27:55 -08:00
Lauryn Menard a7fd994cbd docs: Link to management commands documentation in user facing docs.
Adds links to the documentation about management commands in the
API documentation for creating users, as well as the `/devtools`
documentation, the GDPR compliance article and the incoming
webhooks tutorial.
2023-01-10 08:50:00 -08:00
Alex Vandiver 04cf68b45e uploads: Serve S3 uploads directly from nginx.
When file uploads are stored in S3, this means that Zulip serves as a
302 to S3.  Because browsers do not cache redirects, this means that
no image contents can be cached -- and upon every page load or reload,
every recently-posted image must be re-fetched.  This incurs extra
load on the Zulip server, as well as potentially excessive bandwidth
usage from S3, and on the client's connection.

Switch to fetching the content from S3 in nginx, and serving the
content from nginx.  These have `Cache-control: private, immutable`
headers set on the response, allowing browsers to cache them locally.

Because nginx fetching from S3 can be slow, and requests for uploads
will generally be bunched around when a message containing them are
first posted, we instruct nginx to cache the contents locally.  This
is safe because uploaded file contents are immutable; access control
is still mediated by Django.  The nginx cache key is the URL without
query parameters, as those parameters include a time-limited signed
authentication parameter which lets nginx fetch the non-public file.

This adds a number of nginx-level configuration parameters to control
the caching which nginx performs, including the amount of in-memory
index for he cache, the maximum storage of the cache on disk, and how
long data is retained in the cache.  The currently-chosen figures are
reasonable for small to medium deployments.

The most notable effect of this change is in allowing browsers to
cache uploaded image content; however, while there will be many fewer
requests, it also has an improvement on request latency.  The
following tests were done with a non-AWS client in SFO, a server and
S3 storage in us-east-1, and with 100 requests after 10 requests of
warm-up (to fill the nginx cache).  The mean and standard deviation
are shown.

|                   | Redirect to S3      | Caching proxy, hot  | Caching proxy, cold |
| ----------------- | ------------------- | ------------------- | ------------------- |
| Time in Django    | 263.0 ms ±  28.3 ms | 258.0 ms ±  12.3 ms | 258.0 ms ±  12.3 ms |
| Small file (842b) | 586.1 ms ±  21.1 ms | 266.1 ms ±  67.4 ms | 288.6 ms ±  17.7 ms |
| Large file (660k) | 959.6 ms ± 137.9 ms | 609.5 ms ±  13.0 ms | 648.1 ms ±  43.2 ms |

The hot-cache performance is faster for both large and small files,
since it saves the client the time having to make a second request to
a separate host.  This performance improvement remains at least 100ms
even if the client is on the same coast as the server.

Cold nginx caches are only slightly slower than hot caches, because
VPC access to S3 endpoints is extremely fast (assuming it is in the
same region as the host), and nginx can pool connections to S3 and
reuse them.

However, all of the 648ms taken to serve a cold-cache large file is
occupied in nginx, as opposed to the only 263ms which was spent in
nginx when using redirects to S3.  This means that to overall spend
less time responding to uploaded-file requests in nginx, clients will
need to find files in their local cache, and skip making an
uploaded-file request, at least 60% of the time.  Modeling shows a
reduction in the number of client requests by about 70% - 80%.

The `Content-Disposition` header logic can now also be entirely shared
with the local-file codepath, as can the `url_only` path used by
mobile clients.  While we could provide the direct-to-S3 temporary
signed URL to mobile clients, we choose to provide the
served-from-Zulip signed URL, to better control caching headers on it,
and greater consistency.  In doing so, we adjust the salt used for the
URL; since these URLs are only valid for 60s, the effect of this salt
change is minimal.
2023-01-09 18:23:58 -05:00
Alex Vandiver 58dc1059f3 uploads: Move unauth-signed tokens into view. 2023-01-09 18:23:58 -05:00
Alex Vandiver ed6d62a9e7 avatars: Serve /user_avatars/ through Django, which offloads to nginx.
Moving `/user_avatars/` to being served partially through Django
removes the need for the `no_serve_uploads` nginx reconfiguring when
switching between S3 and local backends.  This is important because a
subsequent commit will move S3 attachments to being served through
nginx, which would make `no_serve_uploads` entirely nonsensical of a
name.

Serve the files through Django, with an offload for the actual image
response to an internal nginx route.  In development, serve the files
directly in Django.

We do _not_ mark the contents as immutable for caching purposes, since
the path for avatar images is hashed only by their user-id and a salt,
and as such are reused when a user's avatar is updated.
2023-01-09 18:23:58 -05:00
Alex Vandiver f0f4aa66e0 uploads: Inline the one callsite of get_local_file_path.
This helps make more explicit the assert_is_local_storage_path which
makes using local_path safe.
2023-01-09 18:23:58 -05:00