Commit Graph

46753 Commits

Author SHA1 Message Date
Tim Abbott d149af936d models: Rework Attachment.is_*_public to be a cache.
Previously, Attachment.is_realm_public and its cousin,
Attachment.is_web_public, were properties that began as False and
transitioned to True only when a message containing a link to the
attachment was sent to the appropriate class of stream, or such a link
was added as part of editing a message.

This pattern meant that neither field was updated in situations where
the access permissions for a message changed:

* Moving the message to a different stream.
* Changing the permissions for a stream containing links to the message.

This correctness issue has limited security impact, because uploaded
files are secured both by a random URL and by these access checks.

To fix this, we reformulate these fields as a cache, with code paths
that change the permissions affecting an attachment responsible for
setting these values to the `None` (uncached) state. We prefer setting
this `None` state over computing the correct permissions, because the
correct post-edit permissions are a function of all messages
containing the attachment, and we don't want to be responsible for
fetching all of those messages in the edit code paths.
2022-03-24 10:49:59 -07:00
Alex Vandiver 4f93b4b6e4 uploads: Skip the outgoing proxy if S3_KEY is unset.
When the credentials are provided by dint of being run on an EC2
instance with an assigned Role, we must be able to fetch the instance
metadata from IMDS -- which is precisely the type of internal-IP
request that Smokescreen denies.

While botocore supports a `proxies` argument to the `Config` object,
this is not actually respected when making the IMDS queries; only the
environment variables are read from.  See
https://github.com/boto/botocore/issues/2644

As such, implement S3_SKIP_PROXY by monkey-patching the
`botocore.utils.should_bypass_proxies` function, to allow requests to
IMDS to be made without Smokescreen impeding them.

Fixes #20715.
2022-03-24 10:21:35 -07:00
Tim Abbott 0d90bb2569 narrow: Fix messages being cached without flags set.
f0c680e9c0 introduced a call to
message_helper.process_new_message without first calling
message_store.set_message_flags on the message.

This resulted in it being possible as a race, when loading the Zulip
app to a stream/topic/near narrow, for a message to have the
`historical` flag be undefined due to not being initialized.

That invalid state, in turn, resulted in the message_list_view code
path for rendering the message feed incorrectly displaying additional
recipient bars around the message.

We could fix this by just calling message_store.set_message_booleans
in this code path. However, this bug exposes the fact that it's very
fragile to expect every code path to call that function before
message_helper.process_new_message.

So we instead fix this by moving message_store.set_message_booleans
inside message_helper.process_new_message.

One call point of concern in this change is maybe_add_narrow_messages,
which could theoretically reintroduce the double set_message_flags
bugs detailed in 9729b1a4ad. However, I
believe that to not be possible, because that call should never
experience a cache miss.

The other existing code paths were already calling
set_message_booleans immediately before
message_helper.process_new_message. They are still changing here, in
that we now do a cache lookup before attempting to call
set_message_booleans. Because the message booleans do not affect the
cache lookup and the local message object is discarded in case of a
cache hit, this should have no functional impact.

Because I found the existing comment at that call site confusing and
almost proposed removing it as pointless, extend the block comment to
explicitly mention that the purpose is refreshing our object.

Fixes #21503.
2022-03-24 08:10:14 -07:00
Tim Abbott 00332fd49d markdown: Simulate message flags in frontend markdown processor.
This eliminates an annoying bundle of complexity that caused the
frontend markdown processor's interface with the rest of Zulip's new
message processing code paths being more similar to that of a new
message from the server.

It also cuts down on code duplication.
2022-03-24 08:10:14 -07:00
Tim Abbott 2dd0b386fe echo: Clean up confusing handling of message flags in local echo.
The previous message.unread block in insert_local_message was
non-functional. markdown.apply_markdown is overriding what that set by
calling message_store.init_booleans, which happens to set the same
value for the `unread` flag, and then setting individual message
booleans as it finds elements like mentions during rendering.

Improve this situation partially by deleting the message_unread block,
and adding conversion logic to translate what the markdown processor
is doing into message flags.

Then, we can call message_store.set_message_flags just like we do when
processing new messages arriving via the API. This will be helpful
shortly, when we deduplicate the calls to
message_store.set_message_flags.

As noted in the lengthy TODO, this exposes the fact that we should
really rework how the frontend markdown processor returns the
mentioned state to its caller.
2022-03-24 08:10:14 -07:00
Lauryn Menard 7dd2ed85db help_docs: Update documentation about moving content.
Moves descriptions of notification and subsetting options
when moving content out of the instructions tab blocks
and into descriptive text.

Adds documentation for editing a topic via the message
recipient bar.

Also cleans up a few changes to UI interactions.

Co-authored-by: Alya Abbott <alya@zulip.com>
2022-03-23 23:46:22 -07:00
Tim Abbott ec54b1e5a8 i18n: Update translation data from Transifex. 2022-03-23 23:41:11 -07:00
Abhishek Reddypalle cab37b4aca tools: Add tool to count contributions across all major repositories.
This will allow Zulip release announcements to credit contributions
made to Zulip projects beyond the server in our release announcements.

Fixes #19044.
2022-03-23 14:20:30 -07:00
Greg Price 4befe4fc30 mailmap: Add a comment explaining this file.
In particular, link to upstream documentation.  This should help
contributors be able to see what it's doing and how to update it.

Copied straight from what we have in the zulip-mobile repo.
2022-03-23 14:12:43 -07:00
ditsuke 4d03a1b0b7 message_feed_view: Fit multiple images horizontally.
At the moment we fit only a single image per message per line. This is
wasteful of space as multiple images can be accomodated per line on
widescreen displays. This commit modifies the rendered_markdown
stylesheet to make this possible.

The comments detail various technical considerations.

Fixes #20975.
2022-03-22 22:35:17 -07:00
Alya Abbott c121bec188 help center: Explain details of content moves. 2022-03-22 20:18:25 -07:00
Alya Abbott 6087f0daf1 help center: Improve organization of left sidebar. 2022-03-22 20:18:25 -07:00
Anders Kaseorg 24070c7ad8 beeminder: Accept float for payload["goal"]["pledge"].
The documentation at https://api.beeminder.com/#goal says this is
“number”; empirically, we do in fact get decimal points.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-22 21:08:48 -04:00
Aman Agrawal 6fcbe4091d web_public_streams: Change globe icon.
This revised globe icon avoids looking like a "language choice" icon
(as the previous one did), while still being recognizably Earth (and
not a disk with some things drawn on it) and not showing only North
America (a flaw with the Font Awesome 4.7 icon).

Used a derivative of icon from
https://unpkg.com/ionicons@5.5.2/dist/svg/earth.svg
with modified outline by Vlad Korobov.
2022-03-22 16:15:55 -07:00
Aman Agrawal 9a7fadbbeb zulip-icons: Move custom icons to shared folder. 2022-03-22 16:14:56 -07:00
Lauryn Menard babe5ed44a help: Update notifications docs for changes.
Adds documentation for unread badge count to help article on
desktop notifications.

Generally, cleans up instructions and article structure for
the help article on pms, mentions and alerts.

Co-authored-by: Alya Abbott <alya@zulip.com>
2022-03-22 15:52:23 -07:00
Alex Vandiver 1ac0035f8c markdown: Allow whitespace overlaps in topic linkifiers.
`prepare_linkifier_pattern`, as of db934be064, adds a match to the
end of the regex, of either the end of string, or a non-word character
-- this is in place of a negative look-ahead, which is no longer
possible in re2.  This causes the regex to consume trailing
whitespace, and thus not be able to match twice in succession with
`pattern.finditer` -- "#1234 #5678" fails to match because the space
is consumed by the first match of the regex.

Rather than use `pattern.finditer`, write own own version, which
rewinds over the non-word character consumed after the match, if any.
This allows the same "after" non-word character to also satisfy the
"before" of the next match.

Fixes #21502.
2022-03-22 15:40:03 -07:00
Sahil Batra d89b5042a9 settings: Send config_data to bot update API only if required.
There is config_data for the embedded bots only for giphy and
followup bots, so we send "config_data" field to API only for
those bots and not others. Send config_data field as {} to
the API for other bots raised InternalError earlier.
2022-03-22 15:38:43 -07:00
Sahil Batra 630abf57d9 settings: Remove bot from "Active bots" list on changing owners.
On changing bot owner, "delete" event is sent to the previous
owner if previous owner is not an admin. We were ignoring the
"delete" event in webapp previously, but now we update the
bots page in personal settings to delete the bot. Note that
we do not remove the bot from the organization list of bots
currently, since list_widget does not support removing a
row as of now.

In case of previous owner being an admin, the previous owner
receives "update" event and thus the bots list is updated
from that event.

The code for ignoring "delete" event was added in fba2708bbc,
to basically avoid failed lookup for id in the organization
list of bots. I have tested and there cannot be a case of
a failed lookup in the list as per current code for list_widget
module and we are anyways safe after a reload or after closing the
overlay as the list will be updated correctly.

Discussion thread -
https://chat.zulip.org/#narrow/stream/321-settings-system/topic/List.20render.20.2315033

Fixes #20856.
2022-03-22 15:38:43 -07:00
Alex Vandiver b9e428dd5d lightbox: Make the "download" link use the new download endpoint.
The "download" attribute on the button only functions for same-origin
requests; thus, the download endpoint must be used in order for the
"Download" button to function for uploaded images which are stored in
S3, and thus served from a different origin.

This is only done for uploaded images; it does not address a similar
problem with Camo, when Camo is hosted on a different hostname.

Fixes: #19238.
2022-03-22 15:05:02 -07:00
Alex Vandiver abed174b12 uploads: Add an endpoint which forces a download.
This is most useful for images hosted in S3, which are otherwise
always displayed in the browser.
2022-03-22 15:05:02 -07:00
Alex Vandiver d7b59c86ce puppet: Build wal-g from source for aarch64.
Since wal-g does not provide binaries for aarch64, build them from
source.  While building them from source for arm64 would better ensure
that build process is tested, the build process takes 7min and 700M of
temp files, which is an unacceptable cost; we thus only build on
aarch64.

Since the wal-g build process uses submodules, which are not in the
Github export, we clone the full wal-g repository.  Because the
repository is relatively small, we clone it anew on each new version,
rather than attempt to manage the remotes.

Fixes #21070.
2022-03-22 15:02:35 -07:00
Alex Vandiver 4d4c320a07 puppet: Switch from ntp to chrony.
Chrony is the recommended time server for Ubuntu since 18.04[1], and
is the default on Redhat; it is more accurate, and has lower-memory
usage, than ntp, which is only getting best-effort security
maintenance.

See:
- https://wiki.ubuntu.com/BionicBeaver/ReleaseNotes#Chrony
- https://chrony.tuxfamily.org/comparison.html
- https://engineering.fb.com/2020/03/18/production-engineering/ntp-service/
2022-03-22 13:07:27 -07:00
Lauryn Menard b67288db67 help_docs: Extend options for getting links to Zulip content.
Extends the linking to Zulip documentation to cover:

- Getting URLs to messages via the message timestamp.
- Getting links to topics via the three-dots menu.
- Getting links to streams via right-click context menu.

Creates a new tabbed section for using the browser
address bar to copy URLs.
2022-03-22 12:38:09 -07:00
Lauryn Menard 453cb409cf help_docs: Update button description for archive a stream doc.
In the current UI, the button for archiving a stream is an icon
while previously it was text ('Archive'). Updates documentation
to refer to it as so.
2022-03-22 12:08:14 -07:00
Lauryn Menard ba7695e5de help_docs: Revise `mute-a-topic` help doc.
Generally, revises a number of out of date information in this
help article.
2022-03-22 12:06:24 -07:00
Tim Abbott 2f929bee2f compose: Match width of topic compose to left sidebar space.
Previously, the maximum width for the topic input in the compose box
was artificially limited to 20% of the width of the compose box.

While this may have had some useful role in encouraging short topics,
we can teach that idea in other ways, and it seems more helpful to
have the input length match what works well for viewing topics in the
left sidebar without being cut off.
2022-03-22 11:52:26 -07:00
yogesh sirsat 8d40199729 settings_users: Display error inside deactivate user modal.
Currently, when deactivating a user, we have a "Saving..." loading
indicator and any error message displayed in the heading area of the
users table.

Migrate this to instead do the loading indicator and error message
inside the modal, where it's more in context.
2022-03-22 11:14:48 -07:00
Tim Abbott ceb9dd5854 migrations: Fix confusing output for migration 0383.
* Don't print the empty list for the vast majority of realms where
  this is a noop.

* Make output a little more clear that this isn't revoking all
  Confirmations, just those associated with deactivated users.
2022-03-21 23:25:06 -07:00
Rishabh-792 3f0b0ee88c settings: Improve help icons in settings.
Improved the contrast of ? and i icons by changing their opacity to a
consistent 0.6, going to 1 on hover.

Changed the colour of playground icon by testing and added spacing
between title and the playground icon by changing the icon margin.

With some TODO comments added by tabbott for readability.

Fixes part of #20484.
2022-03-21 18:07:16 -07:00
SantamRC 373f700736 widgets: Fix line wrapping of todo descriptions.
The simplest solution for doing this involves adding some divs.

Fixes #20523.
2022-03-21 17:52:45 -07:00
My-Name-Is-Nabil cc32a3afff compose: Fix alignment of close buttons in stream invite banners.
Close buttons are misaligned if the warning banner text takes up two lines.

We increase the specificity of the selectors to ensure that this CSS
overrides Bootstrap.

Fixes #20839.
2022-03-21 17:47:12 -07:00
Tim Abbott 7bc0e70693 recent topics: Revert time format changes for now.
As detailed in this conversation:

https://chat.zulip.org/#narrow/stream/137-feedback/topic/recent.20topics.20timestamps/near/1337670

This time format change is not working out as an improvement for at
least some users, myself included.

I think we do want to use some of the refinements attempted here (and
in particular, I'm keeping the new function with its nice test suite),
but I think it's better to revert now and fix forward in a future
release.

See #19775 for added background.
2022-03-21 17:37:32 -07:00
jai2201 040363f6c7 tippy: Render tooltip from tippy.js for stream_sorter_toggle.
Render tooltip to stream_sorter_toggle buttons using
appendTo method of tippy.js to ensure that tooltip doesn't
get hidden behind the parent container and is visible
completely.

Fixes #21329.
2022-03-21 17:35:04 -07:00
somesh202 cf5a70a958 settings: Fix buggy heading hover behavior for some settings panels.
Previously, hovering over the table headers in the code playground
table in the dark theme looked wrong. We were able to trace this issue
to the table-sticky-headers class not having been applied to this
table. The `alert-words` table was also affected and is fixed as well.

This also adds the `actions` class in alert-words table to fix the
inconsistency of the actions column in that table.

We've done an audit and these are the only two instances of either
bug in the HTML templates for a settings table.
2022-03-21 17:24:23 -07:00
Biki-das 64ec62b15c portico: Fixed responsive of integration request buttons.
Previously, these buttons looked broken in mobile size screens.

Fixes #20799.
2022-03-21 16:44:06 -07:00
Aman Agrawal 61cf9d1843 compose: Minor realignment of compose icons. 2022-03-21 16:41:18 -07:00
Alex Vandiver e2f4b284db docs: Remove teleport from example list of services.
This is not expected on generic Zulip servers.
2022-03-21 16:33:28 -07:00
Alex Vandiver f39ee5a16c docs: Remove references to supervisorctl (re)start all. 2022-03-21 16:33:28 -07:00
Alex Vandiver a4d0f03319 scripts: Switch to stop-server/restart-server.
stop-server and restart-server address all services which talk to the
database, and are thus more correct than restarting or stopping
everything in supervisor.

This is possible now that the previous commit ensures that the zulip
user can read the zulip installation directory during
`create-database`; previously, that directory was still owned by root
when `create-database` was run, whereas now it is in
`~zulip/deployments/`.
2022-03-21 16:33:28 -07:00
Alex Vandiver c0cc98c6a8 install: Re-order final steps.
Move database creation to immediately before database initialization;
this means it happens in a directory readable by the `zulip` user, as
well as placing it alongside similar operations.  It removes the check
for the `zulip::postgresql_common` Puppet class; instead it keeps the
check for `--no-init-db`, and switches to require
`zulip::app_frontend_base`.  This is a behavior change for any install
of `zulip::postgresql_common`-only classes, but that is not a common
form -- and such installs likely already pass `--no-init-db` because
they are warm spare replicas.

As a result, all non-`zulip::app_frontend_base` installs now skip
database initialization, even without `--no-init-db`.  This is clearly
correct for, e.g. Redis-only hosts, and makes clearer that the
frontend, not the database host, is responsible for database
initialization.
2022-03-21 16:33:28 -07:00
Alex Vandiver 394f1eadde setup: Rename postgresql-init-db to create-database.
The old name was confusingly similar to initialize-database.
2022-03-21 16:33:28 -07:00
Alex Vandiver 086c0328bd docs: Be explicit about how to skip database creation. 2022-03-21 16:33:28 -07:00
Lauryn Menard b70b925716 help_docs: Update `reading-strategies` for changed icon.
Removes reference to icon (previously a magnifying glass) in
text for filtering streams and replaces it with direction
for clicking on the STREAMS header in the left sidebar.
2022-03-21 16:14:07 -07:00
Lauryn Menard ca395227b2 help_docs: Update `mute-a-stream` for menu option text.
Corrects text in instructions for the menu option when
muting and unmuting streams.
2022-03-21 16:12:58 -07:00
Lauryn Menard ddab1d9b07 help_docs: Update `mobile-notifications` to use 'toggle'.
We'd like to use 'toggle' for consistency when referring to
check boxes in the help center documentation.
2022-03-21 16:12:20 -07:00
Lauryn Menard 210268f264 help_docs: Update `manage-inactive-streams` for settings header.
Correct out of date section header in display settings.
2022-03-21 16:12:20 -07:00
Lauryn Menard 1b1c479333 help_docs: Update `invite-new-users` for dropdown option.
Corrects an out of date dropdown option in organizational
settings for enabling email sign-up.
2022-03-21 16:12:20 -07:00
Lauryn Menard 517b2a5e10 help_docs: Update `email-notifications` for setting header.
Updates out of date reference to a Notifications setting header,
and also updates instructions for a check box to use 'toggle'.
2022-03-21 16:12:19 -07:00
Lauryn Menard 4f735aeb0e help_docs: Update `configure-default-view` to use 'toggle'.
We'd like to use 'toggle' for consistency when referring to
check boxes in help center documentation instructions.
2022-03-21 16:11:42 -07:00