Commit Graph

45321 Commits

Author SHA1 Message Date
Sahil Batra 828927b724 server_events_dispatch: Remove unnecessary 'realm_' prefix.
We do not have 'realm_' prefix to the settings used as keys
in realm_settings object, we directly use the setting name.

This commit removes the 'realm_' prefix from enable_spectator_access
setting.
2021-11-23 10:48:20 -08:00
Alex Vandiver 3efed5f1e6 queue_processors: Shut down background missedmessage_emails thread.
Python's behaviour on `sys.exit` is to wait for all non-daemon threads
to exit.  In the context of the missedmessage_emails worker, if any
work is pending, a non-daemon Timer thread exists, which is waiting
for 5 seconds.  As soon as that thread is serviced, it sets up another
5-second Timer, a process which repeats until all
ScheduledMessageNotificationEmail records have been handled.  This
likely takes two minutes, but may theoretically take up to a week
until the thread exits, and thus sys.exit can complete.

Supervisor only gives the process 30 seconds to shut down, so
something else must prevent this endless Timer.

When `stop` is called, take the lock so we can mutate the timer.
However, since `stop` may have been called from a signal handler, our
thread may _already_ have the lock.  As Python provides no way to know
if our thread is the one which has the lock, make the lock a
re-entrant one, allowing us to always try to take it.

With the lock in hand, cancel any outstanding timers.  A race exists
where the timer may not be able to be canceled because it has
finished, maybe_send_batched_emails has been called, and is itself
blocked on the lock.  Handle this case by timing out the thread join
in `stop()`, and signal the running thread to exit by unsetting the
timer event, which will be detected once it claims the lock.
2021-11-23 10:45:49 -08:00
Rahul Gurung acbeeac037
docs: Remove deprecated stash from example integrations. 2021-11-23 10:38:28 -08:00
Lauryn Menard 136234a7f8 documentation: Fix ordered list text wrapping in some /help articles.
Minor change to some of the `/help` md files to correct the text
wrapping of ordered lists in the content of the article.
2021-11-23 10:15:14 -08:00
Shlok Patel b3c58f454f api: Prevent special characters in topics.
Special characters, including `\r`, `\n`, and more esoteric codepoints
like non-characters, can negatively affect rendering and UI behaviour.

Check for, and prevent making new messages with, characters in the
Unicode categories of `Cc` (control characters), `Cs`, (surrogates),
and `Cn` (unassigned, non-characters).

Fixes #20128.
2021-11-22 22:09:06 -08:00
Aman Agrawal bf93ae1644 ui: Align bottom components in the same horizontal line. 2021-11-22 18:44:32 -08:00
Aman Agrawal 061e274716 compose: Add dividers.
These improves the visual organization of the compose controls.

This is consistent with the divider we have for the navbar.
2021-11-22 18:44:32 -08:00
Aman Agrawal 2b8aa5ce2d compose: Add formatting buttons to bottom of compose box.
This is a very frequently requested feature for organizations that are
new to Markdown, that brings Zulip's UI more in line with that of
competing projects and other markdown editors like the GitHub UI.
2021-11-22 18:44:31 -08:00
Aman Agrawal 2af933678c compose: Refactor bottom part of compose box.
* We use flexbox instead of `position: relative` to align elements.
* Increase clickable area of icons using more padding.
* Increase space between elements.
* Fix mobile compose box icon alignment.
2021-11-22 18:29:45 -08:00
Aman Agrawal 29cdd99220 giphy: Use single element for tooltip and popover.
This works surprisingly unlike my previous attempts to do so.

WARN: This is a pseudo commit and should only be merged with upcoming
compose box bottom refactoring commit since the css changes required
for this change are missing here and are not required after that
commit.
2021-11-22 18:29:45 -08:00
Aman Agrawal 3423797efa compose: Remove drafts link.
Since we have the drafts button in top left corner and we need space
to insert formatting buttons in the bottom of compose box; removing
drafts link makes sense.
2021-11-22 18:29:45 -08:00
Aman Agrawal 9d0dded959 composebox_typeahead: Don't propagate after handling tab.
After we set focus manually to the send button, `tab` propagation
should be stopped.
2021-11-22 18:29:45 -08:00
Julia Bichler 32f206e1e5 popovers: Add sidebar menu to delete all drafts.
This provides a convenient interface to hide all drafts.

Fixes #19360.

However, we may want to continue to implement a button in the drafts
overlay as well for doing this operation.
2021-11-22 17:01:47 -08:00
Johan Ehinger 4207f0a299 settings: Add UI setting option for controlling invite expiration time.
Add a UI for controlling invititation expiration time, supported
options are 1 day, 3 days, 10 days and 30 days.

Fixes #19680.
2021-11-22 16:40:14 -08:00
Tim Abbott a7898ed305 stream_data: Rename id_is_subscribed to is_subscribed. 2021-11-22 15:44:30 -08:00
Tim Abbott 53d9797044 stream_data: Rename is_subscribed to is_subscribed_by_name.
Generally the stream_data module aims to do lookup by ID, not name,
wherever possible, so we should represent that in the function names.
2021-11-22 15:44:30 -08:00
Tim Abbott 043d83b434 unread: Lookup subscription status by ID.
We prefer to use IDs, not stream names, to do lookups wherever both
are available.
2021-11-22 15:39:58 -08:00
Tim Abbott 3f4d66109b stream_data: Rename get_invite_only helper method.
Since this library is intended to primarily work by user ID, we want
to name all methods that do lookups by stream name explicitly.
2021-11-22 15:32:14 -08:00
Aman Agrawal d9338a68d1 compose: Show globe icon for web public streams.
Fixes #20285.
2021-11-22 15:31:22 -08:00
Aman Agrawal a7b2c7f7ea recipient_row: Show globe icon for web public streams. 2021-11-22 15:26:21 -08:00
Aman Agrawal e8ec42bf8a stream_bar: Rename update_lock_icon_for_stream. 2021-11-22 15:26:21 -08:00
Aman Agrawal 4e5e1a2542 recipient_row: Generalize class name for icon in recipient_row. 2021-11-22 15:26:21 -08:00
Aman Agrawal 883732a1d1 portico: Remove unused `moving-label` class.
This class is no longer used in these input fields.
2021-11-22 15:02:00 -08:00
Aman Agrawal a39d9ee4fd portico: Center align moving label text in input. 2021-11-22 15:02:00 -08:00
Ganesh Pawar 291aaf373f modal: Add support for modal event handlers. 2021-11-22 15:00:38 -08:00
Ganesh Pawar 8908ea99ec modal: Add support for linking the submit button to a form element. 2021-11-22 15:00:38 -08:00
Sahil Batra eef9dd963b zcommand: Replace "mode" with "theme" in the returned message.
This commit replaces "dark mode" and "light mode" with "dark theme"
and "light theme" in the message returned and shown in a little
popup in the UI, when color scheme settings are changed through
slash commands.
2021-11-22 14:59:16 -08:00
Sahil Batra 1cdc7306c6 slash: Change description of /light command in typeahead. 2021-11-22 14:59:16 -08:00
Lauryn Menard fee7b10db4 api_documentation: Fix `/update-subscription-settings` api doc.
Adds `wildcard_mentions_notify` as a property that can be updated
by the endpoint and removes mention of potential `null` value in
the return object because it is not possible.

Also cleans up the documentation of `in_home_view` legacy property
and updates the return object description to better reflect what
is actually returned.
2021-11-22 11:50:27 -08:00
Mateusz Mandera 8af7ffd9da rate_limit: Fix logging string when rate limiting email gateway.
realm.name is not the right "name" to log, we should use realm.subdomain
like everywhere else.
2021-11-22 10:28:56 -08:00
Aman Agrawal 2c4af0e111 integration: Fix overflowing display name on open collective page.
Removed extra text to stop display name from flowing out of logo
container.
2021-11-21 21:05:06 -08:00
Aman Agrawal 334f046842 events: Don't send data related to custom profile field to spectator.
Since spectators can't access personal profile settings and
can't view profile for other users. Hence, we don't send realm
custom profile field data and user's profile data to spectators.

Fixes #20301.
2021-11-20 21:01:38 -08:00
YashRE42 717c4ae603 refactor: Remove duplication from reactions.add_clean_reaction.
In commit 3d86267041 we add logic to
`/shared/emoji.js` which duplicated some of the logic in this
function. Since this isn't desirable, we remove the duplicate logic
here and instead just call `emoji.get_emoji_details_for_rendering`.
2021-11-20 20:57:54 -08:00
YashRE42 71a4985ebc refactor: Remove duplication from reactions.view.insert_new_reaction.
In commit 3d86267041 we add logic to
`/shared/emoji.js` which duplicated some of the logic in this
function. Since this isn't desirable, we remove the duplicate logic
here and instead just call `emoji.get_emoji_details_for_rendering`.
2021-11-20 20:57:54 -08:00
YashRE42 bfc705be8f reactions_tests: Remove misleading emoji_params.realm_emoji.zulip obj.
Previously, this test file had an object that encouraged one to
believe that it is defining a realm_emoji with the name, id = "zulip"
and a custom source url, but this is false.
The "zulip" emoji is a special case that's added to our data
structures by the `/shared/emoji.js` code.
Notice how the test never asserts that the returned url is equal to
the source_url defined, it just asserts that the source url is equal
to "/static/generated/emoji/images/emoji/unicode/zulip.png" (which is
the value defined in `/share/emoji.js`).

Hence, we remove this object and replace any references to it with the
values defined in `/shared/emoji.js`.
2021-11-20 20:57:54 -08:00
YashRE42 263a79738f user_status: Fix status emoji handling of deactivated custom emoji.
Previously, if a user had a realm emoji set as their status emoji and
someone deleted the realm emoji, the app would fail to initialize,
because of the error we throw from `./shared/js/emoji.js`.

This commit fixes this by just displaying the deactivated emoji,
similar to how we do when realm_emoji used as reactions are deleted.

As part of the fix, we add a function get_emoji_details_for_rendering,
which duplicates some of the logic used in `reactions.js`, we can
refactor to remove the duplication in `reactions.js` in future
commits.

Note that the following behaviour is a part of our design:
If a user sets their emoji to a particular realm emoji, say for
example "octo-ninja", and "octo-ninja" was then deleted, and a new
emoji was added with the name "octo-ninja", the user's status emoji
would change to show the new emoji instead of the deleted emoji.

Also note that in the `user_status.js` node test, we were able to
change the name for the 991 realm_emoji because it had not been
previously used anywhere in the test (possibly added as just a copy
paste artifact?).

Fixes: #20274.

emoji: Use reaction_type parameter to analyze emoji.
2021-11-20 20:57:54 -08:00
YashRE42 85a566aa80 emoji: Add code comment complaining about "zulip" emoji special case. 2021-11-20 20:57:54 -08:00
Alex Vandiver 3455fc137a upgrade-postgresql: Check for extension upgrade steps. 2021-11-20 07:13:50 -08:00
Alex Vandiver 73055255bb ci: Test pgroonga installation. 2021-11-20 07:13:50 -08:00
Alex Vandiver 86cf3be39f puppet: Fix pgroonga init for custom database names and users. 2021-11-20 07:13:50 -08:00
Alya Abbott 3d31bd79be
settings: Rename "public export" to "export of public data". 2021-11-19 17:36:29 -08:00
Alya Abbott 9b63f6eb5d
user docs: Add documentation page for web public streams.
As noted in the /help/ docs, this feature isn't fully implemented yet.
However, it's useful to merge this documentation, which explains how the
feature will work, to help make sure we've thought through all the details
around how this feature should work and be presented to users as we 
plan to final changes to implement it.

This commit does not edit other help pages to reflect the existence of
web public streams.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2021-11-19 16:10:59 -08:00
Alex Vandiver c514feaa22 puppet: Default go-camo to listening on localhost for standalone deploys.
The default in the previous commit, inherited from camo, was to bind
to 0.0.0.0:9292.  In standalone deployments, camo is deployed on the
same host as the nginx reverse proxy, and as such there is no need to
open it up to other IPs.

Make `zulip::camo` take an optional parameter, which allows overriding
it in puppet, but skips a `zulip.conf` setting for it, since it is
unlikely to be adjust by most users.
2021-11-19 15:58:26 -08:00
Alex Vandiver b982222e03 camo: Replace with go-camo implementation.
The upstream of the `camo` repository[1] has been unmaintained for
several years, and is now archived by the owner.  Additionally, it has
a number of limitations:
 - It is installed as a sysinit service, which does not run under
   Docker
 - It does not prevent access to internal IPs, like 127.0.0.1
 - It does not respect standard `HTTP_proxy` environment variables,
   making it unable to use Smokescreen to prevent the prior flaw
 - It occasionally just crashes, and thus must have a cron job to
   restart it.

Swap camo out for the drop-in replacement go-camo[2], which has the
same external API, requiring not changes to Django code, but is more
maintained.  Additionally, it resolves all of the above complaints.

go-camo is not configured to use Smokescreen as a proxy, because its
own private-IP filtering prevents using a proxy which lies within that
IP space.  It is also unclear if the addition of Smokescreen would
provide any additional protection over the existing IP address
restrictions in go-camo.

go-camo has a subset of the security headers that our nginx reverse
proxy sets, and which camo set; provide the missing headers with `-H`
to ensure that go-camo, if exposed from behind some other non-nginx
load-balancer, still provides the necessary security headers.

Fixes #18351 by moving to supervisor.
Fixes zulip/docker-zulip#298 also by moving to supervisor.

[1] https://github.com/atmos/camo
[2] https://github.com/cactus/go-camo
2021-11-19 15:58:26 -08:00
Alex Vandiver c33562f0a8 puppet: Default to installing smokescreen on application frontends.
This is an additional security hardening step, to make Zulip default
to preventing SSRF attacks.  The overhead of running Smokescreen is
minimal, and there is no reason to force deployments to take
additional steps in order to secure themselves against SSRF attacks.

Deployments which already have a different external proxy configured
will not gain a local Smokescreen installation, and running without
Smokescreen is supported by explicitly unsetting the `host` or `port`
values in `/etc/zulip/zulip.conf`.
2021-11-19 15:29:28 -08:00
Alex Vandiver 44f1ea6bae puppet: Split smokescreen into a non-profile version.
In a subsequent commit, we intend to include it from
`zulip::app_frontend_base`, which is a layering violation if it only
exists in the form of a profile.
2021-11-19 15:29:28 -08:00
Alex Vandiver c2ed3c22b5 puppet: Remove unused smokescreen symlink. 2021-11-19 15:29:28 -08:00
Alex Vandiver 47e16a5d41 puppet: Tidy old smokescreen binaries. 2021-11-19 15:29:28 -08:00
Alex Vandiver 239ac8413e puppet: Embed golang version into binary path, to rebuild on new golang.
This will cause the output binary path to be sensitive to golang
version, causing it to be rebuilt on new golang, and an updated
supervisor config file written out, and thus supervisor also
restarted.
2021-11-19 15:29:28 -08:00
Alex Vandiver 216eeba2dd puppet: Factor out smokescreen binary path. 2021-11-19 15:29:28 -08:00