Commit Graph

45160 Commits

Author SHA1 Message Date
Alex Vandiver 514801c509 puppet: Split out golang toolchain into its own manifest. 2021-11-19 15:29:28 -08:00
Sahil Batra f42b8f2fe1 misc: Replace "night theme" with "dark theme". 2021-11-19 11:49:59 -08:00
Sahil Batra 14b91043d2 misc: Replace "Day mode" with "Light theme". 2021-11-19 11:49:59 -08:00
Sahil Batra b72035abfe misc: Replace "Light mode" with "Light theme". 2021-11-19 11:49:59 -08:00
neiljp (Neil Pilgrim) 31798ab815 bot docs: Migrate script name from zulip-terminal to zulip-bot-shell.
This change is intended to reduce confusion between zulip-bot-shell
(test bots interactively without a server) and the zulip/zulip-terminal
project and its associated command (zulip-term).
2021-11-19 11:49:06 -08:00
Aman Agrawal ecb966c850 landing_page: Explicitly define transparent for gradients.
Safari interprets transparent as rgba(255, 255, 255, 0)
`transparent black` instead of rgba(0, 0, 0, 0).

We explicitly define transparent to help safari understand the
gradients.

This fixes the bug where our gradients look black on safari
on narrow screens.
2021-11-18 17:35:09 -08:00
Aman Agrawal fce9b2dbb4 features: Only change gradient height for feature page.
This gradient works well to display white colored text
on top of it on other pages. So, we only need this change on
features page.
2021-11-18 17:35:09 -08:00
Sahil Batra 89720c5d5a slash: Change description of /dark command in typeahead.
Fixes part of #20228.
2021-11-18 16:57:11 -08:00
Sahil Batra 5024ba98cd misc: Rename "dark mode" to "dark theme".
Fixes part of #20228.
2021-11-18 16:57:11 -08:00
Sahil Batra cf4f31291c misc: Rename "Night mode" to "Dark theme".
This commit renames "Night mode" to "Dark theme" in the
settings dropdown, help pages and api documentation.

Fixes part of #20228.
2021-11-18 16:55:35 -08:00
Sahil Batra 291f8e570d styles: Rename night_mode.css to dark_theme.css.
Fixes part of #20228.
2021-11-18 16:55:35 -08:00
Sahil Batra 08404f2824 help: Rename night-mode.md to dark-theme.md.
Fixes part of #20228.
2021-11-18 16:55:35 -08:00
Sahil Batra b61d48d94b settings: Rename "Automatic" option in color scheme dropdown.
This commit renames "Automatic" option in color scheme setting
dropdown to "Sync with compute". We do not change any variables
used in code just the text in the dropdown visible to user.

Fixes part of #20228.
2021-11-18 16:55:35 -08:00
Aman Agrawal 00d3b88257 realm: Enable spectator access by default in development.
Enable spectator access for test `zulip` realm in developement
setup.

Add option in `do_create_realm` to configure
`enable_spectator_access` field of `Realm`.
2021-11-18 12:55:12 -08:00
Aman Agrawal 2a922409aa realm: Add option to enable/disable spectator login.
We restrict access of messages from web public streams if
anonymous login is disabled via `enable_spectator_access`.

Display of `Anonymous login` button is now controlled by
the value of `enable_spectator_access`.

Admins can toggle `enable_spectator_access` via org settings in UI.
2021-11-18 12:55:12 -08:00
Aman Agrawal 28488d9939 test_decorators: Call mock_home via patched zulip_login_required.
This avoids a weird issue that when mocking both
zulip_login_required and home_real, home_realm doesn't stay
patched.
2021-11-18 12:55:12 -08:00
Sahil Batra 16d35f5b4e streams: Hide fake emails in stream creation form.
We hide fake emails in "People to add" section of
stream creation form and instead show user id
along with specifying that email is hidden.
2021-11-18 12:49:10 -08:00
Ankur c578f186b6
docs: Change chat.zulip.org links to zulip.com/developer-community.
This helps increase the probability that folks read the guidelines for how the
chat.zulip.org community works and what streams to use before arriving there.

Fixes #19827.
2021-11-17 22:25:55 -08:00
Tim Abbott 9da1934c8e deletion: Preserve deleted objects for 30 days rather than 7.
We had an incident where someone didn't notice for a week that they'd
accidentally enabled a 30-day message retention policy, and thus we
were unable to restore the deleted the content.

After some review of what other products do (E.g. Dropbox preserves
things in a restoreable state for 30 days) we're adjusting this
setting's default value to be substantially longer, to give more time
for users to notice their mistake and correct it before data is
irrevocably deleted.
2021-11-17 18:03:31 -08:00
Tim Abbott c64eebcc9e docs: Mention server_name in existing server nginx docs.
As discussed in the issue, it'd be a helpful hint for folks doing this
to mention this detail.

Fixes #16705.
2021-11-17 16:53:11 -08:00
Tim Abbott cc11743047 docs: Advertise docker-zulip on rqeuirements page.
We also remove coverage of 64-bit, since it's no longer a relevant
detail in 2021.

Fixes #18534.
2021-11-17 16:48:51 -08:00
Alex Vandiver 610a0b2d59 nagios: `pg_is_in_recovery()` is better to know replica/primary status.
It is possible to be in recovery, and downloading WAL logs from
archives, and not yet be replicating.  If one only checks the
streaming log status, it reports as "no replicas" which is technically
accurate but not a useful summation of the state of the replica.
2021-11-17 13:38:26 -08:00
Lauryn Menard 5300ad8ca0 api_documentation: Display `null` data type if potential value.
If null is a potential value of data type for a return value or
parameter in the API endpoint, then it is rendered as an option.

This currently relies on the 'nullable' setting in the OpenAPI spec
that was removed in the 3.1.0 release. If/when the OpenAPI version
is updated, then how the `data_type` for parameters and return values
is rendered will need to be reworked.

Fixes #20264.
2021-11-17 13:20:24 -08:00
Ganesh Pawar e7b9173ef5 default_language: Migrate modal to dialog_widget. 2021-11-16 17:02:29 -08:00
YashRE42 3a06025e5a stream_settings_overlay: Fix padding for right section headings.
At some point we must have made a change that caused the "create
stream" and "#stream name" headings to take up more vertical space,
resulting in the dividing line for the headings of the right side of
the subscription overlay to be miss-aligned with the same for the left
side. For the "create stream" panel, it also caused the scroll bar and
some content to be visible through the partially transparent bottom
section in night mode.

In this commit we reduce the padding for those headings so that things
don't look broken anymore.
2021-11-16 16:33:33 -08:00
YashRE42 b0be6a0f92 test_script: Reword provision warning not to assume running tests.
Previously, running `./tools/run-dev.py` when provision was required
would lead to a warning along the lines of:
```
Before we run tests, we make sure your provisioning version
is correct by looking at var/provision_version, which is at
version 165.1, and we compare it to the version in source
control (version.py), which is 165.2.

It looks like you checked out a branch that has added
dependencies beyond what you last provisioned. Your command
is likely to fail until you add dependencies by provisioning.

Do this: `./tools/provision`

If you really know what you are doing, use --skip-provision-check to
run anyway.
```

The assumption that we're trying to run tests might cause some
confusion, especially if its the first time you're seeing the
provision warning. Hence, we reword the first paragraph to avoid
making that assumption.

The second paragraph has also been slightly altered, since (1) it's
possible that we didn't checkout a different branch, but eg just
rebased with upstream and (2) we might not be on a VM.

The warning you'd get after this commit would be along the lines of:
```
Provisioning state check failed! This check compares
`var/provision_version` (currently 165.2) to the version in
source control (`version.py`), which is 164.6, to see if you
likely need to provision before this command can run
properly.

The branch you are currently on expects an older version of
dependencies than the version you provisioned last. This may
be ok, but it's likely that you either want to rebase your
branch on top of upstream/main or re-provision your machine.

Do this: `./tools/provision`

If you really know what you are doing, use --skip-provision-check to
run anyway.
```
or along the lines of:
```
Provisioning state check failed! This check compares
`var/provision_version` (currently 165.2) to the version in
source control (`version.py`), which is 167.2, to see if you
likely need to provision before this command can run
properly.

The branch you are currently on has added dependencies beyond
what you last provisioned. Your command is likely to fail
until you add dependencies by provisioning.

Do this: `./tools/provision`

If you really know what you are doing, use --skip-provision-check to
run anyway.
```
2021-11-16 16:29:34 -08:00
Alya Abbott bd1f3ced4f portico: Clarify on /jobs that openings are remote-friendly. 2021-11-16 16:23:13 -08:00
Alex Vandiver 83091cbc96 puppet: Swap the one use of the `cron` resource for an /etc/cron.d file.
The `cron` resource places its contents in the user's crontab, which
makes it unlike every other cron job that Zulip installs.

Switch to using `/etc/cron.d` files, like all other cron jobs.
2021-11-16 16:17:32 -08:00
Alex Vandiver 90e1a0400e puppet: Add a few more inter-resource dependencies.
None of these are important; they just express semantic dependencies.
2021-11-16 16:17:32 -08:00
Tim Abbott 864fb7c978 api docs: Sync list of realm settings with reality.
It turns out these were just wrong.  We fix a few things:
* Sort the list of settings so that it's possible to compare with reality.
* Deleted additional fields that don't actually exist.
* Fixed various fields missing past feature level updates.
2021-11-16 15:51:03 -08:00
Tim Abbott 744f96df61 api docs: Delete additional improperly documented update_dict fields.
As with 618d0788f0, these fields will
not appear in events requests.
2021-11-16 15:24:34 -08:00
Manan Rathi f3469ac3c8 todo_widget: Fix /todo checkboxes to match style used in menus.
Fixes #20212
2021-11-16 15:02:53 -08:00
Mateusz Mandera 7df5f8e6f5 docs: Update the /help/ instructions for setting up OneLogin SAML.
OneLogin has removed the old app. The new app is nearly identical, just
with some additional configurable settings, that we don't want to touch
anyway as the default are fine - and changing the default Parameters
that are set up, so we also update the screenshot to match how it looks
with the new app.
2021-11-16 14:58:17 -08:00
Tim Abbott 618d0788f0 api docs: Delete docs for nonexistent icon_file_size property.
This appears to have been confused with max_icon_file_size.
2021-11-16 14:57:50 -08:00
Tim Abbott cac8bcbc89 api docs: Fix incorrect realm_ prefixes in update_dict field names.
Neither of these fields use the `realm/update_dict` event type; they
use `realm/update`; we've attempted to clarify that in the previous
commit.

That reality means we don't have automated testing for these values,
and that meant that typos like these could slip through.
2021-11-16 14:57:50 -08:00
Tim Abbott e36f549ec6 api docs: Add more details around realm event types. 2021-11-16 14:57:50 -08:00
Mateusz Mandera 0abbb87155 auth: Include user_id in the params returned at the end of mobile flow.
The user id is a very useful piece of information that the mobile
client should have access to - instead of only getting the email. This
makes it much simpler to impleent clients that might be robust to
changes in email address.
2021-11-16 12:12:59 -08:00
Sahil Batra 9e979e9e66 models: Sort UserBaseSettings.property_types dict by key. 2021-11-16 12:01:04 -08:00
Sahil Batra 375b48f013 models: Sort UserBaseSettings.notification_settings_legacy dict by key. 2021-11-16 12:01:04 -08:00
Sahil Batra cae547cfd0 models: Sort Realm.property_types dict by key. 2021-11-16 12:01:04 -08:00
Alex Vandiver faeffa2466 queue_processors: Set a bounded prefetch size on rabbitmq queues.
RabbitMQ clients have a setting called prefetch[1], which controls how
many un-acknowledged events the server forwards to the local queue in
the client.  The default is 0; this means that when clients first
connect, the server must send them every message in the queue.

This itself may cause unbounded memory usage in the client, but also
has other detrimental effects.  While the client is attempting to
process the head of the queue, it may be unable to read from the TCP
socket at the rate that the server is sending to it -- filling the TCP
buffers, and causing the server's writes to block.  If the server
blocks for more than 30 seconds, it times out the send, and closes the
connection with:

```
closing AMQP connection <0.30902.126> (127.0.0.1:53870 -> 127.0.0.1:5672):
{writer,send_failed,{error,timeout}}
```

This is https://github.com/pika/pika/issues/753#issuecomment-318119222.

Set a prefetch limit of 100 messages, or the batch size, to better
handle queues which start with large numbers of outstanding events.

Setting prefetch=1 causes significant performance degradation in the
no-op queue worker, to 30% of the prefetch=0 performance.  Setting
prefetch=100 achieves 90% of the prefetch=0 performance, and higher
values offer only minor gains above that.  For batch workers, their
performance is not notably degraded by prefetch equal to their batch
size, and they cannot function on smaller prefetches than their batch
size.

We also set a 100-count prefetch on Tornado workers, as they are
potentially susceptible to the same effect.

[1] https://www.rabbitmq.com/confirms.html#channel-qos-prefetch
2021-11-16 11:48:50 -08:00
Alex Vandiver 7c3507feef queue: Allow passing down a prefetch count to pika. 2021-11-16 11:48:50 -08:00
Alex Vandiver 8d239f4984 tests: Move lambda in simulated_queue_client to help typing. 2021-11-16 11:48:50 -08:00
Alex Vandiver d1822b5630 tests: Move simulated_queue_client to the only test it is used in. 2021-11-16 11:48:50 -08:00
Alex Vandiver 64268f47e8 queue_processors: Drop unused current_queue_size, which was local size.
The `current_queue_size` key in the queue monitoring stats file was
the local queue size, not the global queue size -- d5a6b0f99a
renamed the function, but did not adjust the queue monitoring JSON,
despite the last use of it having been removed in cd9b194d88.

The function is still used to mark "we emptied our queue," and it
remains a reasonable metric for that.
2021-11-16 11:48:50 -08:00
Alex Vandiver 800e38016a queue_rate: Output to CSV, and run multiple prefetch values. 2021-11-16 11:48:50 -08:00
Alex Vandiver 49ad188449 rate_limit: Add a flag to lump all TOR exit node IPs together.
TOR users are legitimate users of the system; however, that system can
also be used for abuse -- specifically, by evading IP-based
rate-limiting.

For the purposes of IP-based rate-limiting, add a
RATE_LIMIT_TOR_TOGETHER flag, defaulting to false, which lumps all
requests from TOR exit nodes into the same bucket.  This may allow a
TOR user to deny other TOR users access to the find-my-account and
new-realm endpoints, but this is a low cost for cutting off a
significant potential abuse vector.

If enabled, the list of TOR exit nodes is fetched from their public
endpoint once per hour, via a cron job, and cached on disk.  Django
processes load this data from disk, and cache it in memcached.
Requests are spared from the burden of checking disk on failure via a
circuitbreaker, which trips of there are two failures in a row, and
only begins trying again after 10 minutes.
2021-11-16 11:42:00 -08:00
Mateusz Mandera 2d3d0f862a process_queue: Rename Threaded_worker to ThreadedWorker.
Threaded_worker doesn't fit the python naming convention we rely on.
2021-11-16 11:21:05 -08:00
Mateusz Mandera 3688ff38bd custom_check: Improve check for exit(...) call in management commands.
It's better for this to catch all exit(...) calls with non-zero exit
code, given the purpose is to catch all exits with failure, as opposed
to only exit(1).
2021-11-16 11:21:05 -08:00
Mateusz Mandera 7cc345d7b1 process_queue: Improve handling of exceptions in process_queue.
Unhandled exceptions propagating to process_queue were not caught there,
causing improper logging - errors didn't land in errors.log as expected.
Exceptions should be caught and explicitly logged by the process_queue
logger. Exceptions occurring during consuming events are caught and
handled inside the worker's logic - however those that happen while
setting up the worker were not addressed at all, and that's the core bug
we mean to address here.

Furthermore, in multi-threaded mode we want the autoreload mechanism to
be working - which it doesn't without catching the exceptions. The
correct approach is to - again - catch the exception, log it and then
send SIGUSR1 signal to trigger exit and autoreload.
2021-11-16 11:21:05 -08:00