Commit Graph

45901 Commits

Author SHA1 Message Date
Raghav Luthra 4b8cb0a8a9 docs: Uncapitalize the name for nginx.
This matches how nginx refers to itself on its own website and
documentation.

Fixes #20887.
2022-01-25 11:17:51 -08:00
Aman Agrawal ec86290e2a tippy: Fix persistent message action tooltips on icons after blur.
When users tabs through the message action icons, they used to
persist even when the focus is not on them. We manually
destroy them on blur event since tippy has some issue with
handling elements with opacity hiding effect.
2022-01-25 11:15:30 -08:00
Alex Vandiver 7ec2a2cde7 version: Update version and changelog after 4.9 release. 2022-01-24 18:46:46 -08:00
Alex Vandiver a5496f4098 CVE-2021-43799: Set a secure Erlang cookie.
The RabbitMQ docs state ([1]):

    RabbitMQ nodes and CLI tools (e.g. rabbitmqctl) use a cookie to
    determine whether they are allowed to communicate with each
    other. [...] The cookie is just a string of alphanumeric
    characters up to 255 characters in size. It is usually stored in a
    local file.

...and goes on to state (emphasis ours):

    If the file does not exist, Erlang VM will try to create one with
    a randomly generated value when the RabbitMQ server starts
    up. Using such generated cookie files are **appropriate in
    development environments only.**

The auto-generated cookie does not use cryptographic sources of
randomness, and generates 20 characters of `[A-Z]`.  Because of a
semi-predictable seed, the entropy of this password is thus less than
the idealized 26^20 = 94 bits of entropy; in actuality, it is 36 bits
of entropy, or potentially as low as 20 if the performance of the
server is known.

These sizes are well within the scope of remote brute-force attacks.

On provision, install, and upgrade, replace the default insecure
20-character Erlang cookie with a cryptographically secure
255-character string (the max length allowed).

[1] https://www.rabbitmq.com/clustering.html#erlang-cookie
2022-01-25 02:13:53 +00:00
Alex Vandiver 93a344fc3c configure-rabbitmq: Set -u, and not -x. 2022-01-25 01:52:36 +00:00
Alex Vandiver ece96c9729 configure-rabbitmq: Factor out sudo, instead of rabbitmqctl. 2022-01-25 01:52:36 +00:00
Alex Vandiver bd7deed691 upgrade: Show output from (re)starting zulip.
5c450afd2d, in ancient history, switched from `check_call` to
`check_output` and throwing away its result.

Use check_call, so that we show the steps to (re)starting the server.
2022-01-25 01:52:34 +00:00
Alex Vandiver e705883857 CVE-2021-43799: During upgrades, restart rabbitmq if necessary.
Check if it is listening on a public interface on port 25672, and if
so shut it down so it can pick up the new configuration.
2022-01-25 01:51:56 +00:00
Alex Vandiver da5201b986 upgrade: Make calling shutdown_server twice, only try once. 2022-01-25 01:48:05 +00:00
Alex Vandiver a46f6df91e CVE-2021-43799: Write rabbitmq configuration before starting.
Zulip writes a `rabbitmq.config` configuration file which locks down
RabbitMQ to listen only on localhost:5672, as well as the RabbitMQ
distribution port, on localhost:25672.

The "distribution port" is part of Erlang's clustering configuration;
while it is documented that the protocol is fundamentally
insecure ([1], [2]) and can result in remote arbitrary execution of
code, by default the RabbitMQ configuration on Debian and Ubuntu
leaves it publicly accessible, with weak credentials.

The configuration file that Zulip writes, while effective, is only
written _after_ the package has been installed and the service
started, which leaves the port exposed until RabbitMQ or system
restart.

Ensure that rabbitmq's `/etc/rabbitmq/rabbitmq.config` is written
before rabbitmq is installed or starts, and that changes to that file
trigger a restart of the service, such that the ports are only ever
bound to localhost.  This does not mitigate existing installs, since
it does not force a rabbitmq restart.

[1] https://www.erlang.org/doc/apps/erts/erl_dist_protocol.html
[2] https://www.erlang.org/doc/reference_manual/distributed.html#distributed-erlang-system
2022-01-25 01:48:05 +00:00
Alex Vandiver 43d63bd5a1 puppet: Always set the RabbitMQ nodename to zulip@localhost.
This is required in order to lock down the RabbitMQ port to only
listen on localhost.  If the nodename is `rabbit@hostname`, in most
circumstances the hostname will resolve to an external IP, which the
rabbitmq port will not be bound to.

Installs which used `rabbit@hostname`, due to RabbitMQ having been
installed before Zulip, would not have functioned if the host or
RabbitMQ service was restarted, as the localhost restrictions in the
RabbitMQ configuration would have made rabbitmqctl (and Zulip cron
jobs that call it) unable to find the rabbitmq server.

The previous commit ensures that configure-rabbitmq is re-run after
the nodename has changed.  However, rabbitmq needs to be stopped
before `rabbitmq-env.conf` is changed; we use an `onlyif` on an `exec`
to print the warning about the node change, and let the subsequent
config change and notify of the service and configure-rabbitmq to
complete the re-configuration.
2022-01-25 01:48:02 +00:00
Alex Vandiver 3bfcfeac24 puppet: Run configure-rabbitmq on nodename change.
`/etc/rabbitmq/rabbitmq-env.conf` sets the nodename; anytime the
nodename changes, the backing database changes, and this requires
re-creating the rabbitmq users and permissions.

Trigger this in puppet by running configure-rabbitmq after the file
changes.
2022-01-25 01:46:51 +00:00
Alex Vandiver b6cd89440e setup: Remove unused RABBITMQ_NODE.
This reverts commit 889547ff5e.  It is
unused in the Docker container, as the configurtaion of the `zulip`
user in the rabbitmq node is done via environment variables.  The
Zulip host in that context does not have `rabbitmqctl` installed, and
would have needed to know the Erlang cookie to be able to run these
commands.
2022-01-25 01:46:51 +00:00
Alex Vandiver 694c4dfe8f puppet: Admit we leave epmd port 4369 open on all interfaces.
The Erlang `epmd` daemon listens on port 4369, and provides
information (without authentication) about which Erlang processes are
listening on what ports.  This information is not itself a
vulnerability, but may provide information for remote attackers about
what local Erlang services (such as `rabbitmq-server`) are running,
and where.

`epmd` supports an `ERL_EPMD_ADDRESS` environment variable to limit
which interfaces it binds on.  While this environment variable is set
in `/etc/default/rabbitmq-server`, Zulip unfortunately attempts to
start `epmd` using an explicit `exec` block, which ignores those
settings.

Regardless, this lack of `ERL_EPMD_ADDRESS` variable only controls
`epmd`'s startup upon first installation.  Upon reboot, there are two
ways in which `epmd` might be started, neither of which respect
`ERL_EPMD_ADDRESS`:

 - On Focal, an `epmd` service exists and is activated, which uses
   systemd's configuration to choose which interfaces to bind on, and
   thus `ERL_EPMD_ADDRESS` is irrelevant.

 - On Bionic (and Focal, due to a broken dependency from
   `rabbitmq-server` to `epmd@` instead of `epmd`, which may lead to
   the explicit `epmd` service losing a race), `epmd` is started by
   `rabbitmq-server` when it does not detect a running instance.
   Unfortunately, only `/etc/init.d/rabbitmq-server` would respects
   `/etc/default/rabbitmq-server` -- and it defers the actual startup
   to using systemd, which does not pass the environment variable
   down.  Thus, `ERL_EPMD_ADDRESS` is also irrelevant here.

We unfortunately cannot limit `epmd` to only listening on localhost,
due to a number of overlapping bugs and limitations:

 - Manually starting `epmd` with `-address 127.0.0.1` silently fails
   to start on hosts with IPv6 disabled, due to an Erlang bug ([1],
   [2]).

 - The dependencies of the systemd `rabbitmq-server` service can be
   fixed to include the `epmd` service, and systemd can be made to
   bind to `127.0.0.1:4369` and pass that socket to `epmd`, bypassing
   the above bug.  However, the startup of this service is not
   guaranteed, because it races with other sources of `epmd` (see
   below).

 - Any process that runs `rabbitmqctl` results in `epmd` being started
   if one is not currently running; these instances do not respect any
   environment variables as to which addresses to bind on.  This is
   also triggered by `service rabbitmq-server status`, as well as
   various Zulip cron jobs which inspect the rabbitmq queues.  As
   such, it is difficult-to-impossible to ensure that some other
   `epmd` process will not win the race and open the port on all
   interfaces.

Since the only known exposure from leaving port 4369 open is
information that rabbitmq is running on the host, and the complexity
of adjusting this to only bind on localhost is high, we remove the
setting which does not address the problem, and document that the port
is left open, and should be protected via system-level or
network-level firewalls.

[1]: https://bugs.launchpad.net/ubuntu/+source/erlang/+bug/1374109
[2]: https://github.com/erlang/otp/issues/4820
2022-01-25 01:46:51 +00:00
Alex Vandiver 2713e90eaf puppet: Remove rabbitmq_mochiweb configuration.
mochiweb was renamed to web_dispatch in RabbitMQ 3.8.0, and the plugin
is not enabled.  Nor does this control the management interface, which
would listen on port 15672.
2022-01-25 01:46:51 +00:00
N-Shar-ma a003fe0246 compose: Fix bug where stream color didn't update on mouse selection.
Called the `decorate` function to update stream color in the compose
box on `change` instead of `blur`.

On clicking on a stream option, the input box for the stream name
remained in focus, hence decorate wasn't triggered on blur. Using
the change event instead, ensures that decorate will be called
anytime the stream is changed.

Fixes: #20871
2022-01-24 17:14:47 -08:00
Anders Kaseorg 48024097c4 install-shfmt: Upgrade shfmt from 3.4.1 to 3.4.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 15:55:38 -08:00
Anders Kaseorg 21548ff7c0 install-node: Upgrade Node.js from 16.13.1 to 16.13.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 15:55:38 -08:00
Anders Kaseorg a9dea6ee2d dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 15:55:38 -08:00
Anders Kaseorg 44f9aaec32 eslint: Fix unicorn/prefer-dom-node-dataset.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 15:55:38 -08:00
Anders Kaseorg 392676d4da js: Avoid _.get.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 15:54:21 -08:00
Anders Kaseorg cc40352541 js: Avoid _.first, _.last, _.initial, _.tail.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2022-01-24 15:54:21 -08:00
Anders Kaseorg 42931e69e6 js: Convert _.isFunction to typeof.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 15:54:21 -08:00
Anders Kaseorg 57cc5cb25a js: Convert _.without to filter or other logic.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2022-01-24 15:54:21 -08:00
Anders Kaseorg e2290ef0de js: Convert _.uniq(a) to Array.from(new Set(a)).
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2022-01-24 15:54:21 -08:00
Alex Vandiver 2fc156e556 ci: Cache with the OS name, not the job name.
The job name is just the constant `production_build`.  Renaming it to
have the OS in the key ensures that it is not shared across OS'es (for
instance between `4.x` and `main`, which are now bionic and buster,
respectively), and also allows it to share caches with the install
step, which uses the OS name in that place.
2022-01-24 14:29:49 -08:00
Alya Abbott 669010494e portico: Update contributor count from 700 to 1000.
Note: I did not check whether we have numbers other than 700 that also
need to be updated.
2022-01-24 12:41:49 -08:00
Alex Vandiver a3adaf4aa3 puppet: Fix standalone certbot configurations.
This addresses the problems mentioned in the previous commit, but for
existing installations which have `authenticator = standalone` in
their configurations.

This reconfigures all hostnames in certbot to use the webroot
authenticator, and attempts to force-renew their certificates.
Force-renewal is necessary because certbot contains no way to merely
update the configuration.  Let's Encrypt allows for multiple extra
renewals per week, so this is a reasonable cost.

Because the certbot configuration is `configobj`, and not
`configparser`, we have no way to easily parse to determine if webroot
is in use; additionally, `certbot certificates` does not provide this
information.  We use `grep`, on the assumption that this will catch
nearly all cases.

It is possible that this will find `authenticator = standalone`
certificates which are managed by Certbot, but not Zulip certificates.
These certificates would also fail to renew while Zulip is running, so
switching them to use the Zulip webroot would still be an improvement.

Fixes #20593.
2022-01-24 12:13:44 -08:00
Alex Vandiver 76ce8631c0 setup: Install a temporary certificate, before certbot runs.
Installing certbot with --method=standalone means that the
configuration file will be written to assume that the standalone
method will be used going forward.  Since nginx will be running,
attempts to renew the certificate will fail.

Install a temporary self-signed certificate, just to allow nginx to
start, and then follow up (after applying puppet to start nginx) with
the call to setup-certbot, which will use the webroot authenticator.

The `setup-certbot --method=standalone` option is left intact, for use
in development environments.

Fixes part of #20593; it does not address installs which were
previously improperly configured with `authenticator = standalone`.
2022-01-24 12:13:44 -08:00
N-Shar-ma 6d2de28bf0 todo_widget: Fix bug where new todos sometimes replaced old todos.
Initialised `this.me` for the TaskData using a constructor to the
current user id.

The bug was caused due to `this.me` never being initialised, and hence
`idx` wasn't incremented on page reload, which resulted in duplicate
`data-key` attributes and hence new todos overwriting older todos with
the same `data-key`

Fixes: #20698
2022-01-24 11:25:32 -08:00
Lauryn Menard e479acc809 api_docs: Add line break before return value description text.
Adds a line break before the descriptive text for return
values and events in the api documentation in order to
help with readability of descriptions with multiple
paragraphs of descriptive text.

Adjustments made to the CSS of list items in unordered
lists to visually group the first paragraph of text
to any following paragraphs or unordered lists.
2022-01-24 10:02:02 -08:00
Anders Kaseorg c6b372b471 js: Simplify indexing arrays from the end with Array#at.
https://github.com/tc39/proposal-relative-indexing-method

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 09:58:54 -08:00
Anders Kaseorg 0b03628324 js: Shorten Object.prototype.hasOwnProperty.call to Object.hasOwn.
https://github.com/tc39/proposal-accessible-object-hasownproperty

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 09:57:09 -08:00
Anders Kaseorg 4922632601 mypy: Add types-beautifulsoup4.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 23:39:40 -08:00
Anders Kaseorg 8d9fe9cfb0 mypy: Add types-stripe.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:47:30 -08:00
Anders Kaseorg de1df81ef6 test_stripe: Convert "".format to Python 3.6 f-string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:17:02 -08:00
Anders Kaseorg 2caeb38e9e python: Replace IOError with OSError.
IOError is an alias for OSError in Python ≥ 3.3.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:17:02 -08:00
Anders Kaseorg 3437e38612 string_validation: Use set comprehension syntax.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:17:02 -08:00
Anders Kaseorg 97e4e9886c python: Replace universal_newlines with text.
This is supported in Python ≥ 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:16:01 -08:00
Anders Kaseorg fd6d048efd mypy: Remove unused overrides.
soupsieve has types; sphinx_rtd_theme is no longer directly imported.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:15:03 -08:00
Anders Kaseorg b729f00fc2 test_upload: Uncomment subTest contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:43 -08:00
Anders Kaseorg 5fb935bf60 python: Replace deprecated urllib3 BACKOFF_MAX.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg 2612f57d51 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg ee71650908 python: Replace requests.packages.urllib3 alias with urllib3.
requests stopped vendoring urllib3 in 2.16.0 (2017-05-26).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg 18aa4f4df4 tornado: Replace deprecated urllib3 method_whitelist.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Sahil Batra fd1d4e101b overlay: Fix bug of information overlay not closing when text is selected.
There was a bug where information overlay was not closing on clicking
"x" when some text was selected. This was due to document.getSelection().type
returning "Range" and we do not close the modal in that case as per the code
added in 081d74141b.

As the "x" icon was button, the document.getSelection().type was
still returning "Range" for the text selected, but when the "x"
icon is inside a span, as in settings overlay, clicking on "x"
deselects the already selected text and selection type is not
"Range" and thus modal is closed.

This commit also improves the vertical alignment of "x".

Fixes #20645.
2022-01-21 17:39:01 -08:00
Anders Kaseorg a58a71ef43 Remove Ubuntu 18.04 support.
As a consequence:

• Bump minimum supported Python version to 3.7.
• Move Vagrant environment to Debian 10, which has Python 3.7.
• Move CI frontend tests to Debian 10.
• Move production build test to Debian 10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 17:26:14 -08:00
Anders Kaseorg d035efd467 ci: Test upgrade-postgresql on Ubuntu 20.04.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 17:26:14 -08:00
Alex Vandiver 3bbe5c1110 puppet: Put comments on iptables lines.
In addition to documenting the rules.v4 and rules.v6 files slightly,
these comments show up in `iptables -L`:

```
root@hostname:~# iptables -L INPUT
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
LOGDROP    all  --  anywhere             localhost/8
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh /* ssh */
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:3000 /* grafana */
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:9100 /* node_exporter */
LOGDROP    all  --  anywhere             anywhere
```
2022-01-21 16:46:14 -08:00
Aman Agrawal c4efc97d5a compose: Extend empty compose navigation logic to recipient boxes.
Navigation key presses like `Up` and `PageUp` with an empty recipient
boxes will now close the compose and propagate the keypress to the message
list or recent topics, depending upon the active view.

This extends behavior we've had for a long time with focus in the
compose box itself.
2022-01-21 16:40:13 -08:00