Commit Graph

1482 Commits

Author SHA1 Message Date
Alex Vandiver a5a898ba21 install-aws-server: Switch to 24.04. 2024-08-20 10:04:37 -07:00
Alex Vandiver 1e5c0dc259 kandra: Bump Teleport version to 16. 2024-08-16 08:51:15 -07:00
Shubham Padia d7889221db node_cache: Run pnpm install if node_modules not present in help-beta.
We need this check when switching between branches without `help-beta`
package. `node_modules` will be removed when working on a non `help-beta`
branch, but if `node_modules/.pnpm/lock.yaml` has not been updated by that
branch, we will end up in a situation where we might not have `node_modules`
even when we run the provision command.

We might not need this check when the `help-beta` initial folder
has been merged for a week or two, where almost all active PRs
would have been rebased upon main, making switching branches an ease.
2024-08-02 18:18:01 -07:00
Mateusz Mandera aaca394813 presence: Remove the queue worker. 2024-07-31 16:46:42 -07:00
Alex Vandiver 4e28e1d3ff upgrade-zulip: Factor out a script to run hooks. 2024-07-26 14:31:55 -07:00
Alex Vandiver 1f2f28bbf5 upgrade-zulip-stage-2: Be more specific about OS upgrade steps. 2024-07-26 11:14:11 -07:00
Tim Abbott 54c2c02011 thumbnail: Add support for multiple queue workers.
There's no need for sharding, but this allows one to spend a bit of
extra memory to reduce image-processing latency when bursts of images
are uploaded at once.
2024-07-21 19:15:43 -07:00
Anders Kaseorg 4707edb9d8 install-node: Upgrade Node.js from 20.15.0 to 20.15.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 21:39:03 -07:00
Anders Kaseorg c8eaceff21 flush-memcached: Explicitly close memcached connection.
Fixes warnings like “ResourceWarning: unclosed <socket.socket fd=3,
family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
laddr=('127.0.0.1', 33332), raddr=('127.0.0.1', 11211)>” with warnings
enabled.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-16 14:50:09 -07:00
Alex Vandiver 2e38f426f4 upload: Generate thumbnails when images are uploaded.
A new table is created to track which path_id attachments are images,
and for those their metadata, and which thumbnails have been created.
Using path_id as the effective primary key lets us ignore if the
attachment is archived or not, saving some foreign key messes.

A new worker is added to observe events when rows are added to this
table, and to generate and store thumbnails for those images in
differing sizes and formats.
2024-07-16 13:22:15 -07:00
Anders Kaseorg 48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 3f29bc42b1 ruff: Fix B905 `zip()` without an explicit `strict=` parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Lauryn Menard 80b86c7b16 documentation: Fix "you you" typos in docs/comments/logs. 2024-07-02 11:57:44 -07:00
Anders Kaseorg 5af8dfda3e ruff: Fix FURB167 Use of regular expression alias.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-30 22:37:15 -07:00
Anders Kaseorg 95f5d298de install-node: Upgrade Node.js from 20.13.1 to 20.15.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-26 21:34:18 -07:00
Alex Vandiver 16f8d12285 requirements: Remove now-unused pillow. 2024-06-26 16:42:59 -07:00
Alex Vandiver 475d4800f9 thumbnailing: Add pyvips dependency. 2024-06-20 23:06:08 -04:00
Sayam Samal a7772f86a2 docs: Update CZO links from stream to channel in docs and comments.
Updates all the https://chat.zulip.org/#narrow/stream/ links in the
docs and comments to use the new /channel/ path. All these links are
for documentation/reference purposes only and thus, can be bulk-updated.

This commit is a part of the effort to rename stream to channel.
2024-06-11 10:44:31 -07:00
Alex Vandiver 1b10bf1921 check-rabbitmq-consumers: Add missing f on format string. 2024-06-05 08:59:50 -07:00
Alex Vandiver 5814583694 rabbitmq: Support non-/ vhosts.
Fixes: #30255.
2024-05-31 15:41:19 -07:00
Alex Vandiver a4ff6f9ae5 configure-rabbitmq: Switch to await_startup. 2024-05-31 15:41:19 -07:00
Anders Kaseorg 659d9e39db install-node: Upgrade Node.js from 20.13.2 to 20.13.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-26 19:26:06 -07:00
Alex Vandiver f246b82f67 puppet: Factor out pattern of writing a nagios state file atomically. 2024-05-24 11:31:25 -07:00
Alex Vandiver 2218de0399 management: Disable Sentry for management commands run interactively.
This adds `--automated` and `--no-automated` flags to all Zulip
management commands, whose default is based on if STDIN is a TTY.
This enables cron jobs and supervisor commands to continue to report
to Sentry, and manually-run commands (when reporting to Sentry does
not provide value, since the user can see them) to not.

Note that this only applies to Zulip commands -- core Django
commands (e.g. `./manage.py`) do not grow support for `--automated`
and will always report exceptions to Sentry.

`manage.py` subcommands in the `upgrade` and `restart-server` paths
are marked as `--automated`, since those may be run semi-unattended,
and they are useful to log to Sentry.
2024-05-24 10:30:16 -07:00
Anders Kaseorg b545abe1e2 typos: Fix typos caught by mwic.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-20 13:55:00 -07:00
Anders Kaseorg 18579755a8 apt-repos: Deprioritize libgroonga0 14.0.3-1.
It depends libarrow1600, which is missing from the PGroonga repository
on Debian 12.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-05-10 12:04:56 -07:00
Tim Abbott 0a756c652c push_notifications: Shard mobile push notifications. 2024-05-02 14:25:10 -07:00
Alex Vandiver 9dfaa83aa8 invites: Remove invites worker, make confirmation object in-process.
The "invites" worker exists to do two things -- make a Confirmation
object, and send the outgoing email.  Making the Confirmation object
in a background process from where the PreregistrationUser is created
temporarily leaves the PreregistrationUser in invalid state, and
results in 500's, and the user not immediately seeing the sent
invitation.  That the "invites" worker also wants to create the
Confirmation object means that "resending" an invite invalidates the
URL in the previous email, which can be confusing to the user.

Moving the Confirmation creation to the same transaction solves both
of these issues, and leaves the "invites" worker with nothing to do
but send the email; as such, we remove it entirely, and use the
existing "email_senders" worker to send the invites.  The volume of
invites is small enough that this will not affect other uses of that
worker.

Fixes: #21306
Fixes: #24275
2024-05-02 14:23:04 -07:00
Anders Kaseorg 0c7fd45c83 install-node: Upgrade Node.js from 20.12.0 to 20.12.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-25 16:46:03 -07:00
Alex Vandiver 7f4bc05f8e install: Check for a supported version of the OS in shell.
Testing for it in Python means that we have to worry about keeping the
`upgrade-zulip-stage-2` backwards-compatible with all versions of
Python which we could ever be upgrading from -- which is all of them.

Factor out the "supported operating systems" check, and share it
between upgrade and install codepaths.
2024-04-16 11:13:33 -07:00
Alex Vandiver a853b8849f install: Actually add --puppet-classes argument.
7187146422 only added it to the internal parsing and the help, not
the getopt invocation which would make the command accept the
argument.
2024-04-11 15:34:00 -07:00
Anders Kaseorg 36aa0177bd Revert "apt-repos: Disallow libmagic1 1:5.45-2 (Ubuntu 24.04) again."
This reverts commit 13e28fc3ac.
2024-04-10 16:07:25 -07:00
Alex Vandiver 7187146422 install: Move PUPPET_CLASSES env var to --puppet-classes argument.
`--no-init-db` is used to silence the need for `--hostname` and
`--email` arguments; it is a proxy for "this is not a frontend host."
We would ideally like to use `has_class` to know if the user's
provided puppet classes are include an `app_frontend`, and thus
`--hostname` and `--email` are required -- but doing that requires
several other steps, and we would like this feedback to be immediate.

We make the presence of `--puppet-classes` equivalent to
`--no-init-db`, since nearly every configuration with
`--puppet-classes` does not install both a database and a frontend,
which is what is required to initialize a database.
2024-04-04 16:49:43 -07:00
Alex Vandiver 89484fbbe6 install: Remove the hidden VIRTUALENV_NEEDED parameter.
We can do this behaviour better by using `has_roles`.
2024-04-04 16:49:42 -07:00
Alex Vandiver efdfaaf5a2 install: Only install node on frontend hosts. 2024-04-04 16:49:22 -07:00
Alex Vandiver acc94a5f32 create-database: Skip stop-server and flush-memcached on standalone Pg.
If running on a stand-alone PostgreSQL server, then supervisor does
exist -- but `stop-server` is useless, and in fact cannot run because
the Zulip directory may not be readable by the `zulip` user.

Detect if this is an application front-end server by looking for
`/home/zulip/deployments`, and use the stop-server and flush-memcached
from there if it exists.  The `create-db.sql` and
`terminate-psql-sessions` files are still read from the local
directory, but those already have precautions from being from a
non-world-readable directory, and are more obviously important to keep
in sync with the `create-database` script.
2024-04-04 16:48:58 -07:00
Anders Kaseorg 0bae651e23 upgrade-zulip-stage-2: Add Ubuntu 20.04, Debian 11 to UNSUPPORTED_DISTROS.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-04 16:28:12 -07:00
Anders Kaseorg ad5abbe54f apt-repos: Remove Ubuntu 24.04 gnupg workaround.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-04 10:48:22 -07:00
Anders Kaseorg 13e28fc3ac apt-repos: Disallow libmagic1 1:5.45-2 (Ubuntu 24.04) again.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-03 11:28:49 -07:00
Anders Kaseorg 6e871e7731 ruff: Fix UP036 Version block is outdated for minimum Python version.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-01 18:32:52 -07:00
Anders Kaseorg 70914b0475 Remove support for Ubuntu 20.04 and Debian 11.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2024-04-01 13:27:39 -07:00
Anders Kaseorg 4ebc734632 apt-repos: Temporarily work around Ubuntu 24.04 non-installable gnupg.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-31 13:51:31 -07:00
Anders Kaseorg 5d6e616808 Revert "apt-repos: Disallow libmagic1 1:5.45-2 (Ubuntu 24.04)."
This reverts commit 44e38e8ea7.
2024-03-30 11:29:07 -07:00
Anders Kaseorg da984d0ab3 install-node: Upgrade Node.js from 20.11.1 to 20.12.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-29 09:37:38 -07:00
Anders Kaseorg 708c3e7182 node_cache: Invoke pnpm via corepack.
This avoids prompting the user that corepack is about to download
pnpm.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-29 09:37:38 -07:00
Anders Kaseorg 44e38e8ea7 apt-repos: Disallow libmagic1 1:5.45-2 (Ubuntu 24.04).
This package is replaced by libmagic1t64 1:5.45-3 for the Ubuntu
64-bit time_t transition, but hasn’t been deleted from the archive
yet.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-28 16:47:30 -07:00
Tim Abbott 8f0f41e236 pythonrc: Add various frequently handy imports. 2024-03-27 12:49:24 -07:00