Commit Graph

942 Commits

Author SHA1 Message Date
Alex Vandiver 7f4bc05f8e install: Check for a supported version of the OS in shell.
Testing for it in Python means that we have to worry about keeping the
`upgrade-zulip-stage-2` backwards-compatible with all versions of
Python which we could ever be upgrading from -- which is all of them.

Factor out the "supported operating systems" check, and share it
between upgrade and install codepaths.
2024-04-16 11:13:33 -07:00
Alex Vandiver a853b8849f install: Actually add --puppet-classes argument.
7187146422 only added it to the internal parsing and the help, not
the getopt invocation which would make the command accept the
argument.
2024-04-11 15:34:00 -07:00
Alex Vandiver 7187146422 install: Move PUPPET_CLASSES env var to --puppet-classes argument.
`--no-init-db` is used to silence the need for `--hostname` and
`--email` arguments; it is a proxy for "this is not a frontend host."
We would ideally like to use `has_class` to know if the user's
provided puppet classes are include an `app_frontend`, and thus
`--hostname` and `--email` are required -- but doing that requires
several other steps, and we would like this feedback to be immediate.

We make the presence of `--puppet-classes` equivalent to
`--no-init-db`, since nearly every configuration with
`--puppet-classes` does not install both a database and a frontend,
which is what is required to initialize a database.
2024-04-04 16:49:43 -07:00
Alex Vandiver 89484fbbe6 install: Remove the hidden VIRTUALENV_NEEDED parameter.
We can do this behaviour better by using `has_roles`.
2024-04-04 16:49:42 -07:00
Alex Vandiver efdfaaf5a2 install: Only install node on frontend hosts. 2024-04-04 16:49:22 -07:00
Anders Kaseorg 0bae651e23 upgrade-zulip-stage-2: Add Ubuntu 20.04, Debian 11 to UNSUPPORTED_DISTROS.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-04 16:28:12 -07:00
Anders Kaseorg 6e871e7731 ruff: Fix UP036 Version block is outdated for minimum Python version.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-04-01 18:32:52 -07:00
Anders Kaseorg 70914b0475 Remove support for Ubuntu 20.04 and Debian 11.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2024-04-01 13:27:39 -07:00
Anders Kaseorg 4ebc734632 apt-repos: Temporarily work around Ubuntu 24.04 non-installable gnupg.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-31 13:51:31 -07:00
Anders Kaseorg da984d0ab3 install-node: Upgrade Node.js from 20.11.1 to 20.12.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-29 09:37:38 -07:00
Anders Kaseorg 708c3e7182 node_cache: Invoke pnpm via corepack.
This avoids prompting the user that corepack is about to download
pnpm.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-29 09:37:38 -07:00
Tim Abbott 8f0f41e236 pythonrc: Add various frequently handy imports. 2024-03-27 12:49:24 -07:00
Anders Kaseorg a1a341f0ae ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-21 08:59:17 -07:00
Alex Vandiver 0f70ab7679 config: Use logic from zulip_tools, to support bools. 2024-03-15 15:34:12 -07:00
Alex Vandiver b36f389e9e zulip_tools: Make default default_value be None, not "". 2024-03-15 15:34:12 -07:00
Alex Vandiver f9f9392b93 zulip_tools: Unify get_config and get_config_bool using overloads. 2024-03-15 15:34:12 -07:00
Alex Vandiver 0ff1fb2e21 zulip_tools: Support non-lowercase true values.
This matches the ruby implementation.
2024-03-15 15:34:12 -07:00
Alex Vandiver 3905416a0e zulip_tools: Fix path to zulipconf.rb.
This was moved in 16aa7c0923.
2024-03-15 15:34:12 -07:00
Alex Vandiver e0cf82a526 email-mirror-postfix: Use get_config_bool, instead of checking "true". 2024-03-15 15:34:12 -07:00
Alex Vandiver 5123477d55 upgrade-zulip-from-git: Provide a flag to use a local branch.
While this could be done previously by calling
`upgrade-zulip-from-git --remote-url /srv/zulip.git`, the explicit
argument makes this more straightforward, and avoids churning the
`refs/remotes/origin/` namespace.
2024-03-08 12:15:25 -08:00
Anders Kaseorg 066ea3ebf9 install: Support Ubuntu 24.04.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 17:38:08 -08:00
Anders Kaseorg 55954c9819 build-pgroonga: Upgrade PGroonga from 3.1.0 to 3.1.8.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-03-01 17:38:08 -08:00
Anders Kaseorg 570f3dd447 python: Reformat with Ruff formatter.
https://docs.astral.sh/ruff/formatter/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-29 17:07:16 -08:00
Anders Kaseorg 94cf99bcf2 install-node: Upgrade Node.js from 20.9.0 to 20.11.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-02-22 12:46:40 -08:00
Alex Vandiver ec6f64f7b0 tornado: Perform rolling client restarts after servers are restarted.
Decouple the sending of client restart events from the restarting of
the servers.  Restarts use the new Tornado restart-clients endpoint to
inject "restart" events into queues of clients which were loaded from
the previous Tornado process.  The rate is controlled by the
`application_server.client_restart_rate`, in clients per minute, or a
flag to `restart-clients` which overrides it.  Note that a web client
will also spread its restart over 5 minutes, so artificially-slow
client restarts are generally not very necessary.

Restarts of clients are deferred to until after post-deploy hooks are
run, such that the pre- and post- deploy hooks are around the actual
server restarts, even if pushing restart events to clients takes
significant time.
2024-02-15 15:42:50 -08:00
Alex Vandiver 27d53ecbe1 restart-server: Remove --skip-tornado flag.
This flag was generally used not because we wanted to avoid
restarting Tornado, but because we wanted to avoid increasing load
the server when all of the clients were told to reload.

Since we have laid the groundwork for separately telling Tornado to
tell clients to restart, we remove the --skip-tornado flag; the next
commit will add the ability to skip client restarts.
2024-02-15 15:42:50 -08:00
Alex Vandiver e6fa1f5436 email_mirror_message: Move endpoint under /api/internal/.
This also has the side effect that it saves the endpint from being
improperly redirected by the HostDomainMiddleware's canonicalization
code.
2024-02-15 15:42:50 -08:00
Anders Kaseorg 93198a19ed requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-01-29 10:41:54 -08:00
Alex Vandiver 1ba2f39854 install: Support PostgreSQL 16. 2023-12-23 14:57:12 -08:00
Anders Kaseorg a13e42f18a docs: Add missing spaces around code spans.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-14 00:05:44 -08:00
Anders Kaseorg 223b626256 python: Use urlsplit instead of urlparse.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 13:03:07 -08:00
Anders Kaseorg 8a7916f21a python: Consistently use from…import for datetime.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-12-05 12:01:18 -08:00
Alex Vandiver 7c8eacfa12 queue: Increase paging thresholds for deferred_work queue.
This queue is used to things which definitionally may take longer than
a request, so paging after 60s is rather aggressive.  This is
especially true because this queue has a very long tail of very slow
tasks -- p99 of task time in this queue is 8.5s, while p99.9 is 197s.

Raise the paging threshold to 15 minutes.  While there are
semi-user-facing tasks which use this queue (primarily marking
messages as read), those being delayed for minutes is already a real
possibility if they are stuck behind a large realm export -- and this
is not a situation which should necessarily page, since it is not
solvable by the administrator.
2023-11-09 13:51:00 -08:00
Anders Kaseorg dcdbb39c31 install-node: Upgrade Node.js from 18.18.0 to 20.9.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-11-08 16:11:15 -08:00
Alex Vandiver fb80ecd9c3 upgrade-zulip: Defer cache-filling to just outside the critical period.
Filling caches needs to happen close to when the server is restarted,
as the gap opens us up to race conditions with user modifications.  If
there are migrations, however, it must happen within the critical
period after the migrations are applied.

Move the call to fill the caches to within the `shutdown_server`
function, so that we push it as close to the server shutdown as
possible.
2023-11-08 11:00:00 -08:00
Anders Kaseorg a50eb2e809 mypy: Enable new error explicit-override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-12 12:28:41 -07:00
Anders Kaseorg cc07b8c79c install-node: Upgrade Node.js from 18.17.1 to 18.18.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 14:16:23 -07:00
Anders Kaseorg 835ee69c80 docs: Fix grammar errors found by mwic.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-10-09 13:24:09 -07:00
evykassirer eff0a73925 emoji: Add unified_reactions.json to the sha1sum. 2023-09-13 12:40:59 -07:00
Anders Kaseorg 81bd63cb46 ruff: Fix PIE808 Unnecessary `start` argument in `range`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-01 14:57:01 -07:00
Anders Kaseorg ead7b9177a check-database-compatibility: Ignore twofactor typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-23 22:14:26 -07:00
shu.chen 321776ac49
provision: Add provision support for Fedora 38.
Tweaked provision script to run successfully in Fedora 38 and
included a script to build the groonga libs from source because
the packages in Fedora repos are outdated.

There is a major version jump from the last supported version (F34)
which is EOL so references and support for older versions were
removed.

Fixes: #20635
2023-08-22 11:26:29 -07:00
Anders Kaseorg 344bfbf539 install-node: Upgrade Node.js from 18.17.0 to 18.17.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 18:16:47 -07:00
Anders Kaseorg c43629a222 ruff: Fix PLW1510 `subprocess.run` without explicit `check` argument.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-17 17:05:34 -07:00
David Rosa d205850d54 docs: Update .html links pointing to "Upgrade Zulip" or "Modify Zulip".
Follow-up to #24089.
2023-08-11 16:49:32 -07:00
Anders Kaseorg e932e2ce52 ruff: Fix UP032 Use f-string instead of `format` call.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-08-02 15:58:55 -07:00
Anders Kaseorg 5ccb408f19 build-pgroonga: Upgrade PGroonga from 3.0.3 to 3.1.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-25 17:33:42 -07:00
n0099 2b1d250154 nginx: Switch from `$host` to `$http_host` for HTTP/3 compatibility.
nginx sets the value of the `$http_host` variable to the empty string
when using http/3, as there is technically no `Host:` header sent:
https://github.com/nginx-quic/nginx-quic/issues/3

Users with a browser that support http/3 will send their first request
to nginx with http/2, and get an expected HTTP 200 -- but any
subsequent requests will fail with am HTTP 400, since the browser will
have upgraded to http/3, which has an empty `Host` header, which Zulip
rejects.

Switch to the `$host` variable, which works for all HTTP versions.

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2023-07-25 18:20:34 -04:00
Anders Kaseorg e890033f76 install-node: Upgrade Node.js from 18.16.0 to 18.17.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-21 15:58:42 -07:00
Anders Kaseorg b285813beb error_notify: Remove custom email error reporting handler.
Restore the default django.utils.log.AdminEmailHandler when
ERROR_REPORTING is enabled.  Those with more sophisticated needs can
turn it off and use Sentry or a Sentry-compatible system.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-07-20 11:00:09 -07:00