Commit Graph

407 Commits

Author SHA1 Message Date
Anders Kaseorg c6b6004432 ruff: Fix SIM111 Use `return all(…)` instead of `for` loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-23 11:18:36 -08:00
Lalit 112df91fbd provision: Do not require that no other yarn precedes us in $PATH.
`check_version` in `install-yarn` had the rather careful check that
the yarn it installed into `/usr/bin/yarn` was the yarn which was
first in the user's `$PATH`.  This caused problems when the user had a
pre-existing `/usr/local/bin/yarn`; however, those problems are
limited to the `install-yarn` script itself, since the nearly all
calls to yarn from Zulip's code already hardcode the `/srv/zulip-yarn`
location, and do not depend on what is in `$PATH`.

Remove the checks in `install-yarn` that depend on the local `$PATH`,
and stop installing our `yarn` into it.  We also adjust the two
callsites which did not specify the full path to `yarn`, so use
`/srv/zulip-yarn`.

Fixes: #23993

Co-authored-by: Alex Vandiver <alexmv@zulip.com>
2023-01-19 17:51:52 -05:00
Alex Vandiver 7c0d414aff uploads: Split out S3 and local file backends into separate files.
The uploads file is large, and conceptually the S3 and local-file
backends are separable.
2023-01-09 18:23:58 -05:00
Anders Kaseorg e1ed44907b ruff: Fix SIM118 Use `key in dict` instead of `key in dict.keys()`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 16:25:07 -08:00
Anders Kaseorg cd66448311 install-shfmt: Upgrade shfmt to 3.6.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 14:01:34 -08:00
Anders Kaseorg 986bad8f90 install-shellcheck: Upgrade ShellCheck to 0.9.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 14:01:34 -08:00
Anders Kaseorg 8a8e538af7 install-aws-server: Use correct fd and exit code for usage message.
On --help, write to stdout and exits with 0; on error, write to stderr
and exit with 1.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-01-04 14:01:34 -08:00
Josh Klar 0c44b933c8 provision: Replace transifex-client with new transifex-cli.
transifex-client went EOL on November 30, 2022, replaced by
transifex/cli [^1].

Swap this in-place, since per the upstream README [^2]:

> The current version of the client maintains backwards compatibility
> for the tx push and tx pull commands. So, if you have a CI setup that
> uses them, you should not have to change anything.

As the mobile team found out, this is a partial truth if one previously
used some of the more advanced CLI flags, but all workflows referenced
in tools/ and docs/ use forwards-compatible flags to the new version.

[^1]: https://github.com/transifex/transifex-client/

[^2]: a0f28a1cf3/README.md
2022-12-13 12:34:08 -08:00
Greg Price 214eec007a tools: Add script apns/prep-cert.
This prepares server certificates for talking to APNs.
2022-12-07 22:07:36 -08:00
evykassirer 4b175514aa emoji: Remove variation selector when generating emoji.
Fixes #23625.

The U+FE0F character was appearing in the data supplied by
emoji-datasource-google, but isn't in the CLDR data. By
removing that variation selector character, all remaining
emoji (that were previously marked as "not found in custom
emoji name maps, but also not found in CLDR data. Skipping.")
are now supported.
2022-12-06 12:33:59 -08:00
Anders Kaseorg c850a006d9 emoji: Import modules relative to ZULIP_PATH.
Newer mypy is less confused this way.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
Anders Kaseorg 84bdd6371d emoji: Remove old broken CSV import/export scripts.
These have been broken at least since commit
e331a758c3 (#12787), so clearly nobody
cares.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-12-04 22:11:24 -08:00
evykassirer 512705f267 emoji: Remove commas from emoji names.
Fixes part of #23626.

This was preventing emoji like `:family_man,_woman,_girl,_boy:`
from displaying.
2022-11-29 16:26:15 -08:00
rht 5ce2103b87 Slack import: Cache emoji.json in static/generated/emoji.
Previously, emoji.json was read from
"$ZULIP_PATH/node_modules/emoji-datasource-google/emoji.json".
This path doesn't exist in production when installing from scratch from
a release tarball. And so, we ensure emoji.json exists by copying it to
`static/generated/emoji`.

With tweaks to comments by tabbott.

Fixes: #23469
2022-11-15 10:43:11 -08:00
Alex Vandiver 6f46faa980 bootstrap-awscli: Upgrade awscli version. 2022-11-08 13:26:32 -08:00
Anders Kaseorg fc9f23a85f scripts: Pass --retry 3 to curl.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-08 08:07:36 -08:00
Anders Kaseorg 51d1ccb064 build_emoji: Guard mutable default with read-only type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-04 14:42:33 -07:00
Anders Kaseorg 7021852627 install-node: Silence expected “node: command not found” on first run.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:11:08 -07:00
Alex Vandiver 12a5317b8c bootstrap-aws-installer: Switch to an IMDSv2-compatible URL.
We don't use the token we request for anything, but a straight GET
request would fail in an IMDSv2-only environment.
2022-10-28 16:52:54 -07:00
Alex Vandiver 721a1d7a10 bootstrap-aws-installer: Drop "credential_source" in .aws/config.
Setting `credential_source` is used when assuming role credentials --
that is, when running as one role, use the AssumeRole right to become
someone else.

The AWS command-line tools only do this if `role_arn`, the role to
assume, is also set -- if it is not set, it transparently falls
through to IAM role attached to the EC2 instance profile.  However,
with the `aws-sdk-go` package, used by Teleport, this configuration
produces an error.

Remove the `credential_source = Ec2InstanceMetadata` line, which isn't
necessary for the AWS CLI, and interferes with Teleport operation.
2022-10-28 16:52:54 -07:00
Alex Vandiver 88fcf6862f bootstrap-aws-installer: Add ed-25519 host key. 2022-10-28 16:52:54 -07:00
Alex Vandiver 993fdc3311 bootstrap-awscli: Upgrade awscli version. 2022-10-10 08:46:29 -07:00
evykassirer b2c8ca295f emoji: Support new emoji and fallback from blob to Google for new emoji.
Fixes #19371.
2022-10-04 12:29:35 -07:00
evykassirer 8a9e68e026 emoji: Finish script to generate emoji_names.py with CLDR data.
This script pulls from our previously custom-written emoji strings
and fills in the rest from CLDR. It also removes 4 custom emoji which
collide with some of the new CLDR names (they will now just be called
by their CLDR name).
2022-10-04 12:29:35 -07:00
evykassirer efee77b41f emoji: Add which emoji are supported to the /register call.
Fixes #21037.

This is part of fixing #19371. To bulk-add new emoji regularly,
mobile needs to know which servers support which emoji.
`staticfiles_storage.url` generates a unique URL with a hash
based on the file content, which lets mobile know if it needs
to update its locally stored data.
2022-08-26 17:58:31 -07:00
evykassirer afab7002bf emoji: Add script to generate emoji_names.py, only with custom names.
This script pulls from our previously custom-written emoji strings
to prepare to fill in the rest from CLDR.

This commit has no user-facing changes.
2022-07-21 13:22:07 -07:00
evykassirer 190c0737a7 emoji: Store customized emoji in file separate from emoji_names.py.
Emoji that we specifically choose names or aliases for will be
stored in this new file so that we can generate emoji_names.py
from the custom names and also CLDR (for the rest).

This new file isn't used for anything yet (no user facing change);
it will be used in future commits.
2022-07-21 13:22:07 -07:00
Anders Kaseorg 81892df176 requirements: Upgrade to Django 4.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-13 16:07:17 -07:00
Anders Kaseorg e4e51b0eb4 install-shfmt: Upgrade shfmt from 3.5.0 to 3.5.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-06 17:23:16 -07:00
Alex Vandiver aabbb2ad10 bootstrap-awscli: Leave CWD unchanged. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1b5e058d3b install-aws-server: Add a --debug-key to allow logins during bootstrap. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1a1061e77e install-aws-server: Switch to getopt. 2022-06-28 09:39:31 -04:00
Alex Vandiver 1be9ab2690 install-aws-server: Assume zulip_ops::profile:: prefix on all roles.
This will require that any profile-specific sections of
`$HOME/.zulip-install-server.conf` be renamed to their short form.
2022-06-28 09:39:31 -04:00
Alex Vandiver 9a3be2af64 install-aws-server: Fix default_branch config name. 2022-06-28 09:39:31 -04:00
Alex Vandiver f5d4dea2f0 bootstrap-aws-installer: Leave `ubuntu` user deletion until the last.
This makes it easier to log in and figure out what went wrong if
something failed during the bootstrapping process.
2022-06-28 09:39:31 -04:00
Alex Vandiver a35af3f38b install/upgrade: Allow new packages during `apt-get upgrade`.
`postgresql-14.4` is a notable upgrade in the PostgreSQL series, as it
fixes potential database corruption from `CREATE INDEX CONCURRENTLY`
statements which are run while rows are modified[1].  However, it also
requires an upgrade from `libllvm9` to `libllvm10`, which means it is
not installed by a mere `apt-get upgrade`.

Add the `--with-new-pkgs` flag to all of the potentially relevant
`apt-get upgrade` calls, so that this (and similar) packages are
upgraded successfully.

[1]: https://www.postgresql.org/docs/release/14.4/
2022-06-21 11:21:49 -07:00
Anders Kaseorg 79842fd228 install-shfmt: Upgrade shfmt from 3.4.3 to 3.5.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-02 12:03:49 -07:00
Anders Kaseorg a543dcc8e3 Remove Debian 10 support.
As a consequence:

• Bump minimum supported Python version to 3.8.
• Move Vagrant environment to Ubuntu 20.04, which has Python 3.8.
• Move CI frontend tests to Ubuntu 20.04.
• Move production build test to Ubuntu 20.04.
• Move 3.4 upgrade test to Ubuntu 20.04.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-26 16:32:02 -07:00
Alex Vandiver 6b0876063f bootstrap-awscli: Upgrade awscli version. 2022-03-17 11:44:05 -07:00
Anders Kaseorg bff2741499 install-shfmt: Upgrade shfmt from 3.4.2 to 3.4.3.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-03-01 23:09:46 -08:00
Anders Kaseorg 21cd1c10b3 docs: Add missing space in “time zone”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-24 14:05:12 -08:00
Anders Kaseorg f6a701090c setup-apt-repos: Don’t install lsb_release.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-14 16:38:53 -08:00
Anders Kaseorg 961633fcec timezone: List only canonical timezone identifiers.
For aliases that will no longer be listed, see the third column of

grep '^L ' zulip-py3-venv/lib/python3.*/site-packages/pytz/zoneinfo/tzdata.zi

Time zones previously set to an alias will be canonicalized on demand.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-11 17:38:57 -08:00
Anders Kaseorg b0ce4f1bce docs: Fix many spelling mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:51:06 -08:00
Puneeth Chaganti 3d55a1a5bf emoji: Make large square emoji names consistent. 2022-02-03 15:08:34 -08:00
Puneeth Chaganti d55c137277 emoji: Add yellow_large_square and green_large_square emojis.
Wordle has recently become a thing and it uses green, yellow and white (or
black in dark mode) large square unicode characters to let people share their
gameplay. Zulip converts the white and black large square unicode characters to
emojis, but not the green and yellow ones. This causes the Wordle grid to be
misaligned when shared on Zulip.

This commit adds green and yellow large square emojis to our emoji list to fix
the problem.
2022-02-02 16:26:31 -08:00
Anders Kaseorg 48024097c4 install-shfmt: Upgrade shfmt from 3.4.1 to 3.4.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-24 15:55:38 -08:00
Anders Kaseorg a58a71ef43 Remove Ubuntu 18.04 support.
As a consequence:

• Bump minimum supported Python version to 3.7.
• Move Vagrant environment to Debian 10, which has Python 3.7.
• Move CI frontend tests to Debian 10.
• Move production build test to Debian 10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 17:26:14 -08:00
Alex Vandiver 7b95f38854 install-aws-server: Link to run-instances documentation. 2022-01-12 15:52:19 -08:00
Alex Vandiver 44b8321721 install-aws-server: Shell hygene. 2022-01-12 15:52:19 -08:00