Commit Graph

25 Commits

Author SHA1 Message Date
Lauryn Menard d911e9b43f i18n-tools: Write a newline when unescaping json data contents.
Updates `tools/i18n/unescape-contents` to write a newline at the end
of the file when updating the json data.
2024-06-04 08:20:24 -07:00
Lauryn Menard 593a1f12b7 i18n-tools: Print locale that's being checked for legacy translations.
Adds a print statement to `tools/i18n/update-for-legacy-translations`
to help with debugging to know what locale was being checked if an
error is raised.
2024-06-04 08:20:24 -07:00
Tim Abbott 9bf91d5e1b i18n: Generate legacy stream translation files.
Generated using tools/i18n/create-legacy-stream-translations, which we
also delete in this commit as it will never be run again.
2024-05-02 10:50:24 -07:00
Lauryn Menard d651441f66 i18n: Add scripts for web app legacy "stream" string translations.
Adds tools/i18n/create-legacy-stream-translations to create a
`legacy_stream_translations.json` file for every non-English
language locale that will serve as the legacy translations for
the stream to channel rename.

Adds tools/i18n/update-for-legacy-translations to manage adding
the legacy translations for any stream/channel strings that we'd
like to maintain existing translations for, which are defined in
LEGACY_STRINGS_MAP.
2024-05-02 10:50:24 -07:00
Anders Kaseorg 964cc182a4 sync-translations: Normalize translations to NFC.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-09-08 17:51:47 -07:00
Anders Kaseorg 8f27a6fb73 provision: Disable compilemessages searching for locale directories.
We explicitly configure LOCALE_PATHS, so we can safely disable this
search of the entire tree for other locale directories.

https://github.com/django/django/blob/4.1/django/core/management/commands/compilemessages.py#L92-L100

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2023-05-03 17:20:16 -07:00
Josh Klar 70b30e7792 i18n: Unescape Unicode sequences in JSON.
This greatly improves the readability of the diffs and in-codebase
translation strings over using ASCII sequences for unicode in the JSON.

We've previously noticed [^1] some JSON translation files ending up with
escaped Unicode sequences on disk, which Transifex indicates is expected
behavior [^2], though it is sometimes fixed by `manage.py
compilemessages` [^3]. Further, as noted in #23932 [^4], some JSON
translation files include HTML-escaped entities like quotation marks.

This script will ingest valid JSON files and output them as proper UTF-8
files with appropriately unescaped (unless otherwise necessary, like
double quotes being backslash-escaped) sequences, except when the key
itself contains HTML escape sequences (as it's presumed the value of
such entries must be pre-escaped before being passed to consumers).

[^1]: https://chat.zulip.org/#narrow/stream/58-translation/topic/Transifex.20client/near/1479205

[^2]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481287

[^3]: https://chat.zulip.org/#narrow/stream/58-translation/topic/an.20email.20for.20Transifex.20support/near/1481908

[^4]: Which is not end-to-end fixed yet by this commit: that will
require a new release of Zulip Server.

gitlint-ignore: B1, title-trailing-punctuation, body-min-length, body-is-missing
2023-01-17 13:19:45 -08:00
Anders Kaseorg 97e4e9886c python: Replace universal_newlines with text.
This is supported in Python ≥ 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:16:01 -08:00
Tim Abbott a49e685d97 i18n: Adjust Transifex sync-translations download mode.
It appears that some server-side change to Transifex resulted in the
"onlytranslated" mode deleting some (all?) strings from django.po files that
were not translated.

Testing determined that the "translator" mode appears to now be the
only mode that works with both our django.po and translations.json
files (We want to avoid both copying the English strings and deleting
strings), so we're switching to that.

Background is available here:
https://chat.zulip.org/#narrow/stream/3-backend/topic/4.2Ex.20branch.20translations.20sync/near/1187324
2021-06-02 09:48:58 -07:00
Tim Abbott 3116f72894 tools/i18n: Add support for branch parameters.
One could do other things with this, but what I expect us to do is
pass `-b {branch}`.
2021-04-30 16:37:01 -07:00
Tim Abbott 6312401e58 i18n: Tweak makemessages for Django 3.1 parameters.
Apparently the default for how this works changed in the Django 2.1 to
3.1 upgrade.
2021-03-22 19:37:20 -07:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg aaa7b766d8 python: Use universal_newlines to get str from subprocess.
We can replace ‘universal_newlines’ with ‘text’ when we bump our
minimum Python version to 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-30 11:36:38 -07:00
Tim Abbott 9c0d6becc5 docs: Add comment links to i18n documentation. 2020-09-18 11:44:04 -07:00
Anders Kaseorg 3b257d10df tools: Delete tagmessages script.
It was broken by commit aaedec1fdb which
moved it to tools/i18n without adjusting its relative path references,
and it contains a sketchy injectable os.system call that I’d like to
remove.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:35:51 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 8dd83228e7 python: Convert "".format to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus --keep-percent-format, but with the
NamedTuple changes reverted (see commit
ba7906a3c6, #15132).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-08 15:31:20 -07:00
Anders Kaseorg f8339f019d python: Convert assignment type annotations to Python 3.6 style.
Commit split by tabbott; this has changes to scripts/, tools/, and
puppet/.

scripts/lib/hash_reqs.py, scripts/lib/setup_venv.py,
scripts/lib/zulip_tools.py, and tools/lib/provision.py are excluded so
tools/provision still gives the right error message on Ubuntu 16.04
with Python 3.5.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-shebang_rules: List[Rule] = [
+shebang_rules: List["Rule"] = [

-trailing_whitespace_rule: Rule = {
+trailing_whitespace_rule: "Rule" = {

-whitespace_rules: List[Rule] = [
+whitespace_rules: List["Rule"] = [

-comma_whitespace_rule: List[Rule] = [
+comma_whitespace_rule: List["Rule"] = [

-prose_style_rules: List[Rule] = [
+prose_style_rules: List["Rule"] = [

-html_rules: List[Rule] = whitespace_rules + prose_style_rules + [
+html_rules: List["Rule"] = whitespace_rules + prose_style_rules + [

-    target_port: int = None
+    target_port: int

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-24 13:06:54 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Anders Kaseorg 40f4ead738 mypy: Upgrade from 0.720 to 0.730.
Fixes #13269.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Tim Abbott a84bb89bdc scripts: Move mobile i18n code out scripts/.
Like other code that is only used in the development environment, this
doesn't belong in scripts/.
2019-09-24 12:57:42 -07:00
Anders Kaseorg bbb56df6b0 i18n: Move static/locale back to locale.
As of commit cff40c557b (#9300), these
files are no longer served directly to the browser.  Disentangle them
from the static asset pipeline so we can refactor it without worrying
about them.

This has the side effect of eliminating the accidental duplication of
translation data via hash-naming in our release tarballs.

This reverts commit b546391f0b (#1148).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 14:57:55 -07:00
Tim Abbott cdf987fb14 push-translations: Run manage.py makemessages first.
This saves an extra step in pushing translations to transifex, with no
real negative side effects.
2019-03-13 14:11:27 -07:00
Tim Abbott aaedec1fdb tools: Move a few i18n scripts to tools/i18n.
These are likely to only be run by Zulip maintainers.
2018-08-09 15:46:27 -07:00