zulip/tools
Zixuan James Li 268f858f39 linkifier: Support URL templates for linkifiers.
This swaps out url_format_string from all of our APIs and replaces it
with url_template. Note that the documentation changes in the following
commits  will be squashed with this commit.

We change the "url_format" key to "url_template" for the
realm_linkifiers events in event_schema, along with updating
LinkifierDict. "url_template" is the name chosen to normalize
mixed usages of "url_format_string" and "url_format" throughout
the backend.

The markdown processor is updated to stop handling the format string
interpolation and delegate the task template expansion to the uri_template
library instead.

This change affects many test cases. We mostly just replace "%(name)s"
with "{name}", "url_format_string" with "url_template" to make sure that
they still pass. There are some test cases dedicated for testing "%"
escaping, which aren't relevant anymore and are subject to removal.
But for now we keep most of them as-is, and make sure that "%" is always
escaped since we do not use it for variable substitution any more.

Since url_format_string is not populated anymore, a migration is created
to remove this field entirely, and make url_template non-nullable since
we will always populate it. Note that it is possible to have
url_template being null after migration 0422 and before 0424, but
in practice, url_template will not be None after backfilling and the
backend now is always setting url_template.

With the removal of url_format_string, RealmFilter model will now be cleaned
with URL template checks, and the old checks for escapes are removed.

We also modified RealmFilter.clean to skip the validation when the
url_template is invalid. This avoids raising mulitple ValidationError's
when calling full_clean on a linkifier. But we might eventually want to
have a more centric approach to data validation instead of having
the same validation in both the clean method and the validator.

Fixes #23124.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2023-04-19 12:20:49 -07:00
..
ci dependencies: Switch to pnpm. 2023-03-20 15:48:29 -07:00
documentation_crawler web: Save a 301 redirect from /help to /help/. 2023-03-24 14:51:01 -07:00
droplets run-dev: Drop .py from script name. 2023-03-03 18:02:37 -08:00
i18n i18n: Unescape Unicode sequences in JSON. 2023-01-17 13:19:45 -08:00
lib emails: Inline CSS in emails in build_email. 2023-04-05 12:22:29 -07:00
linter_lib message_send: Edit scheduled message if its ID is present. 2023-04-14 17:38:37 -07:00
node_lib node_tests: Move to web/tests. 2023-02-23 16:04:17 -08:00
oneclickapps oneclick: Wait in a loop until the image build droplet gets an IP. 2022-11-18 12:45:08 -08:00
setup install-transifex-cli: Upgrade Transifex CLI from 1.6.5 to 1.6.7. 2023-04-04 00:16:04 -07:00
test-install install: Support Ubuntu 22.04. 2022-02-25 14:49:07 -08:00
tests stream_privacy: Use new lock and hashtag icons. 2023-04-10 23:41:30 -07:00
wsl
zulip-export
README.md
__init__.py
build-docs tools: Use root-based absolute import for tools.lib, etc. 2021-07-05 12:21:52 -07:00
build-release-tarball emails: Inline CSS in emails in build_email. 2023-04-05 12:22:29 -07:00
cache-zulip-git-version build-release-tarball: Override merge-base for full official releases. 2022-11-15 12:50:41 -08:00
check-capitalization tools: Use root-based absolute import for tools.lib, etc. 2021-07-05 12:21:52 -07:00
check-frontend-i18n docs: Capitalize Handlebars consistently. 2022-06-27 13:35:10 -07:00
check-issue-labels docs: Fix many spelling mistakes. 2022-02-07 18:51:06 -08:00
check-openapi dependencies: Upgrade Prettier to 3.0.0-alpha.4. 2022-11-02 16:50:37 -07:00
check-provision
check-schemas node_tests: Move to web/tests. 2023-02-23 16:04:17 -08:00
check-templates emails: Inline CSS in emails in build_email. 2023-04-05 12:22:29 -07:00
check-thirdparty check-thirdparty: License under GPLv2+. 2022-06-26 17:33:11 -07:00
clean-branches Rename default branch to ‘main’. 2021-09-06 12:56:35 -07:00
commit-message-lint commit-message-lint: Take release branches into account. 2022-07-22 10:39:05 -07:00
commit-msg requirements: Install gitlint from its Debian package. 2021-09-17 17:05:18 -07:00
conf.ini-template
coveragerc models: Remove type prefixes from __str__ values. 2023-03-08 22:56:55 -08:00
deploy-branch Rename default branch to ‘main’. 2021-09-06 12:56:35 -07:00
diagnose provision: Avoid distutils; keep PROVISION_VERSION as a tuple. 2022-07-20 22:03:30 -07:00
documentation.vnufilter dependencies: Upgrade JavaScript dependencies. 2022-10-11 10:50:57 -07:00
duplicate_commits.json
fetch-contributor-data web: Save a needless 301 redirect from /team to /team/. 2023-03-24 14:51:01 -07:00
fetch-pull-request tools: Avoid `readlink -f` in *-pull-request scripts. 2021-04-30 14:57:13 -07:00
fetch-rebase-pull-request Rename default branch to ‘main’. 2021-09-06 12:56:35 -07:00
find-unused-css web: Move web app to ‘web’ directory. 2023-02-23 16:04:17 -08:00
generate-integration-docs-screenshot run-dev: Drop .py from script name. 2023-03-03 18:02:37 -08:00
lint dependencies: Switch to pnpm. 2023-03-20 15:48:29 -07:00
message-screenshot.js puppeteer_tests: Move to web/e2e-tests. 2023-02-23 16:04:17 -08:00
pre-commit
provision docs: Rename setup-vagrant.md -> setup-recommended.md. 2022-09-15 15:54:14 -07:00
push-to-pull-request scripts: Pass --retry 3 to curl. 2022-11-08 08:07:36 -08:00
rebuild-dev-database
rebuild-test-database
release api-docs: Move markdown files to top level directory. 2023-02-02 17:25:40 -08:00
release-tarball-exclude.txt puppeteer_tests: Move to web/e2e-tests. 2023-02-23 16:04:17 -08:00
renumber-migrations black: Reformat with Black 23. 2023-02-02 10:40:13 -08:00
reset-to-pull-request tools: Avoid `readlink -f` in *-pull-request scripts. 2021-04-30 14:57:13 -07:00
review python: Use Python 3.8 shlex.join function. 2022-04-27 12:57:49 -07:00
run-codespell dependencies: Switch to pnpm. 2023-03-20 15:48:29 -07:00
run-dev run-dev: Drop .py from script name. 2023-03-03 18:02:37 -08:00
run-mypy tools: Support running mypy daemon for better performance. 2022-07-06 17:33:13 -07:00
run-tsc
semgrep.yml linkifiers: Add url_template field. 2023-04-19 12:20:49 -07:00
setup-git-repo
show-profile-results
stop-run-dev
tail-ses ruff: Fix RSE102 Unnecessary parentheses on raised exception. 2023-02-04 16:34:55 -08:00
test-all install-shellcheck: Upgrade ShellCheck to 0.9.0. 2023-01-04 14:01:34 -08:00
test-api test_helpers: Rename reset_emails_in_zulip_realm. 2023-03-01 12:17:11 -08:00
test-backend report_error: Remove API endpoint for client error reporting. 2023-04-13 14:59:58 -07:00
test-documentation
test-help-documentation tools: Use root-based absolute import for tools.lib, etc. 2021-07-05 12:21:52 -07:00
test-js-with-node linkifier: Support URL templates for linkifiers. 2023-04-19 12:20:49 -07:00
test-js-with-puppeteer dependencies: Switch to pnpm. 2023-03-20 15:48:29 -07:00
test-locked-requirements test-locked-requirements: Rename may_be_setup_cache to maybe_set_up_cache. 2022-01-12 13:21:35 -08:00
test-migrations requirements: Upgrade Python requirements. 2021-04-15 21:47:33 -07:00
test-queue-worker-reload run-dev: Drop .py from script name. 2023-03-03 18:02:37 -08:00
test-run-dev run-dev: Drop .py from script name. 2023-03-03 18:02:37 -08:00
test-tools tools: Use root-based absolute import for tools.lib, etc. 2021-07-05 12:21:52 -07:00
total-contributions black: Reformat with Black 23. 2023-02-02 10:40:13 -08:00
update-locked-requirements requirements: Remove unused mypy.txt lock file. 2022-09-08 11:08:06 -07:00
update-prod-static emails: Inline CSS in emails in build_email. 2023-04-05 12:22:29 -07:00
update-zuliprc-api-field python: Replace universal_newlines with text. 2022-01-23 22:16:01 -08:00
upload-release upload-release: Update SHA256SUM for existing files if changed. 2023-01-23 17:41:29 -08:00
webpack dependencies: Switch to pnpm. 2023-03-20 15:48:29 -07:00
zanitizer
zanitizer_config.pm.sample

README.md

This directory contains scripts that are used in building, managing, testing, and other forms of work in a Zulip development environment. Note that tools that are also useful in production belong in scripts/ or should be Django management commands.

For more details, see https://zulip.readthedocs.io/en/latest/overview/directory-structure.html.