Commit Graph

4294 Commits

Author SHA1 Message Date
Vishnu KS 656fa15da1 droplets: Make create_droplet take droplet name instead of username. 2021-03-23 14:05:34 -07:00
Vishnu KS 4ba16d6ed9 droplets: Create get_ssh_keys_string_from_github_ssh_key_dicts function. 2021-03-23 14:05:34 -07:00
YashRE42 5b0db2e7ed templates.js: Remove unused "plural" helper.
This helper was added in eac6463031 and
used by the "message.handlebars" file. This is no current call for
this helper in the codebase, hence it is removed to improve coverage.

This commit also marks template.js to have 100% test coverage.
2021-03-23 10:46:04 -07:00
Abhijeet Prasad Bodas f17a52b2f3 refactor: Use confirm_dialog for unstar-all-messages.
This replaces the separate modal shown on clicking
"Unstar all messages" from the left sidebar to use
the confirm_dialog widget instead.
2021-03-23 00:17:15 -07:00
shanukun 7aa89289a3 check_schema: Ignore operation field in schema validation for openapi.
In the openapi specs, the update_message_flags event is documented as
having a `operation` (deprecated) field, alongside the modern `op`.

This causes check_schemas warnings like this:

    NEED SCHEMA to match OpenAPI update_message_flags_add_add_event
    NEED SCHEMA to match OpenAPI update_message_flags_remove_remove_event

as check_schemas uses both `op` and `operation` for constructing the
event name.

Being deprecated (and really only still there for
backwards-compatibility with the original error of having it present),
`operation` will be removed eventually, therefore we can safely
ignore it from being used in openapi schema validation.

Part of #17568.
2021-03-22 23:45:42 -07:00
Tim Abbott 6312401e58 i18n: Tweak makemessages for Django 3.1 parameters.
Apparently the default for how this works changed in the Django 2.1 to
3.1 upgrade.
2021-03-22 19:37:20 -07:00
Ganesh Pawar c1628e7605 provision: Upgrade support for Fedora to version 33.
Note that the `overwrite_symlink` changes fix a bug introduced in
5c20ee998c, that we need root
permissions to do those operations.
2021-03-22 19:34:18 -07:00
Anders Kaseorg 91f048c056 gitlint-rules: Remove convoluted binary search for imperative forms.
This also fixes the suggestions for the following words: disabled,
disables, disabling, implemented, implementing, implements, kept,
made, took, using.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-22 18:08:01 -07:00
Steve Howell 1cee29c2d1 refactor: Extract stream_bar.decorate.
This makes input_pill no longer depend on
stream_data and stream_color, and it
probably reduces some other dependencies.
2021-03-22 13:21:56 -07:00
YashRE42 0b6b5f421f reload_state: Add 100% test coverage.
This commit adds a new reload_state.js file to node_tests/ which
completes the missing coverage from reload_state.
2021-03-21 17:14:28 -07:00
Anders Kaseorg 0d218a4b76 eslint: Enable @typescript-eslint/consistent-type-imports.
TypeScript type-only imports will probably become important eventually
for reducing our circular import problem.

https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-8.html#type-only-imports-and-export

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-18 17:26:19 -07:00
Tim Abbott 6d506ed40a node tests: Mark starred_messages as fully covered. 2021-03-18 16:32:12 -07:00
YashRE42 2b03a74d97 channel: Add while_reloading test cases for 100% coverage.
The branches that depended on reload_state were introduced after a
refactor in 'presence' and never had relevant test cases added.
2021-03-18 10:09:22 -07:00
Anders Kaseorg afbcdc38f4 test-js-with-puppeteer: Propagate --skip-provision-check to test server.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-17 18:26:15 -07:00
Anders Kaseorg 129ea6dd11 nginx: Consistently listen on IPv6 and with HTTP/2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-17 17:46:32 -07:00
Anders Kaseorg 3ef6f6e2e2 js: Convert static/js/blueslip.js to ES6 module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-17 08:47:15 -04:00
Aman Agrawal 76c69b943c github_actions: Explode backend and frontend tests.
We basically move all the tests from backend and frontend test
files to zulip-ci workflow. This results in GitHub Actions
nicely displaying all the tests separately.
2021-03-16 15:11:21 -07:00
Aman Agrawal 2b23609f9a ci: Remove unused circleci config file and update codebase.
We have disabled CircleCI and are using GitHub Actions for automated
testing.

docs: Changed context from CircleCI to Github Actions and wrote
some documentation specific to GH Actions.

tools: Replaced env checks for CIRCLECI with GITHUB_ACTION.

README: Use GitHub Actions build status badge.
2021-03-16 15:11:21 -07:00
Aman Agrawal 59c8f3ed92 ci: Use general terms for CircleCI.
GitHub Actions supports doing more than just CI,
and so in some contexts it's less obvious that we're
talking about just the CI if we refer to it instead of CircleCI.
2021-03-16 14:56:43 -07:00
Riken Shah d6307cc907 puppeteer_test: Add support to run tests with Firefox.
This commit introduces the `--firefox` argument,
which will allow us to run puppeteer tests
with Firefox also.
2021-03-12 12:14:37 -08:00
Anders Kaseorg 3e552268e5 templates: Delete unused markdown_help.html.
Apparently I failed to finish deleting it in commit
7f30e5f9df (#17560).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-11 21:45:48 -08:00
Anders Kaseorg 7f30e5f9df info_overlay: Render Markdown help with frontend Markdown library.
Fixes #15375.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-11 21:24:39 -08:00
Gaurav Pandey 86c883ceb0 tools: Add success message to check-schemas. 2021-03-10 11:54:02 -05:00
Alex Vandiver 3207928418 requirements: Fix paths to 'non-"lock" files'; those are the .in files 2021-03-04 18:08:45 -08:00
Anders Kaseorg d393ac5034 update-prod-static: Remove unused --prev-deploy option.
It’s unused since commit 079ddae4c8
(#12676).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-04 18:04:57 -08:00
Riken Shah fb8dc7732c puppeteer_test: Set the system default locale to `en_US`.
This commit sets the system default locale to `en_US.UTF-8` before
running the tests.

This was necessary because when running tests with Firefox, it was
failing on some test verifies the results of locale-aware sorting.
2021-03-04 09:40:03 -08:00
Anders Kaseorg d6cdd279d0 documentation_crawler: Fix for Twisted 21.2.0 type hints.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:42 -08:00
Anders Kaseorg b8e5e34c01 install-shfmt: Upgrade shfmt from 3.2.1 to 3.2.2.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 21:46:42 -08:00
Anders Kaseorg ccad00b7e9 provision: Suppress exception chaining for CalledProcessError retries.
When exception is raised inside an exception handler, Python 3
helpfully prints both tracebacks separated by “During handling of the
above exception, another exception occurred:”.  But when we’re using
an exception handler to retry the same operation, multiple tracebacks
are just noise.  Suppress the earlier one using PEP 409 syntax.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-03 16:25:03 -08:00
m-e-l-u-h-a-n ec8244b113 tools: Remove -f short option for test-all.
./tools/test-all had a short option -f which was used for running it
by skipping provision check. This short option paired well with its
long version --force, but in an effort to provide more clarity over
the use of force option it is renamed to --skip-provision-check.

Following this rename -f option for test-all does not pair well with
its long part which is now --skip-provision-check. Also, this option
is rarely used. So the short form -f is removed in this commit. We
now provide only its longer version --skip-provision-check.

This is a part of commits done to address issue #17455.
2021-03-03 09:17:49 -08:00
m-e-l-u-h-a-n 63c12d2e59 tools: Change help message for skip-provision-check argument.
This commit changes help message for skip-provision-check argument
used with various test commands. The message is changed with an
intention to be more clear about what this option actually does.

This commit is in series of various changes done to provide clarity
over the use of --force option which is renamed to
--skip-provision-check.

Fixes: #17455
2021-03-03 09:17:49 -08:00
m-e-l-u-h-a-n 911c5f19f1 tools: Rename force argument to skip-provision-check
This commit renames --force argument used with various tests to
--skip-provision-check. As a consequence of this name change all other
files that set --force option for the test commands have been updated.

This change is done in order to provide more clarity for using this
option for runnning tests.

This commit addresses issue #17455.
2021-03-03 09:17:49 -08:00
m-e-l-u-h-a-n ca515e5583 tools: Move registration of --force to test-scripts.py.
This commit moves --force option used with various tests to
test-scripts.py to have it alongside the logic that does provisioning
status assertion.
This is a step towards providing more clarity over use of this
argument with tests as asked in issue #17455.
2021-03-03 09:17:49 -08:00
Anders Kaseorg 79ac5c25b5 zulip_test: Re-export internal functions used by Puppeteer tests.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-02 17:06:35 -08:00
Aman Agrawal 80268c52df ci: Notify in zulip when a build fails in GitHub Actions.
We use the circleci integration which already has a nice setup
for sending messages when triggered to send the build failure
notification.
2021-02-26 08:29:56 -08:00
shanukun 4b67946605 refactor: Make acting_user a mandatory kwarg for do_create_user. 2021-02-25 17:58:00 -08:00
Alex Vandiver 289b5bc100 lint: Re-enable imperative-mood checking.
This was mistakenly disabled in e1f943913a.  Re-enable it.
2021-02-23 14:54:07 -08:00
Alex Vandiver 12ade4c49d lint: Rely on core gitlint skipping of revert commits.
gitlint 0.13.0 and above skip revert commits[1].  This obviates the
need for a custom rule.

[1] https://jorisroovers.com/gitlint/configuration/#ignore-revert-commits
2021-02-23 14:28:09 -08:00
Anders Kaseorg 279c4b0e24 puppeteer_tests: Port to TypeScript.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-22 16:03:10 -08:00
Anders Kaseorg 42f4156b36 test-js-with-puppeteer: Error on unhandled promise rejections.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-22 16:03:10 -08:00
Anders Kaseorg ca0b01033d lint: Remove custom whitespace rules already enforced by Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-17 22:21:42 -08:00
Wesley Aptekar-Cassels 4173070b3d notifications: Fix composebox notification escaping
The "Narrow to PM with" notification above the composebox was
double-escaped, mangling names with single quotes in them. This removes
the escaping in i18next, causing the name to be escaped only in
handlebars.
2021-02-17 14:04:30 -08:00
Anders Kaseorg 57f2b8760a shared: Format our one Flow file with Prettier.
Why not.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-14 07:03:32 -05:00
Anders Kaseorg f21fbea7c7 lint: Replace (most of) pycodestyle with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 5028c081cb python: Merge concatenated string literals that Black would uglify.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 1a4f70f1bc lint: Convert sudo exclusion to double quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 08:34:43 -08:00
Anders Kaseorg fc2900c004 js: Convert static/js/settings_config.js to ES6 module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-10 10:27:14 -08:00
Anders Kaseorg bbf7bd949b eslint: Enable lines-around-directive.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-10 10:27:14 -08:00
aryanshridhar f92f99d92d dependencies: Replace moment.js with date-fns.
Replaced methods/functions of moment.js with date-fns library.
The motive was to replace it with a smaller frontend timezone library.

Date-fns ~ 11.51 kb
moment.js ~ 217.87 kb

Some of the format strings change because date-fns encodes them
differently from how moment did.

Fixes #16373.
2021-02-05 11:04:32 -08:00
Ganesh Pawar 7eeca9da46 provision: Add provision support for Ubuntu 20.10(Groovy).
PostgreSQL 13 is used when os_version is 20.10.
2021-02-05 09:30:34 -08:00
Anders Kaseorg aa650a4c88 js: Escape strings interpolated into CSS selectors with CSS.escape.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-04 11:00:06 -08:00
Anders Kaseorg 4bcf7131c1 push-to-pull-request: Wait for GitHub background update before merge.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-03 14:55:08 -08:00
aditya-mitra b8f57fa6bb provision: Add provision support for KDE Neon 20.04.
PostgreSQL 12 is used for Neon 20.04, which is a fork of Ubuntu 20.04.
2021-02-03 11:51:05 -08:00
Ganesh Pawar 5802bbfb98 ui_report: Add ui_report.client_error. 2021-01-30 07:48:44 -08:00
Riken Shah baae227efb run-dev: Fix dev url showing extra port.
This commit fixes the issue of `run-dev.py`
showing the development URL with an
extra port when EXTERNAL_HOST
is specified.

Fixes: #17054
2021-01-27 11:37:26 -08:00
Anders Kaseorg 2db2f2d6cd dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-25 14:53:19 -08:00
Aman Agrawal 599b86241f postcss-simple-vars: Add media query breakpoint variables. 2021-01-25 14:49:32 -08:00
Ganesh Pawar eefa687832 run-dev: Suppress the notices made by third-party tools.
This limits the `run-dev.py` startup output.
And made the terminal message a bit more clear about
accessing the server.

Fixes #16846
2021-01-22 18:00:30 -08:00
Anders Kaseorg 50121cce5e push-to-pull-request: Add --merge option.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-22 18:00:25 -08:00
Anders Kaseorg 26a7b7196c push-to-pull-request: Use getopt.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-22 18:00:25 -08:00
Steve Howell c693ae8982 event tests: Cover do_update_user_status better.
We often send only one field (away or status_text)
to be updated.

So we have to make our schema support optional
keys.

As a result of the more flexible schema, we no
longer need to exempt the node fixtures from
our schema checks.
2021-01-20 13:17:32 -08:00
Steve Howell 09ff0c400d tools: Rename tool to check-schemas. 2021-01-20 13:17:32 -08:00
Aman Agrawal c685d36821 hipchat_import: Remove tool from codebase.
Remove functions and scripts used by HipChat import tool and
those which will no longer be required in future.
2020-12-23 08:28:49 -08:00
Anders Kaseorg 0c4239e387 js: Convert Object.assign({…}, …) to spread syntax.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-21 15:37:36 -08:00
Mateusz Mandera 160cc5120a api: Require can_create_users permission to create users via API.
Allowing any admins to create arbitrary users is not ideal because it
can lead to abuse issues.  We should require something stronger that
requires the server operator's approval and thus we add a new
can_create_users permission.
2020-12-21 13:20:21 -08:00
Tim Abbott eca67135d1 run-dev: Simplify output and colorize link users should click. 2020-12-20 12:11:16 -08:00
Tim Abbott 47d513240c run-dev: Clean up unnecessary memcached output, and document flush. 2020-12-20 12:11:16 -08:00
Tim Abbott 38ffaad325 django: Add custom runserver wrapper to limit startup logging.
This helps considerably in avoiding the `run-dev.py` startup output
confusing developers.
2020-12-20 12:11:16 -08:00
Sutou Kouhei 0d3f9fc855 install: Use PGroonga packages built for PostgreSQL packages by PGDG
Because we always use PostgreSQL packages by PGDG since Zulip 3.0.

Fixes #16058.
2020-12-18 15:38:21 -08:00
Anders Kaseorg 9359dee75b test_settings: Use TEST_EXTERNAL_HOST to override ‘testserver’ default.
This allows test-backend to work even if the user has EXTERNAL_HOST
set to something else.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-17 13:07:59 -08:00
Vishnu KS 9b6d92d322 docs check: Check only the file and directory GitHub urls of zulip/zulip.
f82cc4ed06 started checking all
zulip/zulip GitHub links in CI. Instead, it should have checked only
zulip/zulip file and directory links since checking other
links require making requests to GitHub servers.
2020-12-16 15:13:25 -08:00
Vishnu KS 0d7a290893 docs check: Improve the names of GitHub url variables. 2020-12-16 15:13:25 -08:00
ryanreh99 10559c05ee build_pygments_data: Include zulip specific data in langs.
This fixes a bug where the typeahead did not include the
zulip specific langs (such a `quote`, `spoiler` and `math`)
as these weren't passed to the typeahead's source.

Introduced in af64c52166.

Fixes #16862.
2020-12-15 16:41:19 -08:00
Tim Abbott 90354c4e5f tools: Fix running test-js-with-node from arbitrary paths. 2020-12-15 12:18:23 -08:00
Anders Kaseorg c7a6902fe3 eslint: Remove and prohibit unused eslint-disable directives.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-10 18:08:45 -08:00
Anders Kaseorg b3aa44c914 install-shfmt: Upgrade shfmt from 3.1.2 to 3.2.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-09 23:07:40 -08:00
Anders Kaseorg bfd9999cf8 desktop_integration: Handle logout et al. events from desktop app.
I added these hooks in Zulip Desktop 5.5.0; handling these events in
the frontend will let us remove the janky desktop-side fallback code
that uses fake click events on menu items with specific indexes.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-12-08 16:39:17 -08:00
Isaac Evans 88a1204ce0 semgrep: Update rule syntax documentation URL. 2020-12-03 17:30:11 -08:00
Dinesh 85d7cdbf50 puppeteer: Add clear success message at end of a test run.
Also makes the failure message more noticeable.
2020-11-29 18:08:20 -08:00
Anders Kaseorg af64c52166 build_pygments_data: Build langs without incorrect mutation.
The previous loop generated broken entries for duplicated language
aliases:

"juttle": {"priority": {"priority": 0, "pretty_name": "Juttle"}, "pretty_name": "Juttle"}

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 18:51:37 -08:00
Anders Kaseorg d0d8c358b3 lint: Migrate typing.Text check to semgrep.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 18:17:41 -08:00
Anders Kaseorg 8115f8417b build_pygments_data: Don’t recompute the name we already have.
This speeds up build_pygments_data from 6 seconds to 0.5 seconds.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-16 17:01:55 -08:00
Anders Kaseorg bcb571c187 test-backend: Fix argparse error type.
ArgumentError takes two arguments.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-11-10 14:17:22 -08:00
Vishnu KS d6743da139 docs test: Verify zulip/zulip file and directory links locally. 2020-11-05 10:07:43 -08:00
Vishnu KS cba292b4bb lint: Ensure that docs don't link directly to code lines. 2020-11-05 10:07:43 -08:00
YashRE42 967efc32d2 widgets: Remove tictactoe example widget.
Steve asked me to remove this, since the tictactoe game was always
intended as a proof of concept. Now that we have poll and todo
widgets, the sample code for tictactoe has much less value.

We replace the content and type in test_widgets.py to maintain
coverage.
2020-11-03 14:46:39 -08:00
Aman Agrawal 87cdd8433d home: Allow logged out user through home.
We allow user to load webapp without log-in. This is only
be enabled for developed purposes now. Production setups will
see no changes.
2020-11-02 17:07:12 -08:00
Vishnu KS 7a6285ede7 email testing: Store the SMTP settings in dev-secrets.
It's super annoying to set this up each time I have to test
email templates in gmail.
2020-10-30 11:50:30 -07:00
Anders Kaseorg cc55393671 python: Open text files as text to skip decode operations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-30 11:36:38 -07:00
Anders Kaseorg aaa7b766d8 python: Use universal_newlines to get str from subprocess.
We can replace ‘universal_newlines’ with ‘text’ when we bump our
minimum Python version to 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-30 11:36:38 -07:00
Vishnu KS d4f5cea559 docs: Mark chat.zulip.org as an external link for testing.
The comment explains the background, but basically there's no purpose
in trying to verify this via CI.
2020-10-29 16:42:17 -07:00
Alex Vandiver c1201bd69c provision: Rename variables to postgresql. 2020-10-28 11:57:03 -07:00
Alex Vandiver 2b0bbbb882 tools: Rename postgres to postgresql in tool names. 2020-10-28 11:57:02 -07:00
Alex Vandiver 5eb8064a1a install: Rename postgres options to postgresql. 2020-10-28 11:55:32 -07:00
Alex Vandiver 1f7132f50d docs: Standardize on PostgreSQL, not Postgres. 2020-10-28 11:55:16 -07:00
Alex Vandiver e124324050 puppet: Rename postgres_appdb in nagios to postgresql. 2020-10-28 11:51:52 -07:00
Anders Kaseorg 4e9d587535 python: Pass query parameters as a dict when making GET requests.
This provides automatic URL-encoding.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-27 13:47:02 -07:00
Alex Vandiver f0b11d62f6 provisioning: Support non-RSA SSH keys.
Use the parts of the JSON to determine the files it should create.
2020-10-27 13:43:45 -07:00
Alex Vandiver 7b2112baf3 lint: Add linters to help enforce profile/base layering. 2020-10-27 13:29:19 -07:00
Alex Vandiver 188af57296 puppet: Rename postgres_appdb to postgresql.
There is only one PostgreSQL database; the "appdb" is irrelevant.
Also use "postgresql," as it is the name of the software, whereas
"postgres" the name of the binary and colloquial name.  This is minor
cleanup, but enabled by the other renames in the previous commit.
2020-10-27 13:29:19 -07:00
Alex Vandiver c2185a81d6 puppet: Move top-level zulip deployments into "profile" directory.
This moves the puppet configuration closer to the "roles and profiles
method"[1] which is suggested for organizing puppet classes.  Notably,
here it makes clear which classes are meant to be able to stand alone
as deployments.

Shims are left behind at the previous names, for compatibility with
existing `zulip.conf` files when upgrading.

[1] https://puppet.com/docs/pe/2019.8/the_roles_and_profiles_method
2020-10-27 13:29:19 -07:00
Anders Kaseorg 31d0141a30 python: Close opened files.
Fixes various instances of ‘ResourceWarning: unclosed file’ with
python -Wd.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Alex Vandiver 50e9e2ed20 puppet: Make zulip::base include zulip::apt_repository.
There was likely more dependency complexity prior to 97766102df, but
there is now no reason to require that consumers explicitly include
zulip::apt_repository.
2020-10-22 11:30:53 -07:00
Anders Kaseorg 5c6202ca97 notifications: Use a dynamic SVG favicon for unread counts.
Closes #2304.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-21 07:43:00 -07:00
Anders Kaseorg 3e8e4b9cad check-thirdparty: Work around deficient Mypy annotations for debian.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-19 11:54:14 -07:00
YashRE42 e344ea25f8 landing-page: Rename to landing_page.css. 2020-10-19 11:50:39 -07:00
YashRE42 7ee3484b7b portico-styles: Rename to portico_styles.css. 2020-10-19 11:50:39 -07:00
Anders Kaseorg 0be17b6b00 semgrep: Treat ugettext_lazy like ugettext.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-18 14:31:15 -07:00
Mateusz Mandera 716df658fa queue_processors: Don't run test queues with run-dev.py. 2020-10-18 14:07:31 -07:00
Mateusz Mandera 269faa342a test-queue-worker-reload: Fix expected worker count.
We didn't bump this when adding some additional test queues, that run
with run-dev.py as of this commit.
2020-10-18 14:07:31 -07:00
Aman Agrawal bdc4721aee BaseDocumentationSpider: Don't crawl webapp. 2020-10-16 16:26:29 -07:00
Aman Agrawal 72b25553b2 BaseDocumentationSpider: Print correct link on error. 2020-10-16 16:23:33 -07:00
Anders Kaseorg 6099612a15 lint: Add shfmt as a linter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-15 15:16:00 -07:00
Anders Kaseorg dfaea9df65 shfmt: Reformat shell scripts with shfmt.
https://github.com/mvdan/sh

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-15 15:16:00 -07:00
Alex Vandiver 0d5760d59f install-aws-server: Force non-interactive dist-upgrade.
Installing an updated linux kernel package, as can happen during the
`apt dist-upgrade` done by the installer, can cause grub to pop up a
prompt to update its configuration file.  In an unattended headless
configuration, this will stop the installation.

Explicitly configure apt to be non-interactive, and prefer the newest
configuration, during the install.
2020-10-15 14:39:20 -07:00
Steve Howell a9356508ca events: Stop sending occupy/vacate events.
We used to send occupy/vacate events when
either the first person entered a stream
or the last person exited.

It appears that our two main apps have never
looked at these events.  Instead, it's
generally the case that clients handle
events related to stream creation/deactivation
and subscribe/unsubscribe.

Note that we removed the apply_events code
related to these events.  This doesn't affect
the webapp, because the webapp doesn't care
about the "streams" field in do_events_register.

There is a theoretical situation where a
third party client could be the victim of
a race where the "streams" data includes
a stream where the last subscriber has left.
I suspect in most of those situations it
will be harmless, or possibly even helpful
to the extent that they'll learn about
streams that are in a "quasi" state where
they're activated but not occupied.

We could try to patch apply_event to
detect when subscriptions get added
or removed. Or we could just make the
"streams" piece of do_events_register
not care about occupy/vacate semantics.
I favor the latter, since it might
actually be what users what, and it will
also simplify the code and improve
performance.
2020-10-14 10:53:10 -07:00
Anders Kaseorg dd48dbd912 docs: Add spaces to “check out”, “log in”, “set up”, “sign up” as verbs.
“Checkout”, “login”, “setup”, and “signup” are nouns, not verbs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-13 15:47:13 -07:00
Steve Howell 9df9934ed6 refactor: Pass realm to bulk_add_subscriptions.
I think it's important that the callers understand
that bulk_add_subscriptions assumes all streams
are being created within a single realm, so I make
it an explicit parameter.

This may be overkill--I would also be happy if we
just included the assertions from this commit.
2020-10-13 11:28:17 -07:00
Anders Kaseorg 7f69c1d3d5 python: Catch specific exceptions from requests.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:41 -07:00
Anders Kaseorg aabef3d9be python: Catch specific exceptions from orjson.
Followup to #16120.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:41 -07:00
Anders Kaseorg a109d68af0 create-test-api-docs: Remove unhelpful exception handler.
encode returns bytes, which cannot possibly be written to a text file.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-11 16:11:35 -07:00
Anders Kaseorg f81a5e87ed run-dev: Wait for children to exit on Ctrl+C after killing them.
In addition to being generally more correct, this works around a bug
in Node.js that causes webpack-dev-server to corrupt the terminal
state when exiting as a background process.

https://github.com/nodejs/node/issues/35536

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg 1c7db528bd test_server: Wait for run-dev after terminating it.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:15:19 -07:00
Anders Kaseorg f92224c744 eslint: Fix unicorn/catch-error-name.
https://github.com/sindresorhus/eslint-plugin-unicorn/blob/master/docs/rules/catch-error-name.md

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-07 16:00:33 -07:00
Alex Vandiver 150d782d6b production-verify: Fix puppet path. 2020-10-06 17:26:14 -07:00
Alex Vandiver 8cf37a0d4b queue: Add a tool to profile no-op enqueue and dequeue actions. 2020-10-06 17:26:14 -07:00
Alex Vandiver 3e6cc49ab5 lint: Add missing spaces at line wraps, remove periods. 2020-10-04 17:57:55 -07:00
Anders Kaseorg cf5ededa35 openapi: Use at most one inline subschema in allOf.
This fixes some of the warnings from openapi-generator.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-30 19:45:28 -07:00
Anders Kaseorg f644bf2063 webpack: Use Webpack’s mode option rather than repurposing env.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-28 10:57:08 -07:00
Tim Abbott 94a9fa1891 event_schema: Add documentation and rename a few functions.
This should help make this revised subsystem readable for more new
contributors.  We still need to make updates to the high-level
documentation.
2020-09-25 12:53:00 -07:00
Steve Howell f6e0171d02 event_schema: Split check_reaction into add/remove.
It happens that whether you add a reaction or remove
a reaction, we send the exact same fields, just using
a different op code.

This sort of symmetry is actually kind of rare, as
usually "add" events have more fields, and "remove" events
might just send an id of something to remove.

Our openapi schema treats these as two seperate events,
so we are more consistent with it, and it helps our
schema-checking tooling for node fixtures, too.

Note that we now have to exempt the two events from
our openapi checks, due to the is_mirror_dummy field
in the deprecated user block.  We can decide how to
handle this later--one possibility is to just add it
as an optional field on the event_schema side.
2020-09-25 11:43:20 -07:00
Steve Howell 0c4286222f event_schema: Extract check_realm_update_dict. 2020-09-25 11:43:20 -07:00
Steve Howell 4084f0b949 event_schema: Extract check_realm_user_add.
Note that we make the schema for profile_data
slightly more realistic, but it doesn't actually get
exercised by our current tests (apart from
making sure it's a dict), since we don't have
profile data for our test realm.

We also don't have the optional fields for bots,
since our tests don't exercise that, nor
delivery_email.

So we exempt realm_user_add_event from openapi
checks for now.

When we try to match the openapi specs better, we
will probably want to add a few tests to test_events.

Obviously getting good coverage for adding users
would be nice for all these scenarios:

    * delivery_email matters
    * bots
    * realm has profile fields
2020-09-25 11:43:19 -07:00
Steve Howell 6c74a44697 data_types: Generalize StringDictType.
This is a prep commit for supporting "presence"
events, where the key of the dictionary is some
arbitrary string like "website" but the value
of the dictionary is another dictionary itself
with keys that are more like variable names.
2020-09-25 11:43:19 -07:00
Steve Howell 4f3d5f2d87 event_schema: Extract check_realm_filters.
We have some known issues with representing
tuples in openapi, so we exempt realm_filters
from the relevant check.
2020-09-25 11:43:19 -07:00
Steve Howell e40a5400e5 event_schema: Extract check_muted_topics.
This also forces us to create TupleType.

We exempt this from the openapi check,
since we haven't figured out how to model
tuples in openapi with the same precision
as event_schema (and it may be impossible).

Long term we just want to stop dealing in
tuples, of course.
2020-09-25 11:43:19 -07:00
orientor 91ca1afe98 data_type: Add StringDict data type.
StringDict is a data type for representing dictionaries where
all keys and values are strings. Add this data type to data_types.py
and edit other files so that this data type is put to use and tested.

(slightly tweaked by @showell to remove a comment and shorten
a var name now that we have a proper data type)
2020-09-25 11:43:19 -07:00
Steve Howell d28c01284c event_schema: Extract check_hotspots.
This forces us to introduce a NumberType.
2020-09-25 11:43:19 -07:00
Steve Howell cf26151cea event_schema: Use realm_user_person_types.
For realm_user events, we now structure the
person type as a union of dicts, which is
more consistent with how we model this in
our openapi spec.
2020-09-25 11:43:19 -07:00
Steve Howell 10952394b0 test_events: Use int value of message_retention_days.
We also make our schema in event_schema reflect this,
which in turn makes us match the already accurate
openapi spec, so we no longer need to exempt four
types of events from our sanity checks.
2020-09-25 11:43:19 -07:00
Steve Howell 73e7f7edec check-node-fixtures: Compare python/openapi schemas.
We might want to rename the tool to something more
general now, since we are really reconciling three
things:

    - node fixtures
    - event_schema checkers for test_events
    - openapi specs

The way we compare python and openapi schemas is
as follows:

    - first convert openapi schemas to be build
      from DictType, ListType, etc. with from_opeapi

    - do a diff on the schemas

Most of the new code is just having the FooType
family of classes serialize themselves with schema().
2020-09-25 11:43:19 -07:00
Steve Howell aca641a4d1 refactor: Extract data_types module.
Defining types with an object hierarchy
of type classes will allow us to build
functionality that was impossible (or
really janky) with the validators.py
approach of composing functions.

Most of the changes to event_schema.py
were automated search/replaces.

This patch doesn't really yet take
advantage of the new FooType classes,
but we will use it soon to audit our
openapi specs.
2020-09-25 11:43:19 -07:00
Anders Kaseorg 3a39c6f67f dependencies: Downgrade simplebar to 5.2.1.
SimpleBar 6.0.0-beta.2 through -beta.6 are built with ES6 syntax (I
assume inadvertently: https://github.com/Grsmto/simplebar/issues/523),
and its latest tag has moved back to 5.2.1 anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-24 16:13:47 -07:00
Aman Agrawal 3ec23e1a9d fetch_initial_state_data: Handle case of web public guests.
user_profile will be None for web_public_guests here.  Hence, for
settings (of which most be inaccessible by web public guest),
which require a user_profile, we either set an empty value for
them or set them to a default value. This will help render
the frontend or extend support to our clients without breaking
a lot of code.

Tweaked by tabbott to add many comments.
2020-09-23 12:11:15 -07:00
Anders Kaseorg 5216b70691 lint: Remove custom rule already enforced by Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 18:11:21 -07:00
Mateusz Mandera cd9b194d88 queue: Eliminate useless "burst" concept in monitoring.
The reason higher expected_time_to_clear_backlog were allowed for queues
during "bursts" was, in simpler terms, because those queues to which
this happens, intrinsically have a higher acceptable "time until cleared"
for new events. E.g. digests_email, where it's completely fine to take a
long time to send them out after putting in the queue. And that's
already configurable without a normal/burst distinction.
Thanks to this we can remove a bunch of overly complicated, and
ultimately useless, logic.
2020-09-21 15:24:04 -07:00
Mateusz Mandera 2365a53496 queue: Fix a race condition in monitoring after queue stops being idle.
The race condition is described in the comment block removed by this
commit. This leaves room for another, remaining race condition
that should be virtually impossible, but nevertheless it seems
worthwhile to have it documented in the code, so we put a new comment
describing it.
As a final note, this is not a new race condition,
it was hypothetically possible with the old code as well.
2020-09-21 15:22:56 -07:00