Commit Graph

4035 Commits

Author SHA1 Message Date
Anders Kaseorg c21818e093 check-openapi: Disallow siblings of $ref.
The specification says “any sibling elements of a $ref are ignored”,
so their presence, although not invalid, indicates a mistake.  yamole
incorrectly merges these siblings into the referenced object, but we
should not rely on this nonstandard behavior.

https://swagger.io/docs/specification/using-ref/#sibling

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:57:13 -07:00
Tim Abbott 45cd979d46 tools: Fix webpack wrapper logic for finding config file to watch. 2020-09-12 11:31:44 -07:00
Alex Vandiver 9ea9752e0e webhooks: Rename UnexpectedWebhookEventType to UnsupportedWebhookEventType.
Any exception is an "unexpected event", which means talking about
having an "unexpected event logger" or "unexpected event exception" is
confusing.  As the error message in `exceptions.py` already explains,
this is about an _unsupported_ event type.

This also switches the path that these exceptions are written to,
accordingly.
2020-09-10 17:47:21 -07:00
Alex Vandiver 8016769613 webhooks: Move UnexpectedWebhookEventType into zerver.lib.exceptions.
8e10ab282a moved UnexpectedWebhookEventType into
`zerver.lib.exceptions`, but left the import into
`zserver.lib.webhooks.common` so that webhooks could continue to
import the exception from there.

This clutters things and adds complexity; there is no compelling
reason that the exception's source of truth should not move alongside
all other exceptions.
2020-09-10 17:47:21 -07:00
Anders Kaseorg b7b7475672 python: Use standard secrets module to generate random tokens.
There are three functional side effects:

• Correct an insignificant but mathematically offensive bias toward
repeated characters in generate_api_key introduced in commit
47b4283c4b4c70ecde4d3c8de871c90ee2506d87; its entropy is increased
from 190.52864 bits to 190.53428 bits.

• Use the base32 alphabet in confirmation.models.generate_key; its
entropy is reduced from 124.07820 bits to the documented 120 bits, but
now it uses 1 syscall instead of 24.

• Use the base32 alphabet in get_bigbluebutton_url; its entropy is
reduced from 51.69925 bits to 50 bits, but now it uses 1 syscall
instead of 10.

(The base32 alphabet is A-Z 2-7.  We could probably replace all of
these with plain secrets.token_urlsafe, since I expect most callers
can handle the full urlsafe_b64 alphabet A-Z a-z 0-9 - _ without
problems.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-09 15:52:57 -07:00
Dinesh 56546170cf puppeteer: Save pid of `run-dev.py --test` in var/puppeteer.
Also replaces a reference to casper in a comment related to --test.
2020-09-09 13:38:39 -04:00
Dinesh ad0f0564b1 puppeteer: Rename build_for_prod_or_casper to build_for_prod_or_puppeteer. 2020-09-09 13:38:39 -04:00
Dinesh c64888048f puppeteer: Rename CASPER_TESTS env variable to PUPPETEER_TESTS.
Also modified few comments to match with the changes.
2020-09-09 13:38:39 -04:00
Dinesh f2b3afcd56 puppeteer: Mention about docs and screeshots on failure. 2020-09-09 13:38:39 -04:00
Anders Kaseorg 3b301f522b python: Tweak some magic trailing commas to avoid Black bugs.
https://github.com/psf/black/issues/1658
https://github.com/psf/black/issues/1671

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg f91d287447 python: Pre-fix a few spots for better Black formatting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg bb4fc3c4c7 python: Prefer --flag=option over --flag option.
For less inflation by Black.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:51:09 -07:00
Anders Kaseorg 75c59a820d python: Convert subprocess.Popen.communicate to run or check_output.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 17:42:35 -07:00
Anders Kaseorg 37028d5abf zulip-export: Specify --stream argument as required.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg a50fae89e2 python: Elide type=str from argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg fbfd4b399d python: Elide action="store" for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg 1f2ac1962f python: Elide default=None for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg b4597a8ca8 python: Elide default for store_{true,false} argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg 3b257d10df tools: Delete tagmessages script.
It was broken by commit aaedec1fdb which
moved it to tools/i18n without adjusting its relative path references,
and it contains a sketchy injectable os.system call that I’d like to
remove.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:35:51 -07:00
Anders Kaseorg 5a1104f1b8 update-zuliprc-api-field: Add missing shell quoting.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:34:27 -07:00
Anders Kaseorg d751e0cece puppet: Don’t install netcat.
It’s been unused since commit 0af22dad18
(#13239).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 10:33:47 -07:00
Steve Howell 81b6583d28 node tests: Show coverage link when we regress coverage.
In 6653e19e3a we added
the convenient line to tell folks about the coverage
report.  But if we failed coverage checks, we didn't
show the link.

Arguably we should just always show this, even if
tests fail, but that can also be potentially confusing.
2020-09-03 07:57:58 -04:00
Steve Howell dc1795a3da node tests: Find files in Python.
The code to run single files was added
in c15695e514,
and it's just kinda strange code.

We already do a lot of file logic in Python
to check for line-coverage, so it's easier
to just have all the logic in Python.

This adds a new feature--you can now specify
the actual file:

    ./tools/test-js-with-node frontend_tests/node_tests/people.js

(This is helpful if you just want to use
shell autocomplete.)

Another minor change is that if you specify
individual files, we won't sort them.  This is
important when you're trying to hunt down test
leaks.

Finally, we have a nicer message if we can't find
the file.
2020-09-03 07:57:58 -04:00
Steve Howell e9bbfbc624 node tests: Clean up nyc arguments.
nyc was added in 29f04511c0

All the stuff after "&&" was actually passed to
node, because we didn't use shell=True, so the
"nyc report" command didn't run, and the ugly
finder.js code just skipped over all the final tokens.
2020-09-03 07:57:58 -04:00
Anders Kaseorg 0682d5c0f3 provision: Convert "".format to Python 3.6 f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 17:27:14 -07:00
Anders Kaseorg 424b441bb1 provision: Check for old Ubuntu or Python before starting Python.
This unblocks us from being able to use Python 3.6 syntax in
provision.py and its dependencies.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 17:27:14 -07:00
Anders Kaseorg 691f8f8e47 test_hash_reqs: Use mock from unittest.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:26:12 -07:00
Anders Kaseorg a276eefcfe python: Rewrite dict() as {}.
Suggested by the flake8-comprehensions plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg a5dbab8fb0 python: Remove redundant dest for argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:04:10 -07:00
Anders Kaseorg edaed497ed lint: Remove unused ignorelongline and lint:ignore comments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:00:44 -07:00
Anders Kaseorg 279e4b819e js: Elide .js and .ts extensions from imports and requires.
This will be required for TypeScript.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-01 16:43:02 -07:00
Anders Kaseorg 10bd7a47c4 webpack: Move webpack.config.ts to top level.
We lost the war against top level configuration files many moons ago.
This is what developers and tools expect.  And it seems to be required
for eslint-import-resolver-webpack (there’s ostensibly a {"config":
"tools/webpack.config.ts"} option, but it doesn’t work correctly:
https://github.com/benmosher/eslint-plugin-import/issues/1861).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-01 16:43:02 -07:00
Ryan Rehman b4f315698f node tests: Create file for `stream_edit.js`.
This fixes the regression introduced in the pervious
commit to regain the 100% line coverage in `user_pill.js`
as well as `stream_pill.js`.

The new `stream_edit.js` mainly tests for:

* The stream related queries of the typeahead in `user_pill.js`
* The "Add subscribers" event handlers.
* The event handler which displays the settings for a stream.
2020-08-31 16:57:32 -07:00
Ryan Rehman c6d9a87d6f streams: Allow creating stream pills to submit Add subscriber form.
We update the pills typeahead logic to also include
stream results and pass the "stream" key in `opts`
to enable this option for the Add subscriber form.

This commit implements the feature of adding all the
subscribers of another stream in the "Add subscribers"
UI, with the help of a new "stream_pill.js` file.

We temporarily add `user_pill.js` to the EXEMPT_FILES
list as typeahead will be set up in `stream_edit.js`
file which does not have any dedicated tests file.

Work towards #15186.
2020-08-31 16:57:32 -07:00
Aman Agrawal 6a2c7327cc boostrap: Reset to v2.3.2.
We merge bootstrap-responsive.css into bootsrap.css since that is
how bootstrap distributes it from this version onwards.

bootstrap.js has a lot of changes to it which completely breaks
our typeaheads and popovers, so we will have to override these
plugins with our version of these plugins. In future versions
of bootstrap when we use npm, we can just choose not to
import them.
2020-08-31 14:01:11 -07:00
Priyank Patel c1ba648b4f dependencies: Add xvfb dependency.
The node package allow use to control xvfb apt package in puppeteer
tests. This help us create a fake display so we can run puppeteer in
headful (headless: false) mode, which is required to use the chrome
extension desktop capture API.
2020-08-30 23:38:52 -07:00
Tim Abbott 2f5f5d7749 dependencies: Remove PhantomJS from development environment. 2020-08-30 17:16:02 -07:00
Tim Abbott 992c6126a8 docs: Update many references to Casper. 2020-08-30 17:16:02 -07:00
Dinesh 2d22d88700 casper: Remove few traces of casper.
Now that all casper tests have been migrated to
puppeteer, there's no need for having casper
related things.

Removed the casperjs package and removed/replaced
casper in few places with puppeteer.

Only removed few of them which I'm confident
about. Also didn't make any changes in docs
as it would be easier to remove them while
adding puppeteer docs.
2020-08-30 17:16:02 -07:00
Alex Vandiver 1e714d8a0f lint: Catch mock asserts whose leading "assert_" is missing. 2020-08-30 15:24:01 -07:00
Tim Abbott 26deac7904 lint: Add template check for invalid translation tag.
This rule is a bit marginal, in that we've only seen this mistake
once, but it is really subtle and took a while for translators to
notice it, so seems worth linting for anyway.
2020-08-27 11:46:41 -07:00
Tim Abbott dbf3894c49 test-backend: Enable --ban-console-output and document. 2020-08-27 11:39:53 -07:00
Mohit Gupta 3390a70bcd tests: Add extra console output detection in test-backend output.
This commit adds automatic detection of extra output (other than
printed by testing library or tools) in stderr and stdout by code under
test test-backend when it is run with flag --ban-console-output.
It also prints the test that produced the extra console output.

Fixes: #1587.
2020-08-27 11:39:53 -07:00
Aman 88821b960a vagrant: Add parallels as a provider.
There are file sharing issues with the macOS 10.15.6 and
vagrant. var/remote_cache_prefix was an empty file when using
VirtualBox and Docker on macOS.
Using parallels as a provider for vagrant fixes the issue.

Use --watch-poll which makes webpack to recompile
automatically on file changes, since inotify is not
working here too.
2020-08-14 16:03:51 -07:00
Steve Howell 196df919f2 minor: Improve error message for check-node-fixtures.
This is helpful now, but it will also reduce confusion
when we extend this tool to also look at openapi schemas.
2020-08-14 10:40:29 -04:00
Anders Kaseorg d0f4af5f8c python: Catch JSONDecodeError instead of ValueError when decoding JSON.
These weren’t wrong since orjson.JSONDecodeError subclasses
json.JSONDecodeError which subclasses ValueError, but the more
specific ones express the intention more clearly.

(ujson raised ValueError directly, as did json in Python 2.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-12 11:59:59 -07:00
Anders Kaseorg 42aa916967 sanity_check: Test for the virtualenv by its path.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 12:30:26 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Anders Kaseorg 768f9f93cd docs: Capitalize Markdown consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:23:06 -07:00