Commit Graph

3437 Commits

Author SHA1 Message Date
Anders Kaseorg bd7ef1be51 test-locked-requirements: Improve logged output.
Run pip-compile with --quiet so we don’t have to redirect its stderr;
then we can see any exceptions it might throw.  Print any resulting
diff in the right order and without extra newlines separating each
line.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-25 15:03:20 -07:00
Anders Kaseorg a053a925b3 requirements: Upgrade pip-tools to 3.8.0; downgrade pip to 19.1.1.
These are not the latest versions, but pip-tools 3.9.0 or 4.0.0 fails
to resolve dependencies from Git URLs:

pip._internal.exceptions.DistributionNotFound: No matching distribution found for zulip==0.6.1_git (from -r requirements/common.in (line 135))

while pip 19.2 breaks pip-tools 3.8.0:

TypeError: __init__() got an unexpected keyword argument 'find_links'

Fixes #10802.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-25 15:03:20 -07:00
Tim Abbott d0d16ed81c Revert "test_mattermost_importer: Fix missing coverage."
This reverts commit c48e53f587.

This new test code had nondeterministic failures; reverting so we can
debug without disruption.
2019-08-21 11:20:20 -07:00
Anders Kaseorg 349f22524e provision: Don’t rely on uid == gid assumption when invoking chown.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-17 12:59:09 -07:00
Anders Kaseorg 88793e80ea provision: Skip some rm, mkdir, chown commands if possible.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-17 12:59:09 -07:00
okmanl 2a1305de9f lint: Add a rule to avoid msgid as a Python variable name.
This is for consistency with our usual patterns, see #12995.  We will
need a similar commit for JavaScript to complete #12995.
2019-08-17 12:47:13 -07:00
Hemanth V. Alluri 243d8ffc51 openapi: Add a test to ensure generated curl examples work.
This new test runs each generated curl example against the Zulip API,
checking whether it returns successfully without errors.

Significantly modified by tabbott for simplicity.
2019-08-17 11:35:08 -07:00
Tim Abbott e8b4e80d45 test-api: Fix double use of django.setup.
django.setup is already called (with different/better environment
variables) inside test_server_running; we shouldn't be calling it just
before that to make imports work.

I discovered this because imports done at the wrong time would
potentially incorrectly have `testserver` as the EXTERNAL_HOST.
2019-08-17 11:35:08 -07:00
Priyank Patel 463ecb375f refactor: Add get_stream_by_narrow_operand_access_unchecked.
This let's us clean up the linter that excludes the use of get_stream
and by adding the access_unchecked in the name we make it clear that
it should be used with caution.

Refactoring idea by Tim Abbott.
2019-08-17 11:10:00 -07:00
Anders Kaseorg fc6a8396ed mypy: In non-daemon mode, follow package imports.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-16 14:13:40 -07:00
Anders Kaseorg 6bca996e2e webpack: Chill. Out. When watching for changes.
Polling for changes every 100 milliseconds was burning enough CPU to
set mid-2015 MacBooks on fire.  Use the default inotify watching,
except on filesystems where that’s known not to work (nfs, vboxsf), in
which case polling once per second is more than enough for even the
fastest typers.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-16 11:05:21 -07:00
Anders Kaseorg ead13c11e5 run-dev: Don’t rewrite the Content-Length header.
A HEAD response has a Content-Length but no body; it’s not correct in
that case to let Tornado default Content-Length to 0.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-12 16:47:41 -07:00
Anders Kaseorg b0859f4b1e linter_lib: Fix mypy errors.
tools/linter_lib/pyflakes.py:35: error: Argument 3 to "run_pyflakes" has incompatible type "List[Tuple[bytes, bytes]]"; expected "List[Tuple[str, str]]"
    tools/linter_lib/custom_check.py:110: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:214: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:214: error: Argument "shebang_rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:502: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:502: error: Argument "shebang_rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:519: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:706: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:728: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:738: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:779: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"
    tools/linter_lib/custom_check.py:779: error: Argument "length_exclude" to "RuleList" has incompatible type "Set[str]"; expected "List[str]"
    tools/linter_lib/custom_check.py:803: error: Argument "length_exclude" to "RuleList" has incompatible type "Set[str]"; expected "List[str]"
    tools/linter_lib/custom_check.py:805: error: Unsupported operand types for + ("List[Rule]" and "List[Dict[str, Any]]")
    tools/linter_lib/custom_check.py:819: error: Argument "rules" to "RuleList" has incompatible type "List[Dict[str, Any]]"; expected "List[Rule]"

These were missed the `zulint` package was missing PEP 561 type
annotation markers, and if it’d had them, mypy daemon mode would’ve
required us to set `follow_imports = skip` for it.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 17:22:45 -07:00
Anders Kaseorg 17acf95437 update-authors-json: Clean up type ignores.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 16:39:16 -07:00
Anders Kaseorg cd1306c8d9 test-install: Add bionic.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-09 16:27:03 -07:00
Anders Kaseorg 671844d667 provision: Add default-jre-headless for vnu-jar.
This was previously pulled in by yui-compressor.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-08 17:28:20 -07:00
Tim Abbott de0a41bc9c provision: Fix missing dependency on unzip.
Because this is often installed by default, we hadn't noticed that our
Slack importer doesn't run without it.

Thanks to Ray Kraesig for the report.
2019-08-08 10:49:20 -07:00
Tim Abbott d9e5becb0e tools: Fix running check-openapi locally. 2019-08-07 14:18:27 -07:00
Anders Kaseorg 0962393933 cleanup: Delete trailing newlines.
Delete trailing newlines from all files, except
tools/ci/success-http-headers.txt and tools/setup/dev-motd, where they
are significant, and static/third, where we want to stay close to
upstream.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
Anders Kaseorg becef760bf cleanup: Delete leading newlines.
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines.  Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
Wyatt Hoodes 8c26183c65 mypy: Assign warn_unreachable in mypy.ini.
Instead of doing this rather clumsily in `run_mypy`,
we configure the option in `mypy.ini`.
2019-08-06 12:43:29 -07:00
Hemanth V. Alluri f280e9cf84 lib: Rename lib/api_test_helpers.py to openapi/python_examples.py
This will make the contained code easier to find.
2019-08-05 21:06:19 -07:00
Hemanth V. Alluri c90056bdb2 tools: Move check-swagger to check-openapi and make it executable. 2019-08-05 21:06:19 -07:00
Anders Kaseorg 68dd8e4ec8 mypy: Migrate from mypy_extensions to typing_extensions.
This gives us access to typing_extensions.Deque, which was not added
to typing until 3.5.4.

(PROVISION_VERSION is not bumped because the transitive dependency set
in dev.txt hasn’t changed.)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-05 17:24:09 -07:00
Wyatt Hoodes e6ae160a0c mypy: Run --warn-unreachable by default. 2019-07-31 16:18:24 -07:00
Wyatt Hoodes f80402c64d tools: Use mypy_extensions for NoReturn.
This is the right way to access it with support for older Python
versions where it might not be in typing yet.
2019-07-31 16:18:24 -07:00
Wyatt Hoodes 4beec5c6b9 typing: Use TYPE_CHECKING when dealing with cyclic dependencies. 2019-07-31 12:19:39 -07:00
Tim Abbott 97b256d1f0 css: Extract rendered_markdown.scss.
This moves our main CSS for rendered Zulip message content into an
external file, which may be reusable but in any case should make it
easier to find this content.
2019-07-31 12:08:17 -07:00
Wyatt Hoodes a109508e34 typing: Remove now-unnecessary conditional import.
As a result of dropping support for trusty, we can remove our old
pattern of putting `if False` before importing the typing module,
which was essential for Python 3.4 support, but not required and maybe
harmful on newer versions.

cron_file_helper
check_rabbitmq_consumers
hash_reqs
check_zephyr_mirror
check_personal_zephyr_mirrors
check_cron_file
zulip_tools
check_postgres_replication_lag
api_test_helpers
purge-old-deployments
setup_venv
node_cache
clean_venv_cache
clean_node_cache
clean_emoji_cache
pg_backup_and_purge
restore-backup
generate_secrets
zulip-ec2-configure-interfaces
diagnose
check_user_zephyr_mirror_liveness
2019-07-29 15:18:22 -07:00
Aman 2183a74040 zulint: Use zulint from the extracted repository.
zulint will be added as a "third-party" dependency in zulip from now
on.  See the new project at https://github.com/zulip/zulint for more
details.
2019-07-26 11:35:43 -07:00
Wyatt Hoodes a920f0396d webpack: Comment why NoReturn can't import normally. 2019-07-25 17:48:33 -07:00
Wyatt Hoodes 927feb615e provision: Comment why NoReturn can't import normally. 2019-07-25 17:48:20 -07:00
Wyatt Hoodes 3060bb2208 requirements: Upgrade to mypy 0.720.
We also add the option of the newly added `--warn_unreachable`
flag.
2019-07-25 17:41:10 -07:00
Anders Kaseorg a32663dc85 lint: Do not forbid ES6 const.
Our pipeline was upgraded in commit
f54a63e2f9 (#12838).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-25 16:12:48 -07:00
Anders Kaseorg 6d5a20ac62 requirements: Remove django-pipeline.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-24 17:40:31 -07:00
Tim Abbott c2bdccbaa8 provision: Use a shorter threshold for clean-unused-caches.
We've had a few reports of folks running out of disk when we're making
rapid changes to dependencies, and this should help prevent that.
2019-07-24 13:35:00 -07:00
Anders Kaseorg 43c9a26afb setup_venvs: Use system python3.
Otherwise python3 will be perpetually copied from virtualenv to
virtualenv and will never receive updates from the system.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-23 16:33:04 -07:00
Anders Kaseorg 82828bdba4 HTML validation: Remove invalid <button href> attribute.
For .start-button, Bootstrap carousel already supports <button
data-target> as a valid alternative to <button href>.  For
.call-to-action, the margin is decreased to exactly offset the lack of
margin collapsing with display: inline-block.  There should be no
visual change.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-23 16:16:22 -07:00
Eeshan Garg 8e10ab282a webhooks: Log unexpected webhook events separately.
This change serves to declutter webhook-errors.log, which is
filled with too many UnexpectedWebhookEventType exceptions.

Keeping UnexpectedWebhookEventType in zerver/lib/webhooks/common.py
led to a cyclic import when we tried to import the exception in
zerver/decorators.py, so this commit also moves this exception to
another appropriate module. Note that our webhooks still import
this exception via zerver/lib/webhooks/common.py.
2019-07-22 18:20:53 -07:00
Anders Kaseorg f54a63e2f9 webpack: Transpile JS code with Babel.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-22 17:55:32 -07:00
Anders Kaseorg ecfb7c6a7f lint: Add TypeScript compiler as a linter.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-22 17:55:18 -07:00
Anders Kaseorg 41209dd32e build-release-tarball: Run with zulip-py-venv symlink present.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-21 18:43:52 -07:00
Rishi Gupta 7d8d0b2284 settings: Update upgrade text and styling.
When we add Plus, the first sentence should change to "Available on Zulip
Standard and Plus".

I copied the styling of .tip out of expediency, but it's also possible that
long term we'll want only 1 tip-like box styling.

The hover styling is a bit random, but I tried to copy other hover styles I
found in settings.scss.

Note that this renames .upgrade_realm_plan_type_suggestion to .upgrade-tip.
2019-07-21 14:32:36 -07:00
Wyatt Hoodes e331a758c3 python: Migrate open statements to use with.
This is low priority, but it's nice to be consistently using the best
practice pattern.

Fixes: #12419.
2019-07-20 15:48:52 -07:00
Tim Abbott c931e76cf2 docs: Rewrite docs on writing API documentation.
This had gotten badly out of date, since it wasn't updated when we did
the big migration to the OpenAPI documentation system.

Fixes part of #12571.
2019-07-19 18:00:01 -07:00
Anders Kaseorg e80b57a18a provision_inner: Remove unused lsb_release run.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 16:36:11 -07:00
Anders Kaseorg c5a4b0501b provision: Start a new Python process after activating the virtualenv.
Mismatching imports from outside and inside the virtualenv in the same
process was causing segfaults after apparently benign changes to the
script!

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 12:16:55 -07:00
Anders Kaseorg a7eb3faf96 storage: Move the staticfiles.json hack to ZulipStorage.
There’s no reason to monkey-patch something that we were already
subclassing.

Removing the PRODUCTION conditional causes us to generate
staticfiles.json in the right place to begin with so we don’t need to
move it later.  It also allows Django to find staticfiles.json if
running the dev server with PIPELINE_ENABLED.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:47:51 -07:00
Anders Kaseorg 695b79c5d8 generate_zulip_bots_static_files: Write to static, not STATIC_ROOT.
Otherwise the files aren’t processed by collectstatic and don’t end up
in the staticfiles.json manifest.

Signed-off-by: Anders Kaseorg <andersk@zulipchat.com>
2019-07-19 11:33:08 -07:00
Anders Kaseorg 5f712b960f update-authors-json: Write to static, not STATIC_ROOT.
Otherwise the file isn’t processed by collectstatic and doesn’t end up
in the staticfiles.json manifest.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:33:08 -07:00