Commit Graph

2128 Commits

Author SHA1 Message Date
Harshit Bansal 420327b558 provision: Install NPM packages in the beginning of provisioning process.
NPM packages should be installed at the beginning of the provisioning
process so that later in the provisioning process if a script requires
any NPM package it can use it. Earlier, we were installing NPM packages
in the last as the installation process can fail due to network issues
but since we now retry in case the installation fails, they can be
installed safely at the beginning of the process as well just like apt
packages.
2017-07-13 21:52:01 -07:00
Steve Howell 7e88fb25b3 Move sent_messages callbacks into transmit_message().
This mostly sets the stage for a subsequent commit to start
using client_message_id as the key into sent_messages.

It has the nice side effect of making it more explicit that
certain things should always happen when transmit_message()
succeeds.

This commit does regress our node test coverage a bit.
2017-07-13 23:42:27 -04:00
Steve Howell f6d670ae3d Extract sent_messages.js.
This is mostly straightforward moving of code out of compose.js.

The code that was moved currently supports sending time
reports for sent messages, but we intend to grow out the new
module to track more state about sent messages.

The following function names in this commit are new, but their
code was basically pulled over verbatim:

    process_success (was process_send_time)
    set_timer_for_restarting_event_loop
    clear
    initialize

All the code in the new module is covered by previous tests that
had been written for compose.js.  This commit only modifies
a few things to keep those tests.

The new module has 100% node coverage, so we updated `enforce_fully_covered`.
2017-07-13 23:42:27 -04:00
Jack Zhang e29893a420 Fix indentation for templates/zerver/integrations/index.html. 2017-07-13 20:37:09 -07:00
Brock Whittaker 1cf7ee966b help: Redesign /help/ pages to be a single page app.
This redesigns the /help/ page sets to be a single page app that uses
history.pushState to work the same as the old app.

The big new feature is that now we have the index in a nicely designed
left sidebar.
2017-07-13 15:43:44 -07:00
Tim Abbott 344af2a7c9 travis: Use a wrapper script for installing moreutils.
This works around a frequent problem we've been having with Travis
CI's apt repository being busted on startup.  This, in turn, caused
`apt-get install moreutils` to fail.

We attempt to fix this by adding our own retry logic.
2017-07-12 13:43:47 -07:00
Aditya Bansal de1dc50204 linter/pep8: Remove code infra to ignore the E261 violations. 2017-07-11 15:17:32 -07:00
Aditya Bansal 84e53f960d pep8: Add compliance with rule E261 to tests/test_upload.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 1b5e3190e3 pep8: Add compliance with rule E261 to tests/test_subs.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 2643a40176 pep8: Add compliance with rule E261 to tests/test_signup.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 59b26e582a pep8: Add compliance with rule E261 to tests/test_realm.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 0970d29ec1 pep8: Add compliance with rule E261 to tests/test_narrow.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ce70ded488 pep8: Add compliance with rule E261 to tests/test_messages.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 8c233198f8 pep8: Add compliance with rule E261 to tests/test_events.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ee97ea1356 pep8: Add compliance with rule E261 to tests/test_bugdown.py. 2017-07-11 15:17:32 -07:00
Abdul Qabiz 7d7f2f78c8 clean-repo: Optimize .pyc removal code.
This optimizes .pyc files removal.

Instead of removing all pyc files, it will only remove pyc files where
corresponding py file is not found.  This saves about 1s in the
startup time of `run-dev.py` (and thus also `test-js-with-casper`).

Fixes #5760.
2017-07-11 14:40:56 -07:00
Aditya Bansal 0c4165a5bd pep8: Add compliance with rule E261 to populate_db.py. 2017-07-11 11:56:50 -07:00
Aditya Bansal 856e005d64 linter_lib/pep8.py: Remove files which don't have E261 conflicts. 2017-07-11 11:55:30 -07:00
Aditya Bansal d46cf59b0d pep8: Add compliance with rule E261 to worker/queue_processors.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal 1e3b57570e pep8: Add compliance with rule E261 to tornado/websocket_client.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal 94b790c28d pep8: Add compliance with rule E261 to tornado/socket.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal b9bffaa446 pep8: Add compliance with rule E261 to prod_settings_template.py. 2017-07-11 11:55:01 -07:00
Aditya Bansal 5f87a0f8f8 pep8: Add compliance with rule E261 zproject/settings.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 7c6ad4a770 pep8: Add compliance with rule E261 zproject/dev_settings.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 44edab1cff pep8: Add compliance with rule E261 tools/run-dev.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 73ad834129 pep8: Add compliance with rule E261 bugdown/__init__.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal f5d5c48831 pep8: Add compliance with rule E261 models.py. 2017-07-11 11:53:33 -07:00
Tim Abbott f75b72ef07 test-migrations: Fix error code bug.
Previously, this was failing unconditionally, because of `set -e`.
2017-07-10 16:20:47 -07:00
Aditya Bansal c493a0f02a test-migrations: Add a check to avoid auto named migrations. 2017-07-10 16:02:52 -07:00
Aditya Bansal 0f2c208a5b tools: Add new tool to pretty-print-html. 2017-07-09 07:01:29 -04:00
neiljp (Neil Pilgrim) 33a6a633eb mypy: Amend tools/check-css functions from Text to str. 2017-07-08 10:49:42 -07:00
Tim Abbott f2379d66a2 i18n: Fix capitalization of Zulip Botserver download code. 2017-07-07 17:02:39 -07:00
Brock Whittaker bd664b0dd5 portico: Fix background color; remove need for bg-image classes. 2017-07-07 15:30:47 -07:00
Jack Zhang c5cdede891 portico-css: Reposition header, portico-page content, and footer. 2017-07-07 15:29:43 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Tim Abbott cd4f35abdf test-backend: Don't run tests in parallel with given test files.
It's usually slower to run in parallel with a single file.
2017-07-07 13:36:03 -07:00
Tim Abbott 919c610abf capitalization: Add APNS to list of valid acronyms. 2017-07-07 10:43:02 -07:00
Tim Abbott 774216713b provision: Document more clearly the is_production_travis block. 2017-07-06 17:39:49 -07:00
sinwar 49db0855d2 provision: Avoid spending ~3s on RabbitMQ in no-op case.
Tweaked by tabbott to reorganize the logic more clearly.

Fixes #5180.
2017-07-06 17:38:39 -07:00
Rishi Gupta c827c7840d lint: Re-indent digest.html and missed_message.html. 2017-07-06 15:46:03 -07:00
James Rowan 0d76da27b4 emails: Add base template for missed message and digest emails.
Also removes the files from lint, so that the new indentation doesn't
obscure the change. The files are reindented in the next commit.
2017-07-06 15:46:03 -07:00
Eeshan Garg 7394734ac3 tools/test-api: Stop inserting api/ in PATH.
Now that we have decided to move our API bindings to a separate
repo and specify them as a pip dependency, we can stop modifying
PATH to include the api/ directory.
2017-07-06 12:24:38 -07:00
rht 8aaf34d06d Lock the recursive requirements in common.txt via pip-compile.
The commit is composed of: (1) distill out top-level dependencies in
common.txt, (2) add -e flag to the vcs-based packages because
pip-compile can't do without, (3) pip-compile/generate the locked files
then remove the -e flags from the lockfile, (4) pin pathlib2 to dev.txt
because it turns out it is a direct requirement of
documentation_crawler, (5) document the structure and add an automation
script (6) remove cryptography==1.9 from requirements/scrapy.txt since
cryptography is automatically added from pyopenssl (7) add sed command
to remove future/futures from the generated lock file in python3 (this
should have been automatically handled by pip-compile, so pending for
the feature from pip-compile)

Tweaked by tabbott to update PROVISION_VERSION and add a missing
`first` dependency.
2017-07-05 23:47:07 -07:00
Tommy Ip 61e24cfbea tools: Remove unused `bright_red_output` context manager.
This context manager is no longer in use since the linter visual
updates.
2017-07-05 23:31:09 -07:00
Tommy Ip 2dc35e6c11 linter: Update custom linters' visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip d188c54ebc linter: Update pyflakes visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip cff2b1ba9e linter: Update pep8 visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip 766abaa1a3 linter: Update external_linter visual output. 2017-07-05 23:31:08 -07:00
Tommy Ip e0d4a007e8 linter: Add `identifier` argument to `custom_check_file` function.
In preparation for the upcoming linter output update, each custom
linter now has a identifier argument. This would be used for separating
each line of lints, eg:

    py        | Missing space around "%" at analytics/views.py line 485:
    py        |         output += '<hr>%s\n'%(string_id,)
    css       | 64c64,65
    css       | <     opacity: 1; }
    css       | ---
    css       | >     opacity: 1;
    css       | > }
    css       | static/styles/lightbox.css seems to be broken:
    swagger   | In static/swagger/zulip.yaml:
    swagger   | Duplicate operationId: registerQueue
2017-07-05 23:23:45 -07:00
Tommy Ip 3b8864bafa linter: Create error printing library.
For performance reasons, we spawn each linter in a separate OS thread.
The downside of this is that all lints would end up in stdout without
much visual separation, resulting in confusing error log. This commit
introduce the `print_err` function, which shows which linter each line
of lint is from.
2017-07-06 13:46:10 +08:00
Steve Howell df4b306c20 coveragerc: Exclude zerver/tornado/ioloop_logging.py.
The file ioloop_logging.py is pretty small, but it seems to have
flaky time-sensitive code that creates noise for coverage deltas
when we do builds.  Since it's small, we just exclude it.

Tweaked by tabbott to add comments.
2017-07-05 15:07:52 -07:00
Cory Lynch 099a7052e4 Enforce 100% test coverage for stream_events.js. 2017-07-05 17:57:52 -04:00
Tim Abbott b2ada7e706 docs: Expand documentation on Internet access in tests. 2017-07-04 13:17:25 -07:00
Aditya Bansal 5b81cc2bcd tests: Block internet access to test suites by overriding requests. 2017-07-04 13:17:22 -07:00
Aditya Bansal 26c345f059 backend: Block network access to test suites.
In this commit we basically start to override the request method of
httplib2.Http() to raise an exception whenever it is called i.e.
a trial is made to access the network from test suits.

Fixes: #1472.
2017-07-04 13:17:20 -07:00
Umair Khan ca14292334 travis: Only generate translatation files once.
This should save us a second or two in Travis builds.
2017-07-03 10:10:43 -04:00
Umair Khan 15866b8a1f check-frontend-i18n: Add no-generate option.
Adds the option to not generate translation files.
2017-07-03 10:10:43 -04:00
Umair Khan f22b2d27f3 check-capitalization: Add no-generate option.
Adds the option to not generate translation files.
2017-07-03 10:10:43 -04:00
Umair Khan f1fded08ab check-frontend-i18n: Generate only English translations. 2017-07-03 10:10:43 -04:00
Umair Khan 50ee14d390 check-capitalization: Generate only English translations. 2017-07-03 10:10:43 -04:00
sinwar 3d0b5b14be provision: Avoid spending ~1.5s on compilemessages in no-op case.
This brings the total no-op provision time down to around 7.3s!

Tweaked by tabbott to clean up the code, close file descriptions, and
avoid issues with a somehow corrupted file.

Fixes #5185.
2017-07-01 11:27:54 -07:00
Harshit Bansal a876ebb52a build_emoji: Close open file objects after reading. 2017-07-01 11:14:24 -07:00
Harshit Bansal 8c92024fae build_emoji: Add `codepoint_to_name` dict to emoji_codes.js.
This dict is a mapping from codepoint to canonical name.
2017-07-01 11:14:24 -07:00
Harshit Bansal fee4065571 build_emoji: Generate `codepoint_to_name.json` file.
Store the `codepoint_to_name` map in a JSON file which can be used
by the rest of codebase(zerver) to convert emoji codepoints to
canonical names.
2017-07-01 11:14:24 -07:00
Umair Khan f60d4849b7 Revert "lint: Don't allow handlerbars within translatable text."
This reverts commit 3c7b9cdc94.
2017-06-30 10:12:31 -04:00
Umair Khan 74d83cc47e lint: Disallow handlebars within {{#tr}}{{/tr}}.
This commit adds a new linter which runs from tools/travis/backend.
It runs over the translations.json file and checks if any of the
translatable string contains handlebars in it.

Fixes #5544
2017-06-30 10:12:31 -04:00
Steve Howell 6cf4c65d92 node tests: Add 100% coverage for channel.js. 2017-06-30 09:48:04 -04:00
Rishi Gupta 85d38bd17b emails: Remove DEFAULT_FROM_EMAIL from prod_settings_template. 2017-06-29 17:54:33 -07:00
Umair Khan 3c7b9cdc94 lint: Don't allow handlerbars within translatable text.
Fixes: #5544
2017-06-29 07:39:20 -04:00
Steve Howell 198c363b89 Fix errors in create-test-api-docs.
Our recent Django upgrade removed some info from the file
used by this tool, but there is still useful info, so I
resurrected the tool.
2017-06-27 08:56:39 -04:00
Yago González 33e4855306 lint: Ban non-unique operationIds in Swagger.
The Swagger specification indicates that all operationId values should be
unique. However, SwaggerParser doesn't complain during validation if that
doesn't happen, so this commit adds our own method to identify these
cases.

Also, the violations of this rule have been fixed.
2017-06-26 22:02:26 -04:00
Yago González 00a7f89ba9 node tests: Add coverage to composebox_typeahead.js. 2017-06-26 19:33:25 -04:00
Yago González 7e3154f7bc lint: Ignore tildes in mustache tag template checker.
Handlebars allows putting tildes (~) by the braces in mustache tags to
strip whitespaces on that side of the tag.

    {{#if foo~}}
        <p>Bar</p>
    {{~/if}}

This way, the linter ignores any potential tilde at the ends of Handlebars
tags so we can use this feature without lint fails.
2017-06-26 19:33:25 -04:00
Aditya Bansal 1cb5c9faa8 node tests: Enforce 100% test coverage upon emoji.js. 2017-06-26 14:24:20 -04:00
Aditya Bansal d1fe75ce3b node tests: Enforce 100% node coverage in activity.js. 2017-06-26 14:24:20 -04:00
Eeshan Garg 4ad951f256 integrations/perforce: Migrate docs to Markdown. 2017-06-26 12:30:04 -04:00
Tommy Ip f521b03474 lint: Check if commit message is not empty before running gitlint. 2017-06-25 11:29:14 -04:00
Joshua Pan a63fd95b6a test-js-with-node: Add check for fully covered files that are not enforced. 2017-06-23 16:05:16 -04:00
Joshua Pan c625a80b90 test-js-with-node: Extract check_line_coverage function. 2017-06-23 16:05:16 -04:00
Joshua Pan 3557c47d76 node tests: Enforce 100% node coverage in user_events.js. 2017-06-23 16:05:16 -04:00
Aditya Bansal eb39e82629 Update enforce_fully_covered list with pm_list.js. 2017-06-23 11:13:47 -04:00
Aditya Bansal 18eeb962fe setup-static-analysis: Add wrapper to retry dependancy installation. 2017-06-23 08:23:45 -04:00
Aditya Bansal a20249b126 tooling: Add tool to clear unused emoji cache.
In this commit, we are adding a new tool which can be used to
clear out the emoji cache. Cache cleanup is a good thing to in
general and as well helps keep travis cache small (meaning faster
builds).

Credits to @HarshitOnGitHub as well for suggesting the use of
emoji symlink.
2017-06-22 19:17:12 -04:00
Harshit Bansal 0b03fd97a2 webpack: Don't include katex bundle in base.html.
Since we use katex only in the main webapp, we don't need it in things
like landing page etc, so it is better to move it to index.html.
2017-06-22 19:06:32 -04:00
Harshit Bansal 99b12ed0ca webpack: Use `common.js` to inject webpack dependencies in dev. 2017-06-22 19:06:32 -04:00
Harshit Bansal c0e4f24a25 webpack: Migrate `common.js` bundle to webpack. 2017-06-22 19:06:32 -04:00
sinwar 0e24930770 provision: Avoid spending ~3s on apt operations in no-op provision.
Fixes #5186
2017-06-22 09:37:40 -04:00
Vaida Plankyte 0ed4e22e54 docs: Fix lint/whitespace errors in GCI docs. 2017-06-22 09:30:28 -04:00
Robert Hönig d45d9c9ab5 bots: Add linter rule to prevent open() calls.
Addresses #5395.
2017-06-21 10:16:59 -04:00
Robert Hönig fd41a481e4 linter: Fix `include_only` rule path comparison.
In order for the `include_only` linter rule to not have
any side effects, we need to explicitly add a trailing '/'
after every directory we want to include.
2017-06-21 10:16:59 -04:00
Steve Howell e49afe3ebc lint: Add code to suggest removing exclusions.
If excluded lines no longer exist in a file, print a message
asking the user to remove the exclusion.
2017-06-21 08:12:58 -04:00
Joshua Pan bd9459f27d node test: Enforce 100% node coverage in typeahead_helper.js. 2017-06-21 00:03:05 -04:00
Steve Howell 572ea370ea Speed up custom checkers in `./tools/lint`.
The main thing here is to make looping over lines be the inner
loop, instead of looping over rules.  This keeps regexes in
cache, and it also avoids some O(N) checks.

This is a significant speedup for me, reducing time from 16s
to 11s.
2017-06-20 22:26:12 -04:00
Harshit Bansal c50b3672c2 webpack: Migrate `signup.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Harshit Bansal 3953734a1c webpack: Migrate `stats.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Umair Khan bccd2caacb testing: Fix search key.
The problem this commit solves is related to how we search for
testcases in the test files. We use a simple 'search_key' in file_data.
This will return a false positive if there is a 'longer_search_key' in
file_data.

While searching for TestCases we should use a longer key to remove the
possibility of collision. Using 'class TestCase(' should be precise
enough.

Fixes #4983
2017-06-20 08:35:07 -04:00
Cynthia Lin 9777496046 repository: Update information with Codecov migration. 2017-06-20 07:17:39 -04:00
Aditya Bansal b19134d4c2 travis: Remove docker-engine from held package list in production.
We are doing this in order to avoid facing troubles when travis
updates its trusty images. Here is a link to the blog post.
https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
2017-06-20 06:45:40 -04:00
Cory Lynch 005d06eb38 message_store.js: Add tests for message id change.
This gets message_store.js to 100% line coverage.
2017-06-20 06:24:27 -04:00
Aditya Bansal 7980462010 setup-backend: Add wrapper to retry provisioning in case it fails.
In this commit we add a wrapper around the provisioning inside
setup-backend to trigger another try at provisioning if it fails
in hopes of it will run this time fine if it was interuppted by a
network issue the first time.
Fixes: #2034.
2017-06-19 16:14:11 -04:00