Commit Graph

2091 Commits

Author SHA1 Message Date
Tim Abbott 774216713b provision: Document more clearly the is_production_travis block. 2017-07-06 17:39:49 -07:00
sinwar 49db0855d2 provision: Avoid spending ~3s on RabbitMQ in no-op case.
Tweaked by tabbott to reorganize the logic more clearly.

Fixes #5180.
2017-07-06 17:38:39 -07:00
Rishi Gupta c827c7840d lint: Re-indent digest.html and missed_message.html. 2017-07-06 15:46:03 -07:00
James Rowan 0d76da27b4 emails: Add base template for missed message and digest emails.
Also removes the files from lint, so that the new indentation doesn't
obscure the change. The files are reindented in the next commit.
2017-07-06 15:46:03 -07:00
Eeshan Garg 7394734ac3 tools/test-api: Stop inserting api/ in PATH.
Now that we have decided to move our API bindings to a separate
repo and specify them as a pip dependency, we can stop modifying
PATH to include the api/ directory.
2017-07-06 12:24:38 -07:00
rht 8aaf34d06d Lock the recursive requirements in common.txt via pip-compile.
The commit is composed of: (1) distill out top-level dependencies in
common.txt, (2) add -e flag to the vcs-based packages because
pip-compile can't do without, (3) pip-compile/generate the locked files
then remove the -e flags from the lockfile, (4) pin pathlib2 to dev.txt
because it turns out it is a direct requirement of
documentation_crawler, (5) document the structure and add an automation
script (6) remove cryptography==1.9 from requirements/scrapy.txt since
cryptography is automatically added from pyopenssl (7) add sed command
to remove future/futures from the generated lock file in python3 (this
should have been automatically handled by pip-compile, so pending for
the feature from pip-compile)

Tweaked by tabbott to update PROVISION_VERSION and add a missing
`first` dependency.
2017-07-05 23:47:07 -07:00
Tommy Ip 61e24cfbea tools: Remove unused `bright_red_output` context manager.
This context manager is no longer in use since the linter visual
updates.
2017-07-05 23:31:09 -07:00
Tommy Ip 2dc35e6c11 linter: Update custom linters' visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip d188c54ebc linter: Update pyflakes visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip cff2b1ba9e linter: Update pep8 visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip 766abaa1a3 linter: Update external_linter visual output. 2017-07-05 23:31:08 -07:00
Tommy Ip e0d4a007e8 linter: Add `identifier` argument to `custom_check_file` function.
In preparation for the upcoming linter output update, each custom
linter now has a identifier argument. This would be used for separating
each line of lints, eg:

    py        | Missing space around "%" at analytics/views.py line 485:
    py        |         output += '<hr>%s\n'%(string_id,)
    css       | 64c64,65
    css       | <     opacity: 1; }
    css       | ---
    css       | >     opacity: 1;
    css       | > }
    css       | static/styles/lightbox.css seems to be broken:
    swagger   | In static/swagger/zulip.yaml:
    swagger   | Duplicate operationId: registerQueue
2017-07-05 23:23:45 -07:00
Tommy Ip 3b8864bafa linter: Create error printing library.
For performance reasons, we spawn each linter in a separate OS thread.
The downside of this is that all lints would end up in stdout without
much visual separation, resulting in confusing error log. This commit
introduce the `print_err` function, which shows which linter each line
of lint is from.
2017-07-06 13:46:10 +08:00
Steve Howell df4b306c20 coveragerc: Exclude zerver/tornado/ioloop_logging.py.
The file ioloop_logging.py is pretty small, but it seems to have
flaky time-sensitive code that creates noise for coverage deltas
when we do builds.  Since it's small, we just exclude it.

Tweaked by tabbott to add comments.
2017-07-05 15:07:52 -07:00
Cory Lynch 099a7052e4 Enforce 100% test coverage for stream_events.js. 2017-07-05 17:57:52 -04:00
Tim Abbott b2ada7e706 docs: Expand documentation on Internet access in tests. 2017-07-04 13:17:25 -07:00
Aditya Bansal 5b81cc2bcd tests: Block internet access to test suites by overriding requests. 2017-07-04 13:17:22 -07:00
Aditya Bansal 26c345f059 backend: Block network access to test suites.
In this commit we basically start to override the request method of
httplib2.Http() to raise an exception whenever it is called i.e.
a trial is made to access the network from test suits.

Fixes: #1472.
2017-07-04 13:17:20 -07:00
Umair Khan ca14292334 travis: Only generate translatation files once.
This should save us a second or two in Travis builds.
2017-07-03 10:10:43 -04:00
Umair Khan 15866b8a1f check-frontend-i18n: Add no-generate option.
Adds the option to not generate translation files.
2017-07-03 10:10:43 -04:00
Umair Khan f22b2d27f3 check-capitalization: Add no-generate option.
Adds the option to not generate translation files.
2017-07-03 10:10:43 -04:00
Umair Khan f1fded08ab check-frontend-i18n: Generate only English translations. 2017-07-03 10:10:43 -04:00
Umair Khan 50ee14d390 check-capitalization: Generate only English translations. 2017-07-03 10:10:43 -04:00
sinwar 3d0b5b14be provision: Avoid spending ~1.5s on compilemessages in no-op case.
This brings the total no-op provision time down to around 7.3s!

Tweaked by tabbott to clean up the code, close file descriptions, and
avoid issues with a somehow corrupted file.

Fixes #5185.
2017-07-01 11:27:54 -07:00
Harshit Bansal a876ebb52a build_emoji: Close open file objects after reading. 2017-07-01 11:14:24 -07:00
Harshit Bansal 8c92024fae build_emoji: Add `codepoint_to_name` dict to emoji_codes.js.
This dict is a mapping from codepoint to canonical name.
2017-07-01 11:14:24 -07:00
Harshit Bansal fee4065571 build_emoji: Generate `codepoint_to_name.json` file.
Store the `codepoint_to_name` map in a JSON file which can be used
by the rest of codebase(zerver) to convert emoji codepoints to
canonical names.
2017-07-01 11:14:24 -07:00
Umair Khan f60d4849b7 Revert "lint: Don't allow handlerbars within translatable text."
This reverts commit 3c7b9cdc94.
2017-06-30 10:12:31 -04:00
Umair Khan 74d83cc47e lint: Disallow handlebars within {{#tr}}{{/tr}}.
This commit adds a new linter which runs from tools/travis/backend.
It runs over the translations.json file and checks if any of the
translatable string contains handlebars in it.

Fixes #5544
2017-06-30 10:12:31 -04:00
Steve Howell 6cf4c65d92 node tests: Add 100% coverage for channel.js. 2017-06-30 09:48:04 -04:00
Rishi Gupta 85d38bd17b emails: Remove DEFAULT_FROM_EMAIL from prod_settings_template. 2017-06-29 17:54:33 -07:00
Umair Khan 3c7b9cdc94 lint: Don't allow handlerbars within translatable text.
Fixes: #5544
2017-06-29 07:39:20 -04:00
Steve Howell 198c363b89 Fix errors in create-test-api-docs.
Our recent Django upgrade removed some info from the file
used by this tool, but there is still useful info, so I
resurrected the tool.
2017-06-27 08:56:39 -04:00
Yago González 33e4855306 lint: Ban non-unique operationIds in Swagger.
The Swagger specification indicates that all operationId values should be
unique. However, SwaggerParser doesn't complain during validation if that
doesn't happen, so this commit adds our own method to identify these
cases.

Also, the violations of this rule have been fixed.
2017-06-26 22:02:26 -04:00
Yago González 00a7f89ba9 node tests: Add coverage to composebox_typeahead.js. 2017-06-26 19:33:25 -04:00
Yago González 7e3154f7bc lint: Ignore tildes in mustache tag template checker.
Handlebars allows putting tildes (~) by the braces in mustache tags to
strip whitespaces on that side of the tag.

    {{#if foo~}}
        <p>Bar</p>
    {{~/if}}

This way, the linter ignores any potential tilde at the ends of Handlebars
tags so we can use this feature without lint fails.
2017-06-26 19:33:25 -04:00
Aditya Bansal 1cb5c9faa8 node tests: Enforce 100% test coverage upon emoji.js. 2017-06-26 14:24:20 -04:00
Aditya Bansal d1fe75ce3b node tests: Enforce 100% node coverage in activity.js. 2017-06-26 14:24:20 -04:00
Eeshan Garg 4ad951f256 integrations/perforce: Migrate docs to Markdown. 2017-06-26 12:30:04 -04:00
Tommy Ip f521b03474 lint: Check if commit message is not empty before running gitlint. 2017-06-25 11:29:14 -04:00
Joshua Pan a63fd95b6a test-js-with-node: Add check for fully covered files that are not enforced. 2017-06-23 16:05:16 -04:00
Joshua Pan c625a80b90 test-js-with-node: Extract check_line_coverage function. 2017-06-23 16:05:16 -04:00
Joshua Pan 3557c47d76 node tests: Enforce 100% node coverage in user_events.js. 2017-06-23 16:05:16 -04:00
Aditya Bansal eb39e82629 Update enforce_fully_covered list with pm_list.js. 2017-06-23 11:13:47 -04:00
Aditya Bansal 18eeb962fe setup-static-analysis: Add wrapper to retry dependancy installation. 2017-06-23 08:23:45 -04:00
Aditya Bansal a20249b126 tooling: Add tool to clear unused emoji cache.
In this commit, we are adding a new tool which can be used to
clear out the emoji cache. Cache cleanup is a good thing to in
general and as well helps keep travis cache small (meaning faster
builds).

Credits to @HarshitOnGitHub as well for suggesting the use of
emoji symlink.
2017-06-22 19:17:12 -04:00
Harshit Bansal 0b03fd97a2 webpack: Don't include katex bundle in base.html.
Since we use katex only in the main webapp, we don't need it in things
like landing page etc, so it is better to move it to index.html.
2017-06-22 19:06:32 -04:00
Harshit Bansal 99b12ed0ca webpack: Use `common.js` to inject webpack dependencies in dev. 2017-06-22 19:06:32 -04:00
Harshit Bansal c0e4f24a25 webpack: Migrate `common.js` bundle to webpack. 2017-06-22 19:06:32 -04:00
sinwar 0e24930770 provision: Avoid spending ~3s on apt operations in no-op provision.
Fixes #5186
2017-06-22 09:37:40 -04:00
Vaida Plankyte 0ed4e22e54 docs: Fix lint/whitespace errors in GCI docs. 2017-06-22 09:30:28 -04:00
Robert Hönig d45d9c9ab5 bots: Add linter rule to prevent open() calls.
Addresses #5395.
2017-06-21 10:16:59 -04:00
Robert Hönig fd41a481e4 linter: Fix `include_only` rule path comparison.
In order for the `include_only` linter rule to not have
any side effects, we need to explicitly add a trailing '/'
after every directory we want to include.
2017-06-21 10:16:59 -04:00
Steve Howell e49afe3ebc lint: Add code to suggest removing exclusions.
If excluded lines no longer exist in a file, print a message
asking the user to remove the exclusion.
2017-06-21 08:12:58 -04:00
Joshua Pan bd9459f27d node test: Enforce 100% node coverage in typeahead_helper.js. 2017-06-21 00:03:05 -04:00
Steve Howell 572ea370ea Speed up custom checkers in `./tools/lint`.
The main thing here is to make looping over lines be the inner
loop, instead of looping over rules.  This keeps regexes in
cache, and it also avoids some O(N) checks.

This is a significant speedup for me, reducing time from 16s
to 11s.
2017-06-20 22:26:12 -04:00
Harshit Bansal c50b3672c2 webpack: Migrate `signup.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Harshit Bansal 3953734a1c webpack: Migrate `stats.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Umair Khan bccd2caacb testing: Fix search key.
The problem this commit solves is related to how we search for
testcases in the test files. We use a simple 'search_key' in file_data.
This will return a false positive if there is a 'longer_search_key' in
file_data.

While searching for TestCases we should use a longer key to remove the
possibility of collision. Using 'class TestCase(' should be precise
enough.

Fixes #4983
2017-06-20 08:35:07 -04:00
Cynthia Lin 9777496046 repository: Update information with Codecov migration. 2017-06-20 07:17:39 -04:00
Aditya Bansal b19134d4c2 travis: Remove docker-engine from held package list in production.
We are doing this in order to avoid facing troubles when travis
updates its trusty images. Here is a link to the blog post.
https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
2017-06-20 06:45:40 -04:00
Cory Lynch 005d06eb38 message_store.js: Add tests for message id change.
This gets message_store.js to 100% line coverage.
2017-06-20 06:24:27 -04:00
Aditya Bansal 7980462010 setup-backend: Add wrapper to retry provisioning in case it fails.
In this commit we add a wrapper around the provisioning inside
setup-backend to trigger another try at provisioning if it fails
in hopes of it will run this time fine if it was interuppted by a
network issue the first time.
Fixes: #2034.
2017-06-19 16:14:11 -04:00
Aditya Bansal 924d8266c0 setup-production: Add a wrapper to retry provisioning when it fails.
In this commit we basically add a wrapper around provisioning
inside setup-production to trigger a retry in case of failure.
2017-06-19 16:14:11 -04:00
Aditya Bansal fab4a30cce production-helper: Add wrapper to retry apt-get dist-upgrade.
In this commit we add a conditional inside production helper to just
re run apt-get dist-upgrade in case it fails the first time.
2017-06-19 16:14:11 -04:00
vaibhav 33c0c00cd6 Outgoing webhook System: first Iteration of outgoing webhook UI. 2017-06-19 16:05:54 -04:00
Joshua Pan 23dc32a0f3 node tests: Enforce 100% coverage in markdown.js and presence.js. 2017-06-19 08:00:55 -04:00
Joshua Pan d1408125b2 tools: Make sure missing lines in node test coverage are correct. 2017-06-19 07:59:50 -04:00
Aditya Bansal 7de4c340f6 tooling: Add tool to clean unused NPM cache. 2017-06-19 07:55:23 -04:00
Tommy Ip e9fd41bb66 lint: Check if the first word of a commit message is imperative.
Only check the first word using a black list approach to reduce
force positive as shown in the current implementation.
2017-06-19 07:01:52 -04:00
Cory Lynch 7f8ecb62c9 fenced_code.js: Improve coverage to 100%. 2017-06-19 06:53:25 -04:00
Umair Khan eeabc0f6a3 Django 1.11: Bump Django version to 1.11.
This commit also makes the following changes:

* Bumps the version of Django to 1.11.2.

* Fixes the HTTP response headers. Now CommonMiddleware sets Content-Length
  header for non-streaming response, see
  https://docs.djangoproject.com/en/1.11/ref/middleware/#module-django.middleware.common.
  Due to this, 'Transfer-Encoding: chunked' header is removed, which signifies
  a streaming response.
2017-06-15 14:44:38 -07:00
Aditya Bansal a2b5d6efcb provision: Add wrapper to retry apt-get install if it fails first time. 2017-06-14 21:49:15 -07:00
Cory Lynch 3bc369f1cf tools: Add wrapper tool 'build-docs' for building documentation. 2017-06-14 20:16:02 -07:00
Eeshan Garg bbe7ef6c5e linter_lib: Exclude webhook URLs from long line checks.
Now, URLs starting with `{{ external_api_uri_subdomain }}` are
exempted from the 120 char per line requirement.
2017-06-14 17:42:17 -07:00
Aditya Bansal 914ab1040d integrations/codebase.html: Migrate to Markdown. 2017-06-13 16:32:00 -07:00
Tim Abbott 4d66d74938 check-templates: Remove exclude for some migrated files.
This integration docs have all been moved to markdown.
2017-06-13 16:30:27 -07:00
Aditya Bansal 9a38b51254 check-templates: Fix filter condition in Jinja2 template linting.
In this commit we basically stop ignoring 'base.html' from linting
point of view. Basically the condition was implemented in a bad way
resulting in ignorance of every template ending with `base.html`.
As well since removing it doesn't complain about the base.html
non parsable(We have made progress with our linter) we can safely
remove the filter condition.
2017-06-14 04:48:07 +05:30
Aditya Bansal 9cc3e8cf62 followup_day2.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 39a88019f8 followup_day1.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 9a0edad604 find_team.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 9fc16888ca digest.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal fa28dcd008 invitation_reminder.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 07e7482228 confirm_registration.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 1b82e90331 invitation.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal e12bb65c0b confirm_new_email.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Pweaver (Paul Weaver) 28874cf26f webpack: Add --minify option to run-dev.py for to test minification. 2017-06-13 12:45:40 -07:00
Mehanig 1e5ce918e3 webpack: Move app_debug.js to webpack bundle. 2017-06-13 12:45:40 -07:00
Mehanig 04626acd5a webpack: Move zxcvbn.js to webpack bundle. 2017-06-13 12:45:39 -07:00
Mehanig ae29d4b8d2 webpack: Move landing-page.js to webpack bundle. 2017-06-13 12:36:55 -07:00
Mehanig 2c6f2fb93a webpack: Move activity js to webpack. 2017-06-13 12:36:51 -07:00
Aditya Bansal 8e33d9e48b travis: Update held packages list in production tests. 2017-06-13 11:12:26 -07:00
Aditya Bansal d2f0e17454 linter: Fix bug resulting in <pre> being indented cyclically. 2017-06-13 08:39:59 -04:00
Rafid Aslam ef2c0f806e update-prod-static: Add code to copy CSS from node_modules 2017-06-11 10:15:48 -07:00
Harshit Bansal e30e244b29 emoji: Moved `unified_reactions.json` file.
Moved `unified_reactions.json` file from `tools/setup/emoji` to
`zerver/management/data` so that it can be used in production
environment.
2017-06-09 16:38:58 -07:00
Cory Lynch 6219128d87 dev: Add /emails/ URL for testing email templates.
Fixes #5270.
2017-06-09 16:21:49 -07:00
Cory Lynch 6653e19e3a Add URL for viewing node coverage data on dev server.
Running test-js-with-node --coverage now provides a URL
to view the output data on the web.

Fixes #5177.
2017-06-09 16:17:30 -07:00
Mehanig 8c0e500d6b webpack: Migrate api and katex js files to Webpack bundler.
This also injects the dev webpack sockets to katex, since that's part
of the main web app bundle.
2017-06-09 12:33:52 -07:00
Joshua Pan a72b1bd98e node tests: Enforce 100% test coverage for util.js. 2017-06-07 23:26:46 -07:00
Joshua Pan fdb244fd3b node tests: Add support for enforcing 100% test coverage.
Fixes #5178.
2017-06-07 22:13:02 -07:00