Commit Graph

2676 Commits

Author SHA1 Message Date
Greg Price 103178ffb2 run-mypy: Move "which mypy" noise into --version, and use in Travis.
Printing the version in Travis builds will help in debugging when we
get different results there from locally.  The new `--version` path
also gives us a handy place to put the "what mypy command are we running"
diagnostic, getting it out of the way of normal interactive use.
2017-09-27 12:22:03 -07:00
Greg Price 9ef68d6d42 run-mypy: Add --version to check the version of mypy used. 2017-09-27 12:22:03 -07:00
Greg Price c46755d48d run-mypy: Reorder CLI options for clearer --help. 2017-09-27 12:22:03 -07:00
Greg Price 6697e3620b run-mypy: Give CLI option help a consistent style.
Most CLI tools (including GNU tools and Mercurial) use lowercase
sentence fragments, with no period, for option glosses, so we
follow that style.  Also make the voice and wording consistent,
and the quote type in the Python source.
2017-09-27 12:22:03 -07:00
Greg Price 8c50f7f8a3 run-mypy: Remove extra exclude quirk for nonexistent "stubs" directory.
This exclusion was getting snuck in at the end bypassing --all (and so
giving the lie to the --help documentation).  There is no "stubs"
directory in the tree in any case.
2017-09-27 12:22:03 -07:00
Greg Price be5b26dc68 tools/run-mypy: Cut redundant data from CLI parsing.
argparse has reasonable default behavior for the `dest` argument,
and for `default` at least in these two obvious cases.
So cut those out where we're just doing the default thing.

Also rewrap a couple of calls to fit at least in 100 columns.
2017-09-27 12:22:03 -07:00
Robert Hönig fa8199819c linter: Remove unnecessary escape sequences in custom rules. 2017-09-27 09:02:44 -07:00
Robert Hönig 48696ac9a1 linter: Catch all malformatted string formatting clauses.
This also catches code like `"foo%s"%('bar')`, whereas before,
the first tuple entry could not be a string.
2017-09-27 09:02:44 -07:00
Robert Hönig 83eb22fc2f lister: Make excluding directories work on Windows. 2017-09-27 08:43:53 -07:00
Eeshan Garg bb34ba37b5 tools: Rename build_pygments_data.py -> build_pygments_data.
Dropping the file extension (.py) conforms to our naming
convention for such scripts.
2017-09-26 16:00:41 -07:00
Umair Khan 5d00f29c68 capitalization: Ignore LDAP. 2017-09-26 15:58:17 -07:00
Tim Abbott 8e2ae66fe6 provision: Update apt operations text. 2017-09-25 14:47:18 -07:00
Tim Abbott 16ed23414d provision: Fix spelling of apt_dependencies_hash.
Also use a variable to avoid duplicating the path.
2017-09-25 14:46:46 -07:00
Robert Hönig 41e6704b99 custom linter rules test: Add pattern test.
The pattern test method `test_rule_patterns` tests each rule by
fetching two strings from it: `test_good` and `test_bad`. Each
string is then presented as an input file to `custom_check_file`,
which should return True or False.
All lines in a string need to end with `\n`. Since the linter
expects an additional newline at the end of a file, the test case
adds `\n` to each string on top of that.

Fixes #6320.
2017-09-25 10:53:06 +02:00
Harshit Bansal 995a0f3cc0 build_emoji: Remove an unnecessary check for symlink support.
We already check for symlink support in provisioning script so no
need of doing so here.
2017-09-24 04:51:33 -07:00
Harshit Bansal 6ff7da04de emoji: Remove `NotoColorEmoji.ttf`.
We no longer use glyphs from `NotoColorEmoji.ttf` so removing this.
2017-09-24 04:51:33 -07:00
Harshit Bansal c8c1c8ef43 emoji: Remove `AndroidEmoji.ttf`. 2017-09-24 04:51:33 -07:00
Harshit Bansal 0d8e68d75e build_emoji: Some code cleanup. 2017-09-24 04:51:33 -07:00
Harshit Bansal 6b1525680e build_emoji: Remove now unused `MissingGlyphError` exception.
This exception was raised if there was no glyph available for a
codepoint. We no longer need this.
2017-09-24 04:51:33 -07:00
Harshit Bansal 898d7d1160 build_emoji: Remove now unused `bw_font()` function.
This function was used get a black and white glyph for an emoji if there
was no corresponding image file present in the `NotoColorEmoji.ttf` but
due to the new emoji farm setup code, we no longer need this.
2017-09-24 04:51:33 -07:00
Harshit Bansal b95fbbe29a build_emoji: Remove now unused `color_font()` function.
This function was used to color transparent number emojis. We no
longer need to do this since now we have remapped number emojis
to the corresponding colored emojis in the new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal 6ebacba425 build_emoji: Remove `code_point_to_file_name_map()`.
We have symlinked the old emoji farm to the old emoji farm and hence
we don't use the images from the `NotoColorEmoji.ttf` file. This
function was used to generate a map from codepoint to filename by
parsing the ttx file passed to it. We no longer need this map.
2017-09-24 04:51:33 -07:00
Harshit Bansal e295bed20f build_emoji: Extract `generate_map_files()` function.
This commit extracts out the `generate_map_files()` function from
the `dump_emojis()` function. This function generates various data
files like `emoji_codes.js`, `name_to_codepoint.json` etc which are
used by webapp, bugdown etc.
2017-09-24 04:51:33 -07:00
Harshit Bansal ef96adf7e5 build_emoji: Remove old emoji farm setup code.
This commit removes the old emoji farm generation code  in favor of
`setup_old_emoji_farm()`. Instead of having individual images in old
emoji farm we now symlink them to the images in the new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal 273e37b2f9 build_emoji: Add `setup_old_emoji_farm()` function.
This commit adds `setup_old_emoji_farm()` function to the build_emoji
script. This will change the way of setting up the old emoji farm.
Earlier we used to extract the glyphs corresponding to each emoji from
the `NotoColorEmoji.ttf` file. But since now we already have individual
images in the new emoji farm(from iamcal's 'emoji-datasource-google' npm
package) we can just symlink old emoji files to the new image files. This
apart from helping us in cleaning up the `build_emoji` script will also
help in reducing the increased size of the release tarball due to the
addition of new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal eed7d4b8d7 build_emoji: Remove old code in favor of `generate_sprite_css_files()`. 2017-09-24 04:51:33 -07:00
Harshit Bansal beab00a5e1 build_emoji: Add `generate_sprite_css_files()` function. 2017-09-24 04:51:33 -07:00
Harshit Bansal dcf6f1e0f0 build_emoji: Build new emoji farm.
This commit adds the 'setup_emoji_farm()' which copies the individual
images from `emoji-datasource-google` packge to target emoji dump.
2017-09-24 04:51:33 -07:00
Eeshan Garg 390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Vishnu Ks 61e48734a7 provision: Add moreutils in APT common dependencies.
This fixes an issue where `tools/update-locked-requirements` would fail.
2017-09-22 14:04:57 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
Tim Abbott b1dafe5d9d renumber-migrations: Fix last_correct_migration code. 2017-09-22 07:51:58 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Steve Howell 6ce68dfc3d minor: Remove obsolete lint check.
Some code that we had a lint exception for got cleaned up in a
recent commit.
2017-09-20 13:05:04 -07:00
Steve Howell f13cb94d84 Have exclude_topic_mutes() accept a stream id.
We want to convert stream names to stream ids as close
to the "edges" of our system as possible, so we let our
caller do the work of finding the stream id for a stream
narrow.
2017-09-20 10:31:33 -07:00
Tim Abbott 02d931a53e tools: Fix running full-stack tests from tools/ directory.
Previously these tests required you to run them with the root of the
Zulip repository as the current working directory, just due to
sloppiness.

We clean this up, while also making the path handling more consistent
and involving less fragile code.

Fixes #4169.
2017-09-17 11:12:51 -07:00
Vishnu Ks f4c69bb126 tools: Create tool for inlining email CSS during provision. 2017-09-16 07:17:48 -07:00
invinciblycool 5475c5cedb tests: Replace test_user_agent_parsing with a normal test.
Previously, this was its own separate test script; now it's a normal
part of the test suite.

Tweaked by tabbott to use a proper test method.

Fixes #6327.
2017-09-16 03:56:18 -07:00
Greg Price ceca96899a deps: Adjust the header comments in lock files to reflect our tools. 2017-09-15 00:23:20 -07:00
Greg Price 7c5f3d7be1 deps: Refactor tools/update-locked-requirements to save repeating the filenames.
My first version of this just replaced the repeated list of two output
files with an array variable, but I decided `"${outputs[@]}"` was too
much to ask people to understand, and the alternative of `$outputs`,
unquoted, encourages bad habits of shell programming.  So just handle
one file at a time; the only at all expensive part here is `pip-compile`.

I am tempted to move this to Python, but holding back.
2017-09-15 00:23:20 -07:00
Greg Price a171f23d63 deps: Remove a redundant `deactivate`.
This has no effect except on the running shell; so at the end
of a script, it has no meaningful effect at all.
2017-09-15 00:23:20 -07:00
Greg Price b362e21ebb deps: Simplify tools/update-locked-requirements slightly.
This makes the code a little more concise and also more uniform,
treating `future` the same in prod and in dev.  The non-uniformity
looks like an oversight in 2be8a793e, one of the commits that updated
this code for the Python 3-only world.
2017-09-15 00:23:20 -07:00
Greg Price bab2078705 deps: Explain the `-e` oddness, and handle it uniformly. 2017-09-15 00:23:20 -07:00
Greg Price 806220cafa deps: Organize tools/update-locked-requirements a bit further.
A couple of remarks still reflected the Python 2 world; and the
"Remove the editable flag" comment was confusingly above a line
that it looks like it could be talking about, but isn't.
2017-09-15 00:23:20 -07:00
Supermanu 184bfe7cfd docs: Expand the french translation style guide.
As the french word organisation is used, the related line is excluded
from the linter spell check.
2017-09-14 08:35:51 -07:00
Robert Hönig 5294e10801 custom linter rules test: Load rules in test setup. 2017-09-14 04:17:03 -07:00
Robert Hönig 402d7fca50 test_linter_custom_check.py: Fix typo in docstring. 2017-09-14 04:17:03 -07:00
Umair Khan df70fa962e linter: Improve i18n related regexes.
There are four regexes which try to ensure that the i18n strings are
properly captured.

1) The one which disallows multiline strings.
```
i18n\.t\([^)]+[^,\{\)]$

// Disallows:
i18n.t('some '
       + 'text');
```

2) The one which disallows concatenation within argument to i18n.t():
```
i18n\.t\([\'\"].+?[\'\"]\s*\+

// Disallows:
i18n.t("some " + "text");
```

3) There are two which disallow concatenation with i18n.t():
```
i18n\.t\(.+\).*\+

// Disallows:
i18n.t('some text') +

\+.*i18n\.t\(.+\)

// Disallows:
+ i18n.t('some text')
```

The ideal case is that you try to bring the string argument to the
i18n.t() on one line. In case this is not possible, you can do the
following:

```
var1 = i18n.t("Some text to be translated");
var2 = i18n.t("Some more text to be translated");
complete = var1 + var2;
2017-09-13 12:58:27 -07:00
Robert Hönig 53436d9ea7 linter: Add test case for custom_check.py
Fixes #5406.
2017-09-12 03:29:18 -07:00
Robert Hönig b5cf31efdc linter: Move custom_check_file to global namespace.
This is needed in order to mock the method when testing
`custom_check.py`. The diff for this commit is a bit broken;
all it really does is moving the method out of `build_custom_checkers`.
2017-09-12 03:24:59 -07:00
Robert Hönig 7a2234dfa2 linter: Remove redundant custom rule file delimitors. 2017-09-12 03:24:59 -07:00
Robert Hönig 2248c7ac0d linter: Add explanation for custom rule file delimitation operators. 2017-09-12 03:24:59 -07:00
Steve Howell 4dfe6bb320 Add migration to fix unread messages. 2017-09-07 07:06:03 -07:00
Robert Hönig 34e3329919 linter: Add rule against misspelled "Terms of Service". 2017-09-07 06:29:33 -07:00
Umair Khan bd43c447c3 travis: Log the Python version.
This allows us to ascertain the Python version under which we ran the
tests.
2017-09-06 13:48:50 -07:00
Umair Khan c039f73656 setup_venv: Link to the correct Python interpreter.
Travis enables different Python versions through virtual environments,
but it seems that there is a little caveat when we try to create Zulip's
virtual environment by referring Travis' virtual environment; Zulip's
virtual environment refers the system Python. We encountered this
behaviour when we tried to run our backend test suite under Python 3.5
in Travis. 'python3 --version' command before activating Zulip's
virtualenv showed 'Python 3.5.3' and after it showed 'Python 3.4.3'.
This happened when we created the virtual environment using
'virtualenv -p python3'.

The solution seems to be to explicitly give the path of the Python
interpreter in the Travis' virtual environment using 'which python3'.
2017-09-06 13:48:50 -07:00
Aditya Bansal d7f10bf3a1 profiling: Add snakeviz to dev tools.
This adds snakeviz to dev tools and also updates the message displayed
upon running `test-backend` with `--profile` option to say how to run
snakeviz correctly when using vagrant development environment.
2017-09-06 13:46:24 -07:00
Umair Khan ee6235d716 lint: Add regex to disallow i18n string concatenation.
Fixes #6398
2017-09-06 07:01:43 -07:00
Greg Price 1ab89b1d89 tools/test-all: Split `tools/lint` to match Travis scripts.
This keeps `test-all` visibly aligned with `travis/frontend` and
`travis/backend`, which helps us keep it substantively aligned.
2017-08-29 15:42:01 -07:00
Tim Abbott b087886d1c travis: Move i18n and capitalization linters to frontend.
Since these usually result from changes to HTML templates and other
frontend-side things, it seems better to group them with the frontend.

[Tweaked by gnprice in whitespace and comments.]
2017-08-29 15:42:01 -07:00
Tim Abbott 6ef55145f5 travis: Merge the static-analysis build into backend.
[Tweaked by gnprice to leave the use of --linecoverage-report
unmodified for now.]
2017-08-29 15:42:01 -07:00
Tim Abbott 6de77db93f lint: Fix problematic merging of pyflakes stdout/stderr.
We were getting pyflakes lint error output without line numbers like
this:

pyflakes  |     if user_profile.is_realm_admin and
pyflakes  |                                       ^
pyflakes  |

Apparently the cause was that stdout and stderr was getting mixed
badly, creating "unused import"s lines that had the first of that
error (containing the line number) just above.

As a result, printing out the lines of output from pyflakes' merged
stdout/stderr feed looked like this:

b"zproject/settings.py:95: 'from .prod_settings import *' used; unable to detect undefined nameszerver/views/users.py:49:39: invalid syntax\n"

Note the lack of newline in between the end of the first error at
"names" and the start of the second at "zerver".

This appears to be a change in Pyflakes behavior when we switched to
Python 3; probably they're missing a flush() somewhere.
2017-08-29 10:44:50 -07:00
Tim Abbott 130b5847ad pyflakes: Remove now-unnecessary exclude lines. 2017-08-29 10:27:29 -07:00
Tim Abbott 126171cb76 api: Remove root_domain_uri from API page links.
These pages should be references the current server's UI anyway.
2017-08-28 14:13:37 -07:00
Tim Abbott a0a1fe1512 settings: Rename SERVER_URI to ROOT_DOMAIN_URI.
This should be a lot less confusing.

See #6013 for discussion.
2017-08-28 14:09:28 -07:00
Brock Whittaker 04888101e2 Replace old portico footer with new sectioned one.
This replaces the old footer that has one section with a small list of
items. This expands the footer to have multiple sections.

Actual content tweaked and tagged for i18n by tabbott.
2017-08-28 11:07:02 -07:00
Tim Abbott b49448de7d message_edit: Eliminat whitespace in recipient row controls.
This is hacky, but I can't figure out another way to do it that
doesn't cause other problems.

Ideally, we'd add some sort of exclude rule to our HTML template
linter so we can check the rest of the file.
2017-08-27 22:18:29 -07:00
Harshit Bansal e5361b4403 clean_emoji_cache: Expand `clean-emoji-cache`.
Expands `clean-emoji-cache` so that it can be used in production
environment as well. Also moves it to `scripts/` from `tools/`.
2017-08-27 17:51:24 -07:00
Harshit Bansal facb5dbe85 zulip_tools.py: Extract `generate_sha1sum_emoji()`.
Given the path of a zulip installation, it returns a hash corresponding
to the emoji infrastructure of that installation.
2017-08-27 17:51:24 -07:00
Harshit Bansal 948cf54ee3 clean-npm-cache: Expand `clean-npm-cache`.
Expands `clean-npm-cache` so that it can be used in production
environment as well. Also moves it to `scripts/` from `tools/`.
2017-08-27 17:51:24 -07:00
Tim Abbott 90012f8109 find_add_class: Don't check frontend tests. 2017-08-27 14:04:46 -07:00
Tim Abbott ce1a8bf5b2 find_add_class: Include full path names in error messages. 2017-08-27 14:04:46 -07:00
Tim Abbott b1d98c1d0f lint: Only run find_add_class when JS files change. 2017-08-27 13:51:06 -07:00
Tim Abbott 79ae262463 lint: Only run check-urls after python files change. 2017-08-27 13:51:06 -07:00
Robert Hönig 3f01907d3b travis: Distribute linting over frontend and backend suites. 2017-08-27 13:51:06 -07:00
Robert Hönig c5da0ada31 linter: Replace --pep8 with new --frontend/--backend options.
Adds the --frontend and --backend options to replace
--pep8.

Significantly modified by tabbott to use a cleaner framework.
2017-08-27 13:50:53 -07:00
Robert Hönig 2186d53cf5 linter: Migrate lint from optparse to argparse. 2017-08-27 13:18:55 -07:00
Tim Abbott 1560646144 renumber-migrations: Fix construction of new number. 2017-08-27 12:01:06 -07:00
Tim Abbott 96dcb8f745 renumber-migrations: Fix unnecessary prompt.
The tool already knows the answer anyway.

Fixes #6157.
2017-08-27 12:01:06 -07:00
rht 437b2a3146 Update "MacOS" text to "macOS" 2017-08-26 09:00:42 -07:00
neiljp (Neil Pilgrim) 92985f888b mypy: Adjust check of json from github in update-authors-json. 2017-08-25 12:04:15 -07:00
neiljp (Neil Pilgrim) a059872856 mypy: Clarify xml.find use in build_emoji:code_point_to_file_name_map. 2017-08-25 12:04:15 -07:00
Tim Abbott 4b709de9c9 python3: Remove six.PY2/six.PY3 checks. 2017-08-25 11:06:43 -07:00
Tim Abbott b8e7369dee mypy: Remove type: ignores not needed in Python 3. 2017-08-25 11:04:20 -07:00
Tim Abbott b4d698f0db js-dep-visualizer: Report the PNG file as well. 2017-08-25 10:43:37 -07:00
Tim Abbott 3ceba5ef1e test-backend: Require coverage on most webhook integrations. 2017-08-24 11:33:14 -07:00
Yago González 7d5d8ac61e lint: Add rule for spacing inside Handlebars translation tags.
This commit also includes the fixes to make all Handlebars templates
compliant with the new rule.
2017-08-23 13:08:57 -07:00
Yago González 3a4c6f83fc lint: Ban trailing spaces in Handlebars translation tags. 2017-08-23 13:08:57 -07:00
Yago González aa5f57fda9 lint: Ban leading spaces in Handlebars translation tags. 2017-08-23 13:08:57 -07:00
rht e455284bb4 requirements: Remove pathlib2.
`pathlib2` is a backport of pathlib to Python 2.x. Ni!
This dependency can be removed since:
- Zulip 1.6.0+git has been on Python 3 nowadays.
  Ekki-ekki-ekki-ptang-zoom-boing.
- As stated in ticket #6211, having this removed, prevents the need to
  have lockfiles for each of 3.4, 3.5, (and maybe 3.6).

This fixes #6211.
2017-08-23 10:00:21 -07:00
Harshit Bansal 3534e328c5 clean-venv-cache: Expand `clean-venv-cache` tool.
Expands `clean-venv-cache` tool so that it can be used in prod.
Also moves it from `tools/` to `scripts/`.
2017-08-23 00:03:25 -07:00
Tim Abbott 50ad679342 test-backend: Recommend snakeviz over runsnake.
It has support for Python 3.
2017-08-22 15:44:30 -07:00
Tim Abbott 8dff14430e tools: Remove old python 3 compatibility checker.
Now that we've migrated over to Python 3, this tool is no longer required.

Fixes #6217.
2017-08-22 14:58:59 -07:00
Rishi Gupta b32fe9f2a0 features: Update screenshot and add tool to generate.
This adds a new management command that should make generating or
updating screenshots a bit easier in the future.
2017-08-22 11:32:22 -07:00
rht 0a469fd4c8 requirements: Lock prod.txt instead of its py3_common dependency.
`py3_common.txt` is no longer locked.
2017-08-22 10:10:57 -07:00
rht 80f4ba7a76 requirements: Replace py3_dev.txt with dev_lock.txt. 2017-08-22 10:10:57 -07:00
rht 2be8a793e4 Lock the recursive requirements in dev.txt via pip-compile.
First, all the lines of py3_dev.txt except for mypy.txt are mv-ed to
dev.txt. Then dev_lock.txt is generated from dev.txt to be used by
py3_dev.txt. `click` is removed from moto.txt since it is already a
dependency of several libraries and will appear as an autogenerated
dependency.
2017-08-21 18:24:55 -07:00
rht 6baef37564 requirements: Remove Python 2-specific files. 2017-08-21 18:24:55 -07:00
Steve Howell c26e000d4e Fix tools/review for python3. 2017-08-17 17:40:24 -04:00
Umair Khan cec45b0ae5 tools: Add inline string replacer for files. 2017-08-17 09:03:35 -07:00
Greg Price b8dcac1b42 run-mypy: Stop running `mypy --py2`.
Also unconditionally use the `mypy` from our virtualenv --
that's how we ensure we use a common version across different
Zulip developers and in CI.

And as a side effect of cutting some Python 2 vs. Python 3 logic,
fix a bug where `--all` was having no effect.
2017-08-16 17:54:43 -07:00
Greg Price 6346861ccd py3: Cut out the `build-release-tarball --py3` flag.
This is now redundant, as we've applied this transformation to the
source tree itself.
2017-08-16 17:54:43 -07:00
Greg Price 33430f83eb py3: Rely on the shebang to invoke provision too.
We had been forcing provision to Python 3 in dev.  Now that everything
is Python 3 and the `tools/lib/provision.py` shebang reflects that, we
can just invoke the script directly like everything else.
2017-08-16 17:54:43 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Greg Price b7ec1b55dd py3: Cut Python 2 codepaths from provision. 2017-08-16 17:54:43 -07:00
Greg Price d968179e54 py3: Stop running on Python 2 in Travis.
This clears the way to dropping logic in lots of places
for supporting Python 2.
2017-08-16 17:54:43 -07:00
Tim Abbott debfe708d0 test-all: Fix buggy PYTHONWARNINGS invocation.
Shell only sets variables if they're the first thing on a line.
2017-08-16 11:56:24 -07:00
Tim Abbott 37eb6a9863 run-dev: Remove now-unnecessary checks for Google/GitHub auth. 2017-08-16 10:05:19 -07:00
Anirudh Jain 28944b6c94 provision: Partially add zesty to supported systems.
We can't fully support it until we fix the tsearch_extras availability
issue, but for now, this is an improvement.

Tweaked by tabbott to cover the outstanding tsearch_extras issue.
2017-08-15 21:51:19 -07:00
Tim Abbott 9b109b4b73 build-release-tarball: Cleanup Git version sanity check.
Checking for whether tests appear is going to be a more stable
long-term check than looking for zilencer, which we might eventually
include in releases.
2017-08-15 20:44:41 -07:00
neiljp (Neil Pilgrim) 7aacda1883 mypy: Mark internet_guard on requests.request as 'type: ignore'
[gnprice: tweaked comment]
2017-08-15 17:50:18 -07:00
neiljp (Neil Pilgrim) 1a3d8ced78 mypy: Allow Optional parameters in html_branches.py. 2017-08-15 17:50:18 -07:00
Greg Price 73cb361896 install-aws-server: Cut out use of /root/zulip .
Our previous dependencies on the `/root/zulip` path should all be
long gone at this point.  Use a fresh temporary directory instead;
that's cleaner.
2017-08-15 17:41:07 -07:00
Greg Price 835c2ca1ce travis: Cut /root/zulip out of prod test path, just to be sure.
Our previous dependencies on the `/root/zulip` path should all be
long gone at this point.  Run our production-install test suite through
a fresh temporary path instead, mainly just to avoid causing any confusion
over whether that's quite the case.
2017-08-15 17:41:07 -07:00
Greg Price ceccd0c0f6 lint: Accept mentions of specific Python versions.
We're about to do this (a) in a number of places mentioning
system packages like `python3-dev` to install, and (b) in the
shebangs of every script in the tree.
2017-08-15 17:30:31 -07:00
Greg Price 61666a9262 zulip_ops: Delete the long-disused `stats1.zulip.net` config and its dependencies.
This consists of the `zulip_ops::stats` Puppet class, which has apparently
not been used since 2014, and a number of files that I believe were
only used for that.  Also a couple of tiny loose ends in other files.
2017-08-15 17:30:31 -07:00
Greg Price e469578a55 py3: Fix up (almost) all script invocations to rely on shebangs.
This follows up on 207cf6302 from last year to clean up cases that
have apparently popped up since then.  Invoking the scripts directly
makes a cleaner command line in any case, and moreover is essential
to how we control running a Zulip install as either Python 2 or 3
(soon, how we always ensure it runs as Python 3.)

One exception: we're currently forcing `provision` in dev to run
Python 3, while still running both Python 2 and Python 3 jobs in CI.
We use a non-shebang invocation to do the forcing of Python 3.
2017-08-15 17:30:31 -07:00
Brock Whittaker 83184d5c60 /about/: Merge with /authors, redesign page.
This adds the authors to the Zulip repository on GitHub from
/authors/ along with re-styling the page to fit the same
aesthetic as /for/open-source/ and other product-pages.
2017-08-15 16:09:01 -07:00
Steve Howell f0de6bd461 node tests: Add "Starting node tests..." message. 2017-08-15 14:50:10 -04:00
Tim Abbott 84da22da67 travis: Improve production test for HTTP success to not check length.
This will make life a lot easier when iterating on the login page.
2017-08-15 10:51:29 -07:00
Tim Abbott ffc1ceeaa4 lint: Ban use of get_stream in the rest of messages.py.
This will help avoid potential future security bugs.
2017-08-15 10:40:02 -07:00
Robert Hönig c5f0478374 linter: Add trailing whitespace check for json files. 2017-08-15 09:28:11 -07:00
Vaida Plankyte 902d2715a5 run-dev.py: Fix typo in GitHub auth warning message. 2017-08-14 14:35:15 -07:00
Greg Price 6e9f8b90dc provision: Remove the last reference to `install-mypy`, and delete it.
This script has been redundant for some time now.
2017-08-10 18:00:19 -07:00
Jack Zhang ffe14e339d Extract /integrations code from landing-page into separate bundle. 2017-08-10 10:21:53 -07:00
Jack Zhang b85d3e28ce portico: Update display text for integrations categories dropdown.
Category 'All' -> text 'Filter by category'; icon chevron right when
the dropdown is closed, icon chevron down when the dropdown is open

All other categories -> text CATEGORIES[state.category]; icon chevron down
2017-08-10 10:21:53 -07:00
Jack Zhang c546016964 portico: Add translations bundle to portico pages. 2017-08-10 10:21:53 -07:00
Greg Price a32a3cfe23 provision: Fix some shell hygiene.
An array expansion should nearly always be double-quoted, like `"$@"`.
See http://mywiki.wooledge.org/Quotes for some discussion.
2017-08-09 14:03:07 -07:00
Greg Price d3a69da829 py3: Fix or remove (almost) all references to Python 2 venv in docs.
I think soon we'll put the Python 3 venv at `/srv/zulip-venv` and
make `/srv/zulip-py3-venv` just an alias to that (then remove the
alias too), but for now the old name is helpful for spotting places
that need an update.

The `/srv/zulip-venv` name still appears in codepaths used by Travis
tests.
2017-08-08 15:48:12 -07:00
rht b0e4233d3f Dev environment setup: default to Python 3.
From the line in tools/provision it should trickle to the rest of the
scripts. This works since almost all the python scripts have been linted
to be generic.

Proof that the setup is python3 only: with this commit, within the
vagrant container env, /srv/zulip-venv is no longer present and
`./tools/run-dev.py` runs just fine.

[gnprice: Added `rm -f` and warning message, and made small edits.]
2017-08-08 15:28:11 -07:00
Tim Abbott 9bbcb712ec travis: Fix content-length test. 2017-08-08 13:22:22 -07:00
neiljp (Neil Pilgrim) b782db48e1 mypy: Remove superfluous older 'type: ignore' annotations. 2017-08-08 11:27:51 -07:00
neiljp (Neil Pilgrim) 982c0481d0 mypy: Mark log_file as Optional[str] in test_server.py. 2017-08-07 21:27:50 -07:00
neiljp (Neil Pilgrim) b7103752dc mypy: Add assertions relating to Popen.stdout with subprocess.PIPE use. 2017-08-07 21:27:50 -07:00
Greg Price b6d90808a5 prod: Make `cp` copy a directory to its target, not into it.
This was causing `language_options.json` and the `django.mo` files
never to get updated by `upgrade-zulip-from-git` after first deploy.
The stale `django.mo` files meant that any message string which wasn't
in the original from-tarball installation of the server would never
get translated.

The same bug exists in several other spots in this file.  Some of
our repetitive-yet-subtly-different uses of `cp` should probably
turn into just a list of arguments to one `rsync -R` command.
Also this should probably really use `rsync --delete` -- for example,
in prod on zulipchat.com we still have lying around a `locale/zh_CN`
directory, even though in a freshly-built static tree there's only
`locale/zh_Hans` instead.
2017-08-07 19:23:27 -07:00
Pweaver (Paul Weaver) a20e1298ab webpack: jQuery and underscore should be compilied normally to make errors readable.
We will use the normal compliation process so we can understand errors in development and production.
2017-08-05 19:53:56 -07:00
neiljp (Neil Pilgrim) 6ac6d6ff5d mypy: Add note re mypy supporting only global type aliases.
https://github.com/python/mypy/issues/3145
2017-08-05 17:53:18 -07:00
Tim Abbott 0d5cd2d5b7 tools: Remove obsolete print-all script.
This is an old script that could be used to literally print the Zulip
codebase to a PDF.  It hasn't been run since ~2013, and probably never
will be used again.  Since it's probably not interesting enough to be
its own open source project, we should just delete it.
2017-08-05 17:53:17 -07:00
Tim Abbott 52100e0b3b run-mypy: Remove requirement to be inside a Zulip venv.
It can be nice to just install+run mypy from outside.
2017-08-05 16:49:51 -07:00
Tim Abbott f7cfd76403 run-mypy: Check PROVISION_VERSION.
Since new mypy versions frequently break old versions, this check is a
useful way to help prevent problems where mypy output is wrong.

We could probably tighten this by checking explicitly the expected
mypy version from requirements.txt, but that's work.

Replaces #5026.
2017-08-05 16:18:48 -07:00
Pweaver (Paul Weaver) 1afaa67c7c deps: Change npm to yarn for reliablity, security, and speed. 2017-08-05 12:29:06 -07:00
neiljp (Neil Pilgrim) 353f6fdb88 mypy: Hoist RuleList type alias out of function. 2017-08-05 12:08:59 -07:00
Rishi Gupta 4bf8ac2498 generate_test_data: Replace source with non-Gutenberg text.
It's hard to find literature with the community tone we're going for, that
is consistent with the Zulip code of conduct, etc.

This commit removes the special tooling for Gutenberg plays, and changes the
text to be some mixture of scigen, Communications From Elsewhere,
chat.zulip.org, and various books from the public domain.
2017-08-05 11:04:35 -07:00
Pweaver (Paul Weaver) 21cad9cc24 webpack: Convert webpack config to typescript.
This helps catch bugs in the configuration.
2017-08-04 11:01:37 -07:00
Pweaver (Paul Weaver) d0b8e3f0c8 webpack: Merge webpack configuration into one file.
This fixes the confusing duplication between our various webpack config files.
2017-08-04 11:01:37 -07:00
Pweaver (Paul Weaver) 42a2e5dfa3 webpack: Turn webpack.config.js into a function.
This is a prerequisite for refactoring this.
2017-08-04 11:01:37 -07:00
neiljp (Neil Pilgrim) f96f71e439 mypy: assert Popen.stdout based on API, in tools/lint. 2017-08-03 11:09:50 -07:00
neiljp (Neil Pilgrim) e5c446c9b3 mypy: Set get_next_page_url in check-issue-labels to return Optional. 2017-08-03 11:03:14 -07:00
Rishi Gupta 3e9a85ecb2 onboarding: Add turtle reaction to an initial message. 2017-08-01 22:51:05 -07:00
Darren Fix 69767a43cb Create complex sample messages for dev VM.
Create a generator script to pull lines from a play, enhancing
random lines with emoji, Markdown and other flair.

With numerous contributions from Rein Zustand and Tim Abbott to finish
the project.

Fixes: #1666.
2017-08-01 20:54:39 -07:00
rht 532bd6eaa6 Ensure virtualenv is run throughout the entire Travis dev provision. 2017-08-01 15:29:13 -07:00
Steve Howell 30646f2a3b test-js-with-node: Add "Success:" to success message. 2017-08-01 13:43:35 -04:00
Greg Price 2d534e7397 Remove local_settings.py, as it grows stale with disuse.
This file hasn't reflected the actual configuration of any live
installation for some time, nor been part of any tests or other
mechanism to regularly validate it, so it's naturally fallen
behind as we make changes to the set of settings and typically
don't update this file accordingly.  Just remove it; all the
documentation functions it serves are already served just as
well by prod_settings_template.py and its ample comments.
2017-07-31 21:27:32 -07:00
Greg Price 5784724d00 api: Update lint and analysis code to forget about now-gone files.
These files don't exist in this repository anymore, so there's
no need to mention them in exclude lists for lint and mypy.
2017-07-31 21:24:52 -07:00
Tim Abbott c638ea8d11 docs: Fix spelling of Travis CI. 2017-07-31 09:36:22 -07:00
Greg Price 81bfab373d clean-repo: Revert optimization to .pyc removal.
Fixes #5887.  It seems there's some sort of issue in CPython where it
can get confused into thinking a `.pyc` file that's actually stale is
up to date -- perhaps when they date from the same second, say from
the middle of a rebase.

For now, rather than dig further to fix this properly and be sure of
having really done so, just go back to wiping out all `.pyc` files.
The impact is about 1s; that's noticeable when running a short test
file on a quad-core machine (about 8s), but not so much on a more
typical dev machine or on a larger set of tests.
2017-07-30 17:24:55 -07:00
Jack Zhang 5a8b1e6253 hotspots: Implement hotspots frontend. 2017-07-28 16:34:15 -07:00
Vaida Plankyte 02d8f62bb2 run-dev.py: Add Google auth test message & link to docs. 2017-07-27 17:35:14 -07:00
Tim Abbott 53e4d8562b lint: Add a lint check for bare `type: ignore`s. 2017-07-27 16:31:55 -07:00
Tim Abbott 27e1891706 mypy: Add flag to warn for unused ignores. 2017-07-27 16:12:26 -07:00
Tim Abbott d7766d7b6e lint: Fix colors being randomized on every file with custom checks.
Now, each custom check language gets a consistent color.
2017-07-27 16:03:12 -07:00
Pweaver (Paul Weaver) 1ae02cce97 Frontend debug: script-loader should load sourceURL. 2017-07-27 14:36:51 -07:00
Steve Howell bc0761b22b Extract topic_data.js.
This new module tracks the recent topic names for any given
stream.

The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.

We introduced a minor one-line function called stream_has_topics.
2017-07-27 14:26:22 -07:00
Greg Price 360c27dedd tests: Bring all the top-level test scripts into alignment on what to test.
Notably, this adds our checks on translated message strings to
`tools/test-all`, so that they don't cause surprise failures in
CI after a branch is pushed.  (Alternately they could have gone
in `tools/lint` to accomplish the same goal, but `makemessages`
which they depend on is quite slow -- on my machine it takes 7s,
compared to 10s for all of `tools/lint`.)
2017-07-25 15:31:32 -07:00
Greg Price cd7cd73538 tests: Reorder tests in top-level test scripts to agree with each other.
This will make it easier to get the scripts to contain the same set of
tests (with explicit, conscious exceptions), and to keep it that way.
2017-07-25 15:31:32 -07:00
Vaida Plankyte 388acbbfcb frontend: Remove old required div & css from registration pages.
Removes class="required" divs that used to contain an asterisk
for valid/invalid input, and their associated css.
2017-07-25 13:39:34 -07:00
Robert Hönig 500ec48215 testing: Properly exclude files for linting in Windows.
For files to be excluded in Windows, their paths have to
be normalized, so that the string matching works.
2017-07-25 11:30:58 -07:00
Greg Price 7cd621bc9d webhooks/updown: Fix a JsonableError to provide a real error message. 2017-07-24 16:41:22 -07:00
Tim Abbott e1e5e15797 node_cache: Move npm_args inside the interface. 2017-07-21 17:28:20 -07:00
Tim Abbott 8523c24a17 node_modules: Move success stamp to root of cache dir.
This location makes more sense, since it's the entire directory we're
certifying, not just the `node_modules` subdirectory.
2017-07-21 17:28:20 -07:00
Tim Abbott 4c37f93238 clean-npm-cache: Rename npm_cache to current_cache_dir_base. 2017-07-21 17:28:20 -07:00
Tim Abbott a3fd3fe1eb clean-npm-cache: Rename cached_node_modules to current_. 2017-07-21 17:28:20 -07:00
Tim Abbott 6bbd08eca3 clean-npm-cache: Rename success_stamp to current_success_stamp. 2017-07-21 17:28:20 -07:00
Tim Abbott 9f57532c3d clean-npm-cache: Rename npm_cache_dir to node_modules_cache_dir. 2017-07-21 17:28:20 -07:00
Tim Abbott fe6925b341 clean-npm-cache: Swap the order of the cleaning logic. 2017-07-21 17:28:20 -07:00
Tim Abbott 169fdd6e69 tools: Remove remaining deprecated tools.
It seems unlikely that we're going to get more future value out of the
old stress test than the cost of trying to keep it.
2017-07-21 17:28:20 -07:00
Tim Abbott 35a041c3bd clean-npm-cache: Move up NODE_MODULES_CACHE_PATH for Travis.
This makes the code more readable and also will help simplify the
patch when we switch to yarn.
2017-07-21 17:28:20 -07:00
Tim Abbott 5d05cc7294 scripts: Rename NPM_CACHE_PATH to NODE_MODULES_CACHE_PATH. 2017-07-21 17:28:20 -07:00
Jack Zhang 1c9426fdce registration: Divide form into user and org sections. 2017-07-21 13:09:06 -07:00
Jack Zhang 47b4056948 register.html: Fix indentiation via check-templates. 2017-07-21 13:09:06 -07:00
Jack Zhang 8e2ec1cddc portico: Update registration form placeholders. 2017-07-21 13:09:06 -07:00
Jack Zhang e3ef056991 portico: Update registration form help messages. 2017-07-21 13:09:06 -07:00
Vishnu Ks fdafe69ba1 tools: Use get_user in test-api. 2017-07-20 16:50:23 -07:00
rht af430fe24d tools/linter_lib: Clean unused imports told by F401 pyflakes.
The '  # NOQA' comment is added so that the unused imports check (F401)
can still be run while still having the Mypy type check.
2017-07-20 16:46:07 -07:00
Aditya Bansal 7840ae0fe4 template_parser: Add 'with' to Django/Jinja block elements.
Fixes #5863.
2017-07-19 16:39:54 -04:00
Pweaver (Paul Weaver) 01b0ec69d6 Move the webpack stats files to var/ and deploy root.
static/ serves static files which get copied around per deploy. Since
the webpack stats files need a consistent name and change per deploy,
they can't live in static/.

This fixes a bug that preventing downgrading a Zulip server to an old
version.
2017-07-18 15:02:58 -07:00
Pweaver (Paul Weaver) d3ffc81726 Enable Hot Module Replacement in webpack.
This allow the webbpack dev server to properly reload JavaScript modules
while running in dev without restarting the server. We need to connect
to webpack-dev-server directly because SockJS doesn't support more than
one connection on the same host/port.
2017-07-18 11:02:05 -07:00
Tim Abbott 0970fb1a8f run-dev: Remove old node_modules/.bin/handlebars deletion hack.
This hack saved a lot of time debugging weird issues back in 2016, but
it's no longer needed.

Anyone rebasing a branch from 2016 will need to provision afterwards
regardless, which will fix this issue automatically, and more
importantly, these changes were made obsolete when we moved to the
cached `node_modules` model.
2017-07-17 17:57:38 -07:00
Tim Abbott 29c5913bb8 travis: Update success-http-headers. 2017-07-17 16:41:50 -07:00
Harshit Bansal d7617085e6 build_emoji: Update `get_success_stamp()` function.
Updates `get_success_stamp()` function to use the `emoji-datasource`
package's version while calculating success stamp so that an emoji
cache rebuild gets triggered automatically if the version is changed.
2017-07-16 12:36:19 -07:00
Tim Abbott 680381c9d0 setup_venv: Add virtualenv as a dependency for Debian.
Apparently, this was missing on Xenial, and we just never noticed
because the package was installed by default on Ubuntu.

It doesn't exist yet in Trusty.
2017-07-14 17:22:20 -07:00
Tim Abbott 4906a47c1a provision: Disable tsearch_extras for Debian. 2017-07-14 17:22:20 -07:00
Tim Abbott 9efc017e8c provision: Add postgres versions for Debian stretch. 2017-07-14 17:22:20 -07:00
Yago González 2079d84018 lint: Add mobile stores to the capitalization exceptions. 2017-07-14 16:27:17 -07:00
Tim Abbott 5cdfd61f2b lint: Require translation tags on aria-label tags.
Since these are, by their nature, directly user-facing strings, they
should always be tagged for translation.
2017-07-14 14:48:17 -07:00
Tim Abbott 67b30aa231 clean-repo: Reorder pycache removal to make output cleaner. 2017-07-14 12:34:18 -07:00
Harshit Bansal 92f6613bdd tools: Remove `tools/setup/emoji/download-emoji-data` tool.
This tool was used for downloading sprite sheets from iamcal's
repository. Since now we have moved to using `emoji-datasource`
npm package, this tool is no longer required.
2017-07-13 21:52:01 -07:00
Harshit Bansal ebdd8c9e3d emoji: Switch to using `emoji-datasource` npm package.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
  `emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
  package.

Bumps PROVISION_VERSION.

Fixes: #4730.
2017-07-13 21:52:01 -07:00
Harshit Bansal 420327b558 provision: Install NPM packages in the beginning of provisioning process.
NPM packages should be installed at the beginning of the provisioning
process so that later in the provisioning process if a script requires
any NPM package it can use it. Earlier, we were installing NPM packages
in the last as the installation process can fail due to network issues
but since we now retry in case the installation fails, they can be
installed safely at the beginning of the process as well just like apt
packages.
2017-07-13 21:52:01 -07:00
Steve Howell 7e88fb25b3 Move sent_messages callbacks into transmit_message().
This mostly sets the stage for a subsequent commit to start
using client_message_id as the key into sent_messages.

It has the nice side effect of making it more explicit that
certain things should always happen when transmit_message()
succeeds.

This commit does regress our node test coverage a bit.
2017-07-13 23:42:27 -04:00
Steve Howell f6d670ae3d Extract sent_messages.js.
This is mostly straightforward moving of code out of compose.js.

The code that was moved currently supports sending time
reports for sent messages, but we intend to grow out the new
module to track more state about sent messages.

The following function names in this commit are new, but their
code was basically pulled over verbatim:

    process_success (was process_send_time)
    set_timer_for_restarting_event_loop
    clear
    initialize

All the code in the new module is covered by previous tests that
had been written for compose.js.  This commit only modifies
a few things to keep those tests.

The new module has 100% node coverage, so we updated `enforce_fully_covered`.
2017-07-13 23:42:27 -04:00
Jack Zhang e29893a420 Fix indentation for templates/zerver/integrations/index.html. 2017-07-13 20:37:09 -07:00
Brock Whittaker 1cf7ee966b help: Redesign /help/ pages to be a single page app.
This redesigns the /help/ page sets to be a single page app that uses
history.pushState to work the same as the old app.

The big new feature is that now we have the index in a nicely designed
left sidebar.
2017-07-13 15:43:44 -07:00
Tim Abbott 344af2a7c9 travis: Use a wrapper script for installing moreutils.
This works around a frequent problem we've been having with Travis
CI's apt repository being busted on startup.  This, in turn, caused
`apt-get install moreutils` to fail.

We attempt to fix this by adding our own retry logic.
2017-07-12 13:43:47 -07:00
Aditya Bansal de1dc50204 linter/pep8: Remove code infra to ignore the E261 violations. 2017-07-11 15:17:32 -07:00
Aditya Bansal 84e53f960d pep8: Add compliance with rule E261 to tests/test_upload.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 1b5e3190e3 pep8: Add compliance with rule E261 to tests/test_subs.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 2643a40176 pep8: Add compliance with rule E261 to tests/test_signup.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 59b26e582a pep8: Add compliance with rule E261 to tests/test_realm.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 0970d29ec1 pep8: Add compliance with rule E261 to tests/test_narrow.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ce70ded488 pep8: Add compliance with rule E261 to tests/test_messages.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 8c233198f8 pep8: Add compliance with rule E261 to tests/test_events.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ee97ea1356 pep8: Add compliance with rule E261 to tests/test_bugdown.py. 2017-07-11 15:17:32 -07:00
Abdul Qabiz 7d7f2f78c8 clean-repo: Optimize .pyc removal code.
This optimizes .pyc files removal.

Instead of removing all pyc files, it will only remove pyc files where
corresponding py file is not found.  This saves about 1s in the
startup time of `run-dev.py` (and thus also `test-js-with-casper`).

Fixes #5760.
2017-07-11 14:40:56 -07:00
Aditya Bansal 0c4165a5bd pep8: Add compliance with rule E261 to populate_db.py. 2017-07-11 11:56:50 -07:00
Aditya Bansal 856e005d64 linter_lib/pep8.py: Remove files which don't have E261 conflicts. 2017-07-11 11:55:30 -07:00
Aditya Bansal d46cf59b0d pep8: Add compliance with rule E261 to worker/queue_processors.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal 1e3b57570e pep8: Add compliance with rule E261 to tornado/websocket_client.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal 94b790c28d pep8: Add compliance with rule E261 to tornado/socket.py. 2017-07-11 11:55:02 -07:00
Aditya Bansal b9bffaa446 pep8: Add compliance with rule E261 to prod_settings_template.py. 2017-07-11 11:55:01 -07:00
Aditya Bansal 5f87a0f8f8 pep8: Add compliance with rule E261 zproject/settings.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 7c6ad4a770 pep8: Add compliance with rule E261 zproject/dev_settings.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 44edab1cff pep8: Add compliance with rule E261 tools/run-dev.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 73ad834129 pep8: Add compliance with rule E261 bugdown/__init__.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal f5d5c48831 pep8: Add compliance with rule E261 models.py. 2017-07-11 11:53:33 -07:00
Tim Abbott f75b72ef07 test-migrations: Fix error code bug.
Previously, this was failing unconditionally, because of `set -e`.
2017-07-10 16:20:47 -07:00
Aditya Bansal c493a0f02a test-migrations: Add a check to avoid auto named migrations. 2017-07-10 16:02:52 -07:00
Aditya Bansal 0f2c208a5b tools: Add new tool to pretty-print-html. 2017-07-09 07:01:29 -04:00
neiljp (Neil Pilgrim) 33a6a633eb mypy: Amend tools/check-css functions from Text to str. 2017-07-08 10:49:42 -07:00
Tim Abbott f2379d66a2 i18n: Fix capitalization of Zulip Botserver download code. 2017-07-07 17:02:39 -07:00
Brock Whittaker bd664b0dd5 portico: Fix background color; remove need for bg-image classes. 2017-07-07 15:30:47 -07:00
Jack Zhang c5cdede891 portico-css: Reposition header, portico-page content, and footer. 2017-07-07 15:29:43 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Tim Abbott cd4f35abdf test-backend: Don't run tests in parallel with given test files.
It's usually slower to run in parallel with a single file.
2017-07-07 13:36:03 -07:00
Tim Abbott 919c610abf capitalization: Add APNS to list of valid acronyms. 2017-07-07 10:43:02 -07:00
Tim Abbott 774216713b provision: Document more clearly the is_production_travis block. 2017-07-06 17:39:49 -07:00
sinwar 49db0855d2 provision: Avoid spending ~3s on RabbitMQ in no-op case.
Tweaked by tabbott to reorganize the logic more clearly.

Fixes #5180.
2017-07-06 17:38:39 -07:00
Rishi Gupta c827c7840d lint: Re-indent digest.html and missed_message.html. 2017-07-06 15:46:03 -07:00
James Rowan 0d76da27b4 emails: Add base template for missed message and digest emails.
Also removes the files from lint, so that the new indentation doesn't
obscure the change. The files are reindented in the next commit.
2017-07-06 15:46:03 -07:00
Eeshan Garg 7394734ac3 tools/test-api: Stop inserting api/ in PATH.
Now that we have decided to move our API bindings to a separate
repo and specify them as a pip dependency, we can stop modifying
PATH to include the api/ directory.
2017-07-06 12:24:38 -07:00
rht 8aaf34d06d Lock the recursive requirements in common.txt via pip-compile.
The commit is composed of: (1) distill out top-level dependencies in
common.txt, (2) add -e flag to the vcs-based packages because
pip-compile can't do without, (3) pip-compile/generate the locked files
then remove the -e flags from the lockfile, (4) pin pathlib2 to dev.txt
because it turns out it is a direct requirement of
documentation_crawler, (5) document the structure and add an automation
script (6) remove cryptography==1.9 from requirements/scrapy.txt since
cryptography is automatically added from pyopenssl (7) add sed command
to remove future/futures from the generated lock file in python3 (this
should have been automatically handled by pip-compile, so pending for
the feature from pip-compile)

Tweaked by tabbott to update PROVISION_VERSION and add a missing
`first` dependency.
2017-07-05 23:47:07 -07:00
Tommy Ip 61e24cfbea tools: Remove unused `bright_red_output` context manager.
This context manager is no longer in use since the linter visual
updates.
2017-07-05 23:31:09 -07:00
Tommy Ip 2dc35e6c11 linter: Update custom linters' visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip d188c54ebc linter: Update pyflakes visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip cff2b1ba9e linter: Update pep8 visual output. 2017-07-05 23:31:09 -07:00
Tommy Ip 766abaa1a3 linter: Update external_linter visual output. 2017-07-05 23:31:08 -07:00
Tommy Ip e0d4a007e8 linter: Add `identifier` argument to `custom_check_file` function.
In preparation for the upcoming linter output update, each custom
linter now has a identifier argument. This would be used for separating
each line of lints, eg:

    py        | Missing space around "%" at analytics/views.py line 485:
    py        |         output += '<hr>%s\n'%(string_id,)
    css       | 64c64,65
    css       | <     opacity: 1; }
    css       | ---
    css       | >     opacity: 1;
    css       | > }
    css       | static/styles/lightbox.css seems to be broken:
    swagger   | In static/swagger/zulip.yaml:
    swagger   | Duplicate operationId: registerQueue
2017-07-05 23:23:45 -07:00
Tommy Ip 3b8864bafa linter: Create error printing library.
For performance reasons, we spawn each linter in a separate OS thread.
The downside of this is that all lints would end up in stdout without
much visual separation, resulting in confusing error log. This commit
introduce the `print_err` function, which shows which linter each line
of lint is from.
2017-07-06 13:46:10 +08:00
Steve Howell df4b306c20 coveragerc: Exclude zerver/tornado/ioloop_logging.py.
The file ioloop_logging.py is pretty small, but it seems to have
flaky time-sensitive code that creates noise for coverage deltas
when we do builds.  Since it's small, we just exclude it.

Tweaked by tabbott to add comments.
2017-07-05 15:07:52 -07:00
Cory Lynch 099a7052e4 Enforce 100% test coverage for stream_events.js. 2017-07-05 17:57:52 -04:00
Tim Abbott b2ada7e706 docs: Expand documentation on Internet access in tests. 2017-07-04 13:17:25 -07:00
Aditya Bansal 5b81cc2bcd tests: Block internet access to test suites by overriding requests. 2017-07-04 13:17:22 -07:00
Aditya Bansal 26c345f059 backend: Block network access to test suites.
In this commit we basically start to override the request method of
httplib2.Http() to raise an exception whenever it is called i.e.
a trial is made to access the network from test suits.

Fixes: #1472.
2017-07-04 13:17:20 -07:00
Umair Khan ca14292334 travis: Only generate translatation files once.
This should save us a second or two in Travis builds.
2017-07-03 10:10:43 -04:00
Umair Khan 15866b8a1f check-frontend-i18n: Add no-generate option.
Adds the option to not generate translation files.
2017-07-03 10:10:43 -04:00
Umair Khan f22b2d27f3 check-capitalization: Add no-generate option.
Adds the option to not generate translation files.
2017-07-03 10:10:43 -04:00
Umair Khan f1fded08ab check-frontend-i18n: Generate only English translations. 2017-07-03 10:10:43 -04:00
Umair Khan 50ee14d390 check-capitalization: Generate only English translations. 2017-07-03 10:10:43 -04:00
sinwar 3d0b5b14be provision: Avoid spending ~1.5s on compilemessages in no-op case.
This brings the total no-op provision time down to around 7.3s!

Tweaked by tabbott to clean up the code, close file descriptions, and
avoid issues with a somehow corrupted file.

Fixes #5185.
2017-07-01 11:27:54 -07:00
Harshit Bansal a876ebb52a build_emoji: Close open file objects after reading. 2017-07-01 11:14:24 -07:00
Harshit Bansal 8c92024fae build_emoji: Add `codepoint_to_name` dict to emoji_codes.js.
This dict is a mapping from codepoint to canonical name.
2017-07-01 11:14:24 -07:00
Harshit Bansal fee4065571 build_emoji: Generate `codepoint_to_name.json` file.
Store the `codepoint_to_name` map in a JSON file which can be used
by the rest of codebase(zerver) to convert emoji codepoints to
canonical names.
2017-07-01 11:14:24 -07:00
Umair Khan f60d4849b7 Revert "lint: Don't allow handlerbars within translatable text."
This reverts commit 3c7b9cdc94.
2017-06-30 10:12:31 -04:00
Umair Khan 74d83cc47e lint: Disallow handlebars within {{#tr}}{{/tr}}.
This commit adds a new linter which runs from tools/travis/backend.
It runs over the translations.json file and checks if any of the
translatable string contains handlebars in it.

Fixes #5544
2017-06-30 10:12:31 -04:00
Steve Howell 6cf4c65d92 node tests: Add 100% coverage for channel.js. 2017-06-30 09:48:04 -04:00
Rishi Gupta 85d38bd17b emails: Remove DEFAULT_FROM_EMAIL from prod_settings_template. 2017-06-29 17:54:33 -07:00
Umair Khan 3c7b9cdc94 lint: Don't allow handlerbars within translatable text.
Fixes: #5544
2017-06-29 07:39:20 -04:00
Steve Howell 198c363b89 Fix errors in create-test-api-docs.
Our recent Django upgrade removed some info from the file
used by this tool, but there is still useful info, so I
resurrected the tool.
2017-06-27 08:56:39 -04:00
Yago González 33e4855306 lint: Ban non-unique operationIds in Swagger.
The Swagger specification indicates that all operationId values should be
unique. However, SwaggerParser doesn't complain during validation if that
doesn't happen, so this commit adds our own method to identify these
cases.

Also, the violations of this rule have been fixed.
2017-06-26 22:02:26 -04:00
Yago González 00a7f89ba9 node tests: Add coverage to composebox_typeahead.js. 2017-06-26 19:33:25 -04:00
Yago González 7e3154f7bc lint: Ignore tildes in mustache tag template checker.
Handlebars allows putting tildes (~) by the braces in mustache tags to
strip whitespaces on that side of the tag.

    {{#if foo~}}
        <p>Bar</p>
    {{~/if}}

This way, the linter ignores any potential tilde at the ends of Handlebars
tags so we can use this feature without lint fails.
2017-06-26 19:33:25 -04:00
Aditya Bansal 1cb5c9faa8 node tests: Enforce 100% test coverage upon emoji.js. 2017-06-26 14:24:20 -04:00
Aditya Bansal d1fe75ce3b node tests: Enforce 100% node coverage in activity.js. 2017-06-26 14:24:20 -04:00
Eeshan Garg 4ad951f256 integrations/perforce: Migrate docs to Markdown. 2017-06-26 12:30:04 -04:00
Tommy Ip f521b03474 lint: Check if commit message is not empty before running gitlint. 2017-06-25 11:29:14 -04:00
Joshua Pan a63fd95b6a test-js-with-node: Add check for fully covered files that are not enforced. 2017-06-23 16:05:16 -04:00
Joshua Pan c625a80b90 test-js-with-node: Extract check_line_coverage function. 2017-06-23 16:05:16 -04:00
Joshua Pan 3557c47d76 node tests: Enforce 100% node coverage in user_events.js. 2017-06-23 16:05:16 -04:00
Aditya Bansal eb39e82629 Update enforce_fully_covered list with pm_list.js. 2017-06-23 11:13:47 -04:00
Aditya Bansal 18eeb962fe setup-static-analysis: Add wrapper to retry dependancy installation. 2017-06-23 08:23:45 -04:00
Aditya Bansal a20249b126 tooling: Add tool to clear unused emoji cache.
In this commit, we are adding a new tool which can be used to
clear out the emoji cache. Cache cleanup is a good thing to in
general and as well helps keep travis cache small (meaning faster
builds).

Credits to @HarshitOnGitHub as well for suggesting the use of
emoji symlink.
2017-06-22 19:17:12 -04:00
Harshit Bansal 0b03fd97a2 webpack: Don't include katex bundle in base.html.
Since we use katex only in the main webapp, we don't need it in things
like landing page etc, so it is better to move it to index.html.
2017-06-22 19:06:32 -04:00
Harshit Bansal 99b12ed0ca webpack: Use `common.js` to inject webpack dependencies in dev. 2017-06-22 19:06:32 -04:00
Harshit Bansal c0e4f24a25 webpack: Migrate `common.js` bundle to webpack. 2017-06-22 19:06:32 -04:00
sinwar 0e24930770 provision: Avoid spending ~3s on apt operations in no-op provision.
Fixes #5186
2017-06-22 09:37:40 -04:00
Vaida Plankyte 0ed4e22e54 docs: Fix lint/whitespace errors in GCI docs. 2017-06-22 09:30:28 -04:00
Robert Hönig d45d9c9ab5 bots: Add linter rule to prevent open() calls.
Addresses #5395.
2017-06-21 10:16:59 -04:00
Robert Hönig fd41a481e4 linter: Fix `include_only` rule path comparison.
In order for the `include_only` linter rule to not have
any side effects, we need to explicitly add a trailing '/'
after every directory we want to include.
2017-06-21 10:16:59 -04:00
Steve Howell e49afe3ebc lint: Add code to suggest removing exclusions.
If excluded lines no longer exist in a file, print a message
asking the user to remove the exclusion.
2017-06-21 08:12:58 -04:00
Joshua Pan bd9459f27d node test: Enforce 100% node coverage in typeahead_helper.js. 2017-06-21 00:03:05 -04:00
Steve Howell 572ea370ea Speed up custom checkers in `./tools/lint`.
The main thing here is to make looping over lines be the inner
loop, instead of looping over rules.  This keeps regexes in
cache, and it also avoids some O(N) checks.

This is a significant speedup for me, reducing time from 16s
to 11s.
2017-06-20 22:26:12 -04:00
Harshit Bansal c50b3672c2 webpack: Migrate `signup.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Harshit Bansal 3953734a1c webpack: Migrate `stats.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Umair Khan bccd2caacb testing: Fix search key.
The problem this commit solves is related to how we search for
testcases in the test files. We use a simple 'search_key' in file_data.
This will return a false positive if there is a 'longer_search_key' in
file_data.

While searching for TestCases we should use a longer key to remove the
possibility of collision. Using 'class TestCase(' should be precise
enough.

Fixes #4983
2017-06-20 08:35:07 -04:00
Cynthia Lin 9777496046 repository: Update information with Codecov migration. 2017-06-20 07:17:39 -04:00
Aditya Bansal b19134d4c2 travis: Remove docker-engine from held package list in production.
We are doing this in order to avoid facing troubles when travis
updates its trusty images. Here is a link to the blog post.
https://blog.travis-ci.com/2017-06-19-trusty-updates-2017-Q2
2017-06-20 06:45:40 -04:00
Cory Lynch 005d06eb38 message_store.js: Add tests for message id change.
This gets message_store.js to 100% line coverage.
2017-06-20 06:24:27 -04:00
Aditya Bansal 7980462010 setup-backend: Add wrapper to retry provisioning in case it fails.
In this commit we add a wrapper around the provisioning inside
setup-backend to trigger another try at provisioning if it fails
in hopes of it will run this time fine if it was interuppted by a
network issue the first time.
Fixes: #2034.
2017-06-19 16:14:11 -04:00
Aditya Bansal 924d8266c0 setup-production: Add a wrapper to retry provisioning when it fails.
In this commit we basically add a wrapper around provisioning
inside setup-production to trigger a retry in case of failure.
2017-06-19 16:14:11 -04:00
Aditya Bansal fab4a30cce production-helper: Add wrapper to retry apt-get dist-upgrade.
In this commit we add a conditional inside production helper to just
re run apt-get dist-upgrade in case it fails the first time.
2017-06-19 16:14:11 -04:00
vaibhav 33c0c00cd6 Outgoing webhook System: first Iteration of outgoing webhook UI. 2017-06-19 16:05:54 -04:00
Joshua Pan 23dc32a0f3 node tests: Enforce 100% coverage in markdown.js and presence.js. 2017-06-19 08:00:55 -04:00
Joshua Pan d1408125b2 tools: Make sure missing lines in node test coverage are correct. 2017-06-19 07:59:50 -04:00
Aditya Bansal 7de4c340f6 tooling: Add tool to clean unused NPM cache. 2017-06-19 07:55:23 -04:00
Tommy Ip e9fd41bb66 lint: Check if the first word of a commit message is imperative.
Only check the first word using a black list approach to reduce
force positive as shown in the current implementation.
2017-06-19 07:01:52 -04:00
Cory Lynch 7f8ecb62c9 fenced_code.js: Improve coverage to 100%. 2017-06-19 06:53:25 -04:00
Umair Khan eeabc0f6a3 Django 1.11: Bump Django version to 1.11.
This commit also makes the following changes:

* Bumps the version of Django to 1.11.2.

* Fixes the HTTP response headers. Now CommonMiddleware sets Content-Length
  header for non-streaming response, see
  https://docs.djangoproject.com/en/1.11/ref/middleware/#module-django.middleware.common.
  Due to this, 'Transfer-Encoding: chunked' header is removed, which signifies
  a streaming response.
2017-06-15 14:44:38 -07:00
Aditya Bansal a2b5d6efcb provision: Add wrapper to retry apt-get install if it fails first time. 2017-06-14 21:49:15 -07:00
Cory Lynch 3bc369f1cf tools: Add wrapper tool 'build-docs' for building documentation. 2017-06-14 20:16:02 -07:00
Eeshan Garg bbe7ef6c5e linter_lib: Exclude webhook URLs from long line checks.
Now, URLs starting with `{{ external_api_uri_subdomain }}` are
exempted from the 120 char per line requirement.
2017-06-14 17:42:17 -07:00
Aditya Bansal 914ab1040d integrations/codebase.html: Migrate to Markdown. 2017-06-13 16:32:00 -07:00
Tim Abbott 4d66d74938 check-templates: Remove exclude for some migrated files.
This integration docs have all been moved to markdown.
2017-06-13 16:30:27 -07:00
Aditya Bansal 9a38b51254 check-templates: Fix filter condition in Jinja2 template linting.
In this commit we basically stop ignoring 'base.html' from linting
point of view. Basically the condition was implemented in a bad way
resulting in ignorance of every template ending with `base.html`.
As well since removing it doesn't complain about the base.html
non parsable(We have made progress with our linter) we can safely
remove the filter condition.
2017-06-14 04:48:07 +05:30
Aditya Bansal 9cc3e8cf62 followup_day2.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 39a88019f8 followup_day1.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 9a0edad604 find_team.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 9fc16888ca digest.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal fa28dcd008 invitation_reminder.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 07e7482228 confirm_registration.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal 1b82e90331 invitation.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Aditya Bansal e12bb65c0b confirm_new_email.html: Cleanup to use 4 space indentation. 2017-06-14 04:48:07 +05:30
Pweaver (Paul Weaver) 28874cf26f webpack: Add --minify option to run-dev.py for to test minification. 2017-06-13 12:45:40 -07:00
Mehanig 1e5ce918e3 webpack: Move app_debug.js to webpack bundle. 2017-06-13 12:45:40 -07:00
Mehanig 04626acd5a webpack: Move zxcvbn.js to webpack bundle. 2017-06-13 12:45:39 -07:00
Mehanig ae29d4b8d2 webpack: Move landing-page.js to webpack bundle. 2017-06-13 12:36:55 -07:00
Mehanig 2c6f2fb93a webpack: Move activity js to webpack. 2017-06-13 12:36:51 -07:00
Aditya Bansal 8e33d9e48b travis: Update held packages list in production tests. 2017-06-13 11:12:26 -07:00
Aditya Bansal d2f0e17454 linter: Fix bug resulting in <pre> being indented cyclically. 2017-06-13 08:39:59 -04:00
Rafid Aslam ef2c0f806e update-prod-static: Add code to copy CSS from node_modules 2017-06-11 10:15:48 -07:00
Harshit Bansal e30e244b29 emoji: Moved `unified_reactions.json` file.
Moved `unified_reactions.json` file from `tools/setup/emoji` to
`zerver/management/data` so that it can be used in production
environment.
2017-06-09 16:38:58 -07:00
Cory Lynch 6219128d87 dev: Add /emails/ URL for testing email templates.
Fixes #5270.
2017-06-09 16:21:49 -07:00
Cory Lynch 6653e19e3a Add URL for viewing node coverage data on dev server.
Running test-js-with-node --coverage now provides a URL
to view the output data on the web.

Fixes #5177.
2017-06-09 16:17:30 -07:00
Mehanig 8c0e500d6b webpack: Migrate api and katex js files to Webpack bundler.
This also injects the dev webpack sockets to katex, since that's part
of the main web app bundle.
2017-06-09 12:33:52 -07:00
Joshua Pan a72b1bd98e node tests: Enforce 100% test coverage for util.js. 2017-06-07 23:26:46 -07:00
Joshua Pan fdb244fd3b node tests: Add support for enforcing 100% test coverage.
Fixes #5178.
2017-06-07 22:13:02 -07:00
Aditya Bansal ae7781dd76 Fix bug in pep8 linter.
Basically pep8 linter was not running on a selected ignore list of
files for which an extra pep8 rule e261 was to be ignored.
2017-06-07 16:00:01 -06:00
Aditya Bansal 24b1efc74e Fix pep8 linter bug.
In this commit we fix the bug where pep8 linter does not report
false and exit when we got linting issues.
2017-06-07 14:05:03 -07:00
Harshit Bansal 821ff06629 Add `unified_reactions.json` to be used for reactions migration.
This file contains the mapping from old emoji names to new codepoints
from iamcal's dataset to be used in reaction migration.
2017-06-06 23:52:41 -07:00
Aditya Bansal 8f8149c0a5 Clean invite_user.html to use 4 space and consistent indentation. 2017-06-06 22:04:55 -07:00
Aditya Bansal 8555770646 Clean zephyr.html to use 4 space and consistent indentation. 2017-06-06 22:04:55 -07:00
Aditya Bansal 864ee6aaf9 Clean left_sidebar.html to use 4 space indentation. 2017-06-06 22:04:55 -07:00
Aditya Bansal 9219298774 Clean main.html to use 4 space indentation. 2017-06-06 22:04:55 -07:00
Aditya Bansal f9edf8bc0e Clean accounts_home.html to use 4 space and consistent indentation. 2017-06-06 22:04:32 -07:00
Aditya Bansal f58d102664 Clean confirm_email_change.html to use 4 space indentation. 2017-06-06 22:00:37 -07:00
Aditya Bansal d8bfeb6b98 Clean realm_summary_table.html to use 4 space indentation. 2017-06-06 22:00:36 -07:00
Aditya Bansal f2679448e8 Clean ad_hoc_query.html to use 4 space indentation. 2017-06-06 22:00:36 -07:00
Aditya Bansal 1143aec53b Clean 404.html to use 4 space indentation. 2017-06-06 22:00:36 -07:00
Aditya Bansal cb36ce26de Remove files from ignore list which were migrated to markdown. 2017-06-06 22:00:36 -07:00
Aditya Bansal c579a0515b Clean confirm.html to use 4 space and consistent indentation. 2017-06-06 21:28:08 -07:00
Tim Abbott 2215af4b57 docs: Add a bunch of documentation on Travis CI. 2017-06-06 13:39:51 -07:00
Umair Khan 1d183eabf1 testing: Write the .coverage file.
We need to call save explicitly to make coverage write the
.coverage file.
2017-06-06 08:03:30 -07:00
Umair Khan a926559889 testing: Use concurrency parameter.
Coverage includes a concurrency parameter which can be used
to track files if multiprocessing is used.
2017-06-06 08:03:25 -07:00
Umair Khan b09b60dfff Revert "test-backend: Enable test coverage in multi-process mode."
This reverts commit a22d6d2c2a.
2017-06-06 13:07:16 +05:00
Eeshan Garg fab20ec6fd templates: Add macro for appending branch names to Git webhook URLs. 2017-06-05 11:20:48 -07:00
Steve Howell a606628b16 Extract tools/linter_lib/pep8.py 2017-06-05 09:20:21 -07:00
Steve Howell 9fab41aebb Extract tools/linter_lib/pyflakes.py 2017-06-05 09:20:21 -07:00
Steve Howell 29affda956 Extract tools/linter_lib/exclude.py 2017-06-05 09:20:21 -07:00
Steve Howell 40a2ffad98 Extract tools/linter_lib/custom_check.py.
As part of extracting this, we exempt the library from all custom
checks on itself.  This is expedient, since a lot of our
custom checks are naive about whether things are in strings, and
it is also a pain to configure individual rules.
2017-06-05 09:20:21 -07:00
Steve Howell 8ee50f019b lint: Add "nolint" handling for custom checks. 2017-06-05 09:20:21 -07:00
Aditya Bansal acefc67f2f lint: Start enforcing PEP-E261 by default in our linters.
In this commit we start to check for violations to PEP-E261 in our
codebase by default in our linters. Also we have introduced a
ignore list which has around 20 files still in violation to
PEP-E261 which we intend to clear up soon.
2017-06-04 09:18:23 -07:00
Aditya Bansal a9e6106d54 pep8: Add compliance with rule E261 lint. 2017-06-04 09:18:22 -07:00
Aditya Bansal 1be11cbb56 pep8: Add compliance with rule E261 replace-tarball-shebang. 2017-06-04 15:06:12 +05:30
Aditya Bansal a9e0dbe085 pep8: Add compliance with rule E261 pretty_print.py. 2017-06-04 15:05:51 +05:30
Tim Abbott ba74d74dca lint: Fix incorrectly placed docstring. 2017-06-03 16:02:24 -07:00
Yago González 9602f73d05 lint: Add validation for the Swagger file. 2017-06-03 15:48:04 -07:00
Yago González 99b004942f lint: Explain the purpose of target_langs.
Tweaked by tabbott to be a longer docstring.
2017-06-03 15:48:00 -07:00
Aditya Bansal 23baaae980 travis: Stop deleting npm cache in production tests.
Now that we're not using the copy_modules functionality (basically
because including node_modules in production tarballs was a huge disk
sink), the production Zulip code isn't using `zulip-npm-cache` anyway.
And deleting that cache had a huge impact on the performance of the
development environment provisioning that we do as part of this suite.

Commit message rewritten by tabbott.
2017-06-03 15:29:08 -07:00
Eeshan Garg c065f23090 tools: Remove build-api-tarball. 2017-06-02 17:08:00 -07:00
Tim Abbott cecf7f1740 provision: Don't regenerate dev database unnecessarily.
This reuses the work we did some time ago to avoid regenerating the
test database unnecessarily.

In addition to being a nice convenience for developers (since any
accomulated test data is still available), this also saves about half
the time consumed in a no-op provision.

Fixes #5182.
2017-06-02 13:27:34 -07:00
Tim Abbott a0ef2babab text_fixtures: Rename migration status file to have test in name.
This is preparation for making things clear as we move towards not
always rebuilding the test database in provision.
2017-06-02 13:27:34 -07:00
Eeshan Garg 829491fde7 webhooks/papertrail: Migrate docs to Markdown. 2017-06-01 22:16:12 -07:00
Eeshan Garg 5d868b1612 webhooks/helloworld: Migrate docs to Markdown. 2017-06-01 22:16:12 -07:00
Eeshan Garg ccd079b425 webhooks/solano: Migrate docs to Markdown. 2017-06-01 22:16:12 -07:00
Umair Khan bf2bc8d44b lint: Allow empty string in alt arg. 2017-06-01 22:07:43 -07:00
Rohitt Vashishtha a4b76e78ac tools: Update clean-branches to clean review branches. 2017-06-01 21:55:33 -07:00
Rohitt Vashishtha 443e31e348 tools: Add script to see PRs as authors intended. 2017-06-01 21:55:08 -07:00
Rohitt Vashishtha db9918f3d6 bots: Move contrib_bots to api/bots*.
This will make it convenient to include these bots in Zulip API
releases on pypi.

Fix #5009.
2017-06-01 12:31:54 -07:00
Aditya Bansal feffbb963b pep8: Add compliance with rule E261 run-mypy. 2017-05-31 17:07:15 -07:00
Aditya Bansal 5b0b8fd8bb pep8: Add compliance with rule E261 emoji_setup_utils.py. 2017-05-31 17:07:15 -07:00
Aditya Bansal be66369b72 pep8: Add compliance with rule E261 tools/js-dep-visualizer.py. 2017-05-31 17:07:15 -07:00
Pweaver (Paul Weaver) 2efecd4809 webpack: Fix webpack-dev-server to not inline serve bad sockjs url.
Fixes #5118.
2017-05-31 09:31:29 -07:00
Pweaver (Paul Weaver) eb1a22a1d2 webpack: Fix webpack-dev-server urls for external resources.
This fixes issues some users found using the remote development environment.
2017-05-30 22:38:25 -07:00
Tim Abbott e321891f17 test-all: Don't run SVG optimizer by default. 2017-05-30 11:19:53 -07:00
Cynthia Lin 0600e1eb30 tools: Add optimize-svg tool for checking for unoptimized SVG files. 2017-05-30 11:15:56 -07:00
Tim Abbott 4040cadf7d coverage: Omit some test runner files from coverage.
These files are part of the test runner and can't realisitically have
test coverage due to being imported before coverage in the setup
sequence.
2017-05-29 15:30:49 -07:00
Tim Abbott ce26f0e086 integrations: Remove legacy basecamp integration.
Now that we have the webhook integration, there's no reason to
maintain the pre-webhook version.
2017-05-29 15:10:59 -07:00
Rohitt Vashishtha 2d73e03e37 ui-refactor: Rename modals.js to overlays.js.
Fixed #4702.
2017-05-29 11:24:46 -07:00
Tim Abbott 5ffdce9aab test-js-with-node: Set timezone to UTC for tests. 2017-05-29 08:47:26 -07:00
Steve Howell 5f48b51b60 provision: Add `cd /srv/zulip` to profile.
I didn't bump the provision version on this change, as it
won't break anything if somebody fails to make this change.
2017-05-29 08:40:49 -07:00
Pweaver (Paul Weaver) bbb3aad611 Add typescript support to asset compilation. 2017-05-28 17:28:28 -07:00
Tim Abbott 9b8641d9eb lint: Add iOS to list of valid weird capitalizations. 2017-05-26 20:59:32 -07:00
Umair Khan 7f43ec05f2 lint: Add rule to check alt arg for translation. 2017-05-26 15:28:46 -07:00
Umair Khan ad5780c03e lint: Mark alt argument for translation. 2017-05-26 15:28:46 -07:00
Reid Barton ccb4c5c26f bots: Move zephyr-related files to api/integrations/zephyr/. 2017-05-26 15:07:02 -07:00
rht a1f82e02d6 python: Replace os.system with subprocess.call.
Generally, we avoid os.system, since it shells out and thus can be
a cause of security issues.
2017-05-26 15:03:16 -07:00
derAnfaenger 024101be6b bots: Add mypy annotations for bots framework.
This commit adds mypy annotations for both the main
bots and the bots testing runner. It involves a change
to the BotHandlerApi send_message and update_message
funtions, which is compatible with every bot.

Tweaked by tabbott to use more expressive annotations.
2017-05-26 10:25:06 -07:00
Pweaver (Paul Weaver) cb311e99d8 Change backend tests to use a webpack stats stub file. 2017-05-25 16:15:32 -07:00
Pweaver (Paul Weaver) de98dd127d Remove webpack dependency from tools/minify-js. 2017-05-25 16:15:32 -07:00
Pweaver (Paul Weaver) fb0d2d6a8e webpack: Move assets to a separate JSON file so we can process in python. 2017-05-25 16:15:32 -07:00
Mehanig 2d5097868a webpack: Skip compilation for minified 3rd party libs. 2017-05-25 16:15:32 -07:00
Reid Barton 6d2a82ee3c build-release-tarball: Give helpful message on update-prod-static failure. 2017-05-24 19:17:00 -07:00
Mehanig 3f5d0e69fb Build and handle jsfiles using webpack instead of django-pipeline.
Also renames bundle.js to translations.js.
2017-05-24 18:38:03 -07:00
Mehanig 4f39d4fc22 Split webpack config into 3 files (base, dev, production). 2017-05-24 18:37:58 -07:00
Eeshan Garg e9b3b56105 webhooks/appfollow: Use parametric Markdown macros. 2017-05-24 17:00:19 -07:00
Reid Barton 2e4078cbd2 update-prod-static: Redirect update-authors-json stderr to log file. 2017-05-24 16:44:02 -07:00
Tim Abbott 8aaf9b1426 Revert "Fix emoji cache setup when not using provision.py."
This reverts commit ab2cfadcaa.

This broke production installation.
2017-05-24 10:35:41 -07:00
Mehanig 6038777891 npm: Upgrade webpack to 2.0 version dev and production. 2017-05-23 23:58:25 -07:00
Eklavya Sharma 2314c0fb1b tools/minify-js: Improve type checking.
* Use Optional where needed.
* Add type to variables because otherwise it'll be Any.
2017-05-23 22:02:41 -07:00
Eklavya Sharma b805e0ca10 mypy: Assert non-Noneness.
Convince mypy that an object cannot be None by using an assert.
2017-05-23 21:56:50 -07:00
Eklavya Sharma 7636972a6c mypy: Wrap return value in Optional. 2017-05-23 21:56:50 -07:00
Eklavya Sharma cc1937c8d5 mypy: Use Optional with strings where required. 2017-05-23 21:56:50 -07:00
Eklavya Sharma 481d14df39 mypy: Remove a type: ignore.
https://github.com/python/typeshed/pull/693 was merged.
2017-05-23 21:56:48 -07:00
Steve Howell 5003283b64 Enforce 100% test coverage for lib/avatar.py. 2017-05-23 20:07:07 -07:00
Jason Michalski a22d6d2c2a test-backend: Enable test coverage in multi-process mode.
We enable data_suffix option when creating Coverage instances which
causes the output files to include the hostname, pid, and random id.
Before each run erase is called which clears all existing coverage data
files. And then at the end of the test run use the combine method which
merges the reports.

We collect coverage in the main process which collects data from
imports and also when running in single process mode. In the workers we
collect coverage in run_subsuite. This creates more stats files than
strictly required but I don't see a better place to save the stats when
stopping workers.

Note that this has the side effect of enabling parallel testing in
Travis CI.
2017-05-23 18:25:13 -07:00
Steve Howell ecdc500440 test-backend: Run only one process for --rerun option. 2017-05-23 16:59:22 -07:00
Eklavya Sharma 63cc8183ad tools/run-mypy: Add flags. 2017-05-23 12:02:57 -07:00
Eklavya Sharma bda5517b5c mypy: Upgrade to latest version of mypy. 2017-05-23 10:42:44 -07:00
umkay 09e1136cc8 mypy: Fix casing for List type. 2017-05-23 10:42:44 -07:00
Eeshan Garg ed8c6943f2 webhooks/airbrake: Use Markdown macros for documentation. 2017-05-22 18:19:14 -07:00
Jordan Gedney a9f4d56a95 mypy: Add type to info dictionary. 2017-05-22 11:19:06 -07:00
Carey Metcalfe ab2cfadcaa Fix emoji cache setup when not using provision.py.
Moves creating the emoji folder from the provisioning script to
the build_emoji script.

Fixes the fact that the emoji cache directory wasn't being created
when not using the provision.py script.
2017-05-22 11:08:43 -07:00
Aditya Bansal 521bdccb9d pep8: Add compliance with rule E261 to test-tools. 2017-05-18 03:00:32 +05:30
Aditya Bansal 8b219bea13 pep8: Add compliance with rule E261 to test-api. 2017-05-18 03:00:32 +05:30
Aditya Bansal 306e74c60d pep8: Add compliance with rule E261 to replace-tarball-shebang. 2017-05-18 03:00:32 +05:30
Aditya Bansal d6f97f09aa pep8: Add compliance with rule E261 to renumber-migrations. 2017-05-18 03:00:32 +05:30
Aditya Bansal 171a0ddd4e pep8: Add compliance with rule E261 to minify-js. 2017-05-18 03:00:32 +05:30
Aditya Bansal cd7fda993a pep8: Add compliance with rule E261 to lint. 2017-05-18 03:00:32 +05:30
Aditya Bansal 4d06c567f4 pep8: Add compliance with rule E261 to html-grep. 2017-05-18 03:00:32 +05:30
Aditya Bansal 151712b9cc pep8: Add compliance with rule E261 to get-handlebar-vars. 2017-05-18 03:00:32 +05:30
Aditya Bansal bc987c99ca pep8: Add compliance with rule E261 to create-test-api-docs. 2017-05-18 03:00:32 +05:30
Aditya Bansal 9c29da17fc pep8: Add compliance with rule E261 to compile-handlebars-templates. 2017-05-18 03:00:32 +05:30
Harshit Bansal a549013c50 emoji_pickers: Fix broken flag emojis.
Due to differences between the codepoints of flag emojis in
`emoji_map.json` and iamcal's dataset, we need to patch the
css classes for the flag emojis temporarily until the migration
to iamcal's dataset is complete inorder to render them properly.
There is a difference between the images of flag emojis in our
old emoji farm and iamcal's spritesheets and since we have not
yet switched to using spritesheets for displaying emojis in
messages, there is a difference between the flag emojis as
rendered in messages and in emoji pickers.
2017-05-17 08:09:50 -07:00
Tim Abbott b64a4f9291 minify-js: Fix failure to update JS files on Git deploys.
This fixes a significant static asset pipeline bug, which mean that
when we added `moment.js` to the Zulip npm dependencies, it wasn't
properly included in common.js; caching prevented common.js from ever
being rebuilt.
2017-05-16 17:57:20 -07:00
Harshit Bansal 0d28d5d808 build_emoji: Fix the spritesheet CSS generation code.
Modify the spritesheet generation code to account for the differences
between emoji_map.json and iamcal's dataset. Due to the differences
between the two mappings, some emojis like 1️⃣, 2️⃣ etc were not
getting rendered properly in the two emoji pickers where we are using
the iamcal's spritesheets for rendering them. This was so because there
was no CSS class corresponding to the codepoints of these emojis(as
mapped using emoji_map.json) in our spritesheet CSS(generated using
iamcal's dataset).

Fixes: #4775.
2017-05-16 13:06:12 -07:00
Harshit Bansal 0667297c6e build_emoji: Fix the coloring of the white emojis.
Fix the coloring of the white emojis to match that of the images
in the iamcal's spritesheets.
2017-05-16 13:06:12 -07:00
Umair Khan 286f9a40e7 push_notifications: Bring file to 100% coverage. 2017-05-16 12:26:55 -07:00
Eeshan Garg 21e0db03e1 webhooks/appfollow: Migrate docs to Markdown. 2017-05-15 20:14:23 -07:00
Eeshan Garg 0b5711aa7e webhooks/airbrake: Migrate docs to Markdown. 2017-05-15 20:14:23 -07:00
Aditya Bansal 1769a444de Remove unnecessary ignore files.
In this commit we remove user_sidebar_actions.handlebars from
IGNORE_FILES as well remove the check for files to be in a
IGNORE list thus reaching 100% 4 space indent checking for
handlebar templates.

Fixes: #1661.
2017-05-15 11:51:21 -07:00
Aditya Bansal debcf507bd Clean organization-settings-admin to use 4 space indents. 2017-05-15 11:51:21 -07:00
Aditya Bansal 8530997542 Clean default-streams-list-admin to use 4 space indents. 2017-05-15 11:51:21 -07:00
Aditya Bansal 33f3d773b0 Clean user-list-admin to use 4 space indents. 2017-05-12 12:23:01 -07:00
Aditya Bansal 0c39a45d72 Clean bot-list-admin to use 4 space indents. 2017-05-12 12:23:01 -07:00
Tim Abbott 232592cc9f test_sessions: Extends tests to reach 100% coverage of sessions.py.
Fixes #3980.
2017-05-10 10:12:21 -07:00
Tim Abbott 9f7236eec1 message: Remove unused old gravatar_hash field from message dicts.
This was deprecated and replaced some 4 years ago.
2017-05-09 22:33:27 -07:00
Steve Howell 86a9283471 Update JS dependency configuration. 2017-05-08 22:04:56 -07:00
Tim Abbott abd09cebb2 git tools: Make error messages more clear. 2017-05-08 10:51:59 -07:00
Aditya Bansal f4a7791d95 pep8: Add compliance with rule E261 to test_template_parser.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 9fb6d976ca pep8: Add compliance with rule E261 to test_server.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal 88811e0d34 pep8: Add compliance with rule E261 to test_user_agent_parsing.py. 2017-05-07 23:21:50 -07:00