Commit Graph

3130 Commits

Author SHA1 Message Date
Steve Howell b9c9dd9b5a lint: Turn off the CSS checker (officially).
Our CSS checker globs for .css files.  Since the
SCSS cutover, it has been a no-op, so there's no
sense launching it.  See #8894 for details on
future plans.
2018-07-20 10:03:28 -04:00
Anders Kaseorg 8698198452 tools/update-authors-json: Avoid shelling out for cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 6a8f0165ee tools/test-help-documentation: Avoid shelling out for mkdir.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg f01942e510 tools/test-emoji-name-scripts: Avoid shelling out for cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 07f5b81406 tools/minify-js: Avoid shelling out for mkdir, cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 2d12b5b3d9 tools/diagnose: Avoid shelling out for pwd.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Tim Abbott 811ac718d6 build_emoji: Document a quirk of generate_sprite_css_files. 2018-07-13 21:19:26 +05:30
Harshit Bansal ec1ce08523 emoji_setup_utils: Migrate to python3 type annotations. 2018-07-13 21:19:26 +05:30
Harshit Bansal f6fe5d6266 build_emoji: Migrate to python3 type annotations. 2018-07-13 21:19:25 +05:30
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Tim Abbott 7a86e83a6f build_emoji: Use with for writing to files.
This is a nice code cleanup.
2018-07-13 21:07:08 +05:30
Tim Abbott 85243c525d build_emoji: Pass around emoji_map data structure less.
It wasn't used in either of these functions.
2018-07-13 21:04:37 +05:30
Harshit Bansal b9f974ba61 tools: Add a tool to validate emoji name listing scripts.
This tools emoji name import/export tooling by verifying that no
information gets modified in a round-trip conversion.
2018-07-13 20:53:25 +05:30
Harshit Bansal 628e868d1e tools: Add `export_emoji_names_to_csv`.
This tool is used for exporting `emoji_names.py` to a csv file.
2018-07-13 20:53:08 +05:30
Harshit Bansal 05f85eb94d emoji: Add emoji_names.py file.
Credits to @rishig, Alice Lai, and @rntharu for naming all the emoji.

Names are inspired by iamcal, gemoji, and unicode names, sources like
emojipedia and iemoji, google search results for articles about emoji,
and emoji usage on twitter.
2018-07-13 20:53:07 +05:30
Harshit Bansal ae17a15b29 tools: Add generate_emoji_names_table tool.
This tool generates a listing of the emojis as per the current cache
in use.
2018-07-13 20:53:07 +05:30
Harshit Bansal fef5b43e22 minor: Fix directory name label in 'import_emoji_names_from_csv'. 2018-07-13 20:53:02 +05:30
Max Nussenbaum 8fd900626d billing: Style billing page.
This adds styles to /billing.
2018-07-13 18:30:38 +05:30
Tim Abbott 3cc93fd721 setup_venv: Fix missing libssl-dev dependency.
We were already correctly including libssl-dev in Zulip's dependencies
in development environment provisioning, but (at least now) it's
needed to build certain Python packages like pycurl when building a
Zulip virtualenv in production.  I haven't investigated why we didn't
need this on Ubuntu, but one possible reason would be that some other
library in our dependencies list happens to depend on it on Ubuntu.

We fix this by moving the dependency over to the shared
VENV_DEPENDENCIES list.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Harshit Bansal 8ada7cfe5b tools: Add import_emoji_names_from_csv tool. 2018-07-12 18:05:02 +05:30
Greg Price 36cf898589 mypy: Add a place to put type stubs.
This will allow us to begin to add our own stubs for external
libraries.  Writing stubs can be surprisingly little work to do, and
can have high leverage in keeping our type annotations high-quality.
2018-07-12 14:10:09 +05:30
Yago González a3d42d9901 test-api: Pass non-admin client to the test helpers. 2018-07-11 23:54:00 +05:30
Cynthia Lin 7d9281fa6c simplebar: Add simplebar dependency for help page scrolling. 2018-07-11 20:04:55 +05:30
Puneeth Chaganti 3dace27ed6 tools: Only files starting with same migration number conflict. 2018-07-10 21:09:34 +05:30
Shayan Toqraee 0757d022f5 messages: Add support for right-to-left messages.
This implements right-to-left message automatic detection support in
the compose box as well as the message feed.  Full unit tests and
support in the message-editing UI are for future work (as are
potentially more fancy things like supporting things like
right-to-left multi-word names for users/streams/etc.).

Fixes #3123.
2018-07-10 10:47:56 +05:30
Rishi Gupta 16334a1ba7 billing: Update billing system. 2018-07-09 14:33:07 +05:30
Aditya Bansal b6ba162769 generate-fixtures: Extract zulip_test_template creation as a function.
This will be helpful in the upcoming changes which will make use
of this extracted function to re-create zulip_test_template after
migrating zulip_test db so that we have latest schema in tests.
2018-07-09 09:11:48 +05:30
Armaan Ahluwalia ba6543b850 webpack: Fix missing exposed modules left out of transition.
This commit fixes some modules that were erroneously left out while
transitioning app.js to webpack. This commit exposes them using
expose-loader or setting them directly to window.
2018-07-07 08:00:34 +02:00
Shubham Dhama ced6f967bc voting_widget: Increase node test coverage to 100%. 2018-07-06 11:30:12 -04:00
Tim Abbott 0bd139af0c webpack: Fix missing expose for toMarkdown. 2018-07-05 13:24:19 +02:00
Armaan Ahluwalia 54d3d8e8b3 webpack: Transition app.js to be compiled by webpack.
This commit moves all files previously under the 'app' bundle in
the Django pipeline to being compiled by webpack under the 'app'
entry point. In the process, it moves assets under the app entry
to a file called app.js that consumes all relevant css and js files.

This commit also edits the webpack config to be able to expose certain
variables for third party libraries that are currently required by
some modules. This is bad coding form and should be refactored to
requiring whatever dependencies a module may have; we're just
deferring that to the future to simplify the series of transitions we
need to do here. The variable exposure is done using expose-loader in
webpack.

The app/index.html template is edited to override the newly introduced
'commonjs' block in the base template. This is done as a temporary
measure so as not to disrupt other pages on the app during the transition.

It also fixes the value of the 'this' context that was being inferred
as window by third party libraries. This is done using imports-loader
in the webpack config.  This is also messy and probably isn't how we
want things to work long term.
2018-07-05 11:03:08 +02:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Shubham Dhama b938523592 widgets: Increase node tests coverage to 100%. 2018-07-03 08:48:49 -04:00
Yashashvi Dave 2b478268f3 third: Add sortablejs library. 2018-07-01 02:05:00 -07:00
Vishnu Ks e1bf749d3c provision: Run build_pygments_data only if required. 2018-06-23 00:15:39 +05:30
Vishnu Ks 109fa85614 provision: Rename file_hash_updated to file_or_package_hash_updated.
Check for changes in package version as well along
with the files.
2018-06-22 23:40:31 +05:30
Vishnu Ks 431d9d6076 provision: Run generate-custom-icon-webfont only if required. 2018-06-22 23:40:31 +05:30
Shubham Padia 6853530eba search: Add search_pill.js with basic operations.
Adds search_pill.js to the static asset pipeline. The items
for search pill contain 2 keys, display_value and search_string.
Adding all the operator information i.e the operator, operand and
negated fields along with the search_string and description was tried out.
It was dropped because it didn't provide any advantage as one had to
always calculate the search_string and the description from the operator.
2018-06-22 09:05:36 -04:00
Shubham Dhama c167227ddc search: Add 100% coverage for search.js. 2018-06-22 07:58:06 -04:00
Greg Price 0462e85d02 tools: Make remote configurable in {reset,push}-to-pull-request.
The appropriate name for the remote pointing at the repo we maintain
may be `upstream` for most of our repos... but not when we're
downstream of someone else, e.g. for react-native.  So, make it easy
to configure per-repo.
2018-06-21 17:28:21 -07:00
Vishnu Ks 59169cc302 provision: Run inline-email-css only if required.
This saves a few hundred milliseconds on provision time.

Edited by tabbott to include email.css in the dependencies.

Fixes #9468.
2018-06-18 06:58:55 -07:00
Vishnu Ks 7b8e79ae48 provision: Refactor hashing of compilemessages into a function.
This allows it to be reused for other tools.

Edited by tabbott to remove the use of "compilemessages" in variable
names.
2018-06-18 06:55:36 -07:00
Raymond Akornor e82581e5bd provision: Close opened files automatically. 2018-06-12 13:38:58 -04:00
Aditya Bansal 1205e02c64 provisioning: Don't rebuild DB if running migrations is sufficient.
This results in a significant optimization in the performance of
re-provisioning Zulip if all that you're doing is rebasing onto a
newer version of master (which just adds new migrations).

The change carries some risk of generating unpleasant-to-debug
situations, because if we merge a buggy migration and then later fix
it, some clients may not have a properly migrated database (and also,
this changes how populate_db commutes with migrations).  But it seems
worth it, given how much time is currently wasted by not having this.

Fixes: #9512.
2018-06-06 13:43:59 -07:00
Aditya Bansal 65dc80fe9d test_fixtures: Add run_generate_fixtures_if_required function.
In this commit we are adding run_generate_fixtures_if_required,
a new function which is meant to de-duplicate a bit of code
between test-server and test-backend which is essentially
responsible for rebuilding the test database if that was required.
2018-06-06 13:37:35 -07:00
Aditya Bansal f7c11d1747 test_fixtures: Refactor to have template_database_status API.
In this commit we are essentially just refactoring the function
is_template_database_current to be called template_database_status
and adjusting the return values accordingly.
This is essentially a preparatory commit for the upcoming commits
which will essentially enable us to not throw away entire DB and
rebuild from scratch if only running migrations could do the job.
2018-06-06 13:37:35 -07:00
Tim Abbott e9312b921c tests: Disable test-queue-worker-reload test.
This test is low value (it's for testing a very specific feature/bug),
is slow, and apparently flakes in CI occasionally.
2018-06-05 09:24:03 -07:00
Tim Abbott 4cdcb7e0d3 webpack: Use more consistent bundle naming scheme.
This way, both our JS and CSS files using .chunkhash.ext as their
naming scheme.
2018-06-03 16:49:59 -07:00
Tim Abbott 7813376934 webpack: Use filenames based on hashes.
This should avoid us creating duplicate webpack bundles every time we
do a deployment, even if none of the files in the bundles themselves
have changed at all.
2018-06-03 16:49:43 -07:00
Tim Abbott 714ad67db1 storage: Move library code into zerver/lib.
There's no particularly good reason for this to have been a top-level
file.
2018-06-03 16:02:02 -07:00
Shubham Padia e32dd53cd9 lint: Check for occurrences of `.includes` except in `frontend_tests/`.
Adds a custom check to js_rules in `/tools/lint/lib/custom_check.py`.
2018-06-03 14:30:22 -07:00
Greg Price f686d55fa0 push-to-pull-request: Broaden one error message.
I discovered this case just now while testing other changes.
2018-06-01 16:52:18 -07:00
Greg Price 954c71e90c push-to-pull-request: Use `jq -r` to simplify a test slightly.
This option (aka `--raw-output`) prints a string as itself, rather
than JSON-encoded; which makes it fit a bit better in a shell script,
saving us a layer of quoting.
2018-06-01 16:52:18 -07:00
Greg Price 8e81ca0fb2 push-to-pull-request: Fix regex usage in parsing remote.
This replaces ad4617c95 with a different fix for the same issue:
instead of stripping the `.git` off separately, we can just correct
the regex, using `+?` to fix our stepping in a classic regex pitfall.
2018-06-01 16:52:18 -07:00
Tim Abbott 751c602a0b update-prod-static: Don't copy/minify templates and styles.
This is a performance optimization: Rather than copying these files
into the `prod-static` directory and then deleting them, we just don't
copy them over in the first place.

For styles, it might have once been the case that this did something,
but we've moved them all to being managed by webpack some time ago.

For the js directory, I think it was never useful to copy and then
delete them; these files were always compiled via tools/minify-js,
and the raw JS files weren't needed, anyway.
2018-06-01 16:39:35 -07:00
Tim Abbott cefdf6f6d4 update-authors-json: Pretty-print output with sorted keys.
This makes working with this file for debugging/etc. much nicer.
2018-06-01 16:39:04 -07:00
Tim Abbott 60eae53327 push-to-pull-request: Provide better output for invalid PR.
Previously, we were very confusingly showing the error message for
permissions when the issue was a not-found error.
2018-06-01 16:00:27 -07:00
Tim Abbott ad4617c953 push_to_pull_request: Fix handling of .git.
This was throwing the "you can't access the repo" error for this
reason.
2018-06-01 16:00:27 -07:00
Tim Abbott 95f1f1d363 mypy: Remove logic for linecoverage report. 2018-06-01 11:14:32 -07:00
Vishnu Ks 69f8b7e8df circleci: Add a base image for Ubuntu Bionic. 2018-05-31 18:51:32 -07:00
Tim Abbott 5b5d1f3e87 update-prod-static: Rewrite to use the run() scripting tool.
This provides nice `set -x` style logging of which commands are
running when.
2018-05-31 16:45:51 -07:00
Sampriti Panda e95f972e24 migrate: Add do_batch_update method for running batch updates.
* Refactor pgroonga_0002 migration to use new method.
2018-05-31 13:00:34 -07:00
Tim Abbott a2c9517f8d test_server: Don't continuously recompile handlebars templates.
This changes run-dev.py to ensure that we have in fact compiled
handlebars templates before running webpack, which is the right model.

Future work will likely include running the handlebars compiler from
webpack, and thus eliminating this extra process.
2018-05-30 20:16:06 -07:00
Tim Abbott 5a35507a7a success-http-headers: Add X-Frame-Options. 2018-05-30 09:41:50 -07:00
Tim Abbott 0fabff6dda setup_venv: Clean up VENV_DEPENDENCIES using parsed lsb_release.
This is mostly a cleanup, but it should also save 50ms in the runtime
of create-production-venv.
2018-05-29 10:57:36 -07:00
Raymond Akornor 5d39a0f0fc scripts: Replace calls to lsb release with our own parsing.
This improves the performance of these operations, by saving a ~50ms
Python process startup.  While not a major performance improvement, it
seems worth it, given how often these commands get run.

Fixes #9571.
2018-05-29 10:57:36 -07:00
Robert Hönig afa806a0ce linter: Add rule to favor Botserver over botserver or bot server. 2018-05-29 07:27:36 -07:00
Tim Abbott 2f571f9a45 webpack: Move font and pygments CSS to common.css.
First, it's silly that these weren't in common.css in the first place,
since that meant these were a bunch of duplicated code, but
additionally, that meant that these weren't available on the
`/activity` page (or other pages that don't include the portico styles).

Fixes #9561.
2018-05-27 23:19:40 -07:00
Yago González b4084d30d2 emails: Move user role logic for followup_day1 to notifications.py. 2018-05-26 07:29:58 -07:00
Yago González 5b0b9feb7d i18n: Atomize translation tags.
Makes the i18n strings in this file much easier to translate by splitting
them into smaller chunks (which avoids having a lot of code in the tagged
strings), and adds a string that was missing as well.
2018-05-26 07:06:43 -07:00
Tim Abbott 20aa3233b0 lint: Improve the check for realm->organization to support identifiers. 2018-05-25 12:03:28 -07:00
Tim Abbott 07b4da7659 Revert "node tests: Change timezone for node tests."
This reverts commit 311ddefd32.

This was causing tests to fail in some but not other environments.
2018-05-25 08:52:13 -07:00
Vishnu Ks abade460a9 tools: Add tool for listing outdated pip packages. 2018-05-24 15:16:02 -07:00
Aditya Bansal f9aa012486 linter: Fix issue with check-templates output.
We fix the issue of check-templates spitting out diff between
expected and found indentation of a file before mentioning the
error message and the file name. Basically stuff was being in the
wrong order despite the fact that in code stuff was happening in the
correct order ie, first print the error message along with the filename
and then the actual diff between expected and found file indentation.

Fixes: #9533.
2018-05-24 12:42:32 -07:00
Vishnu Ks 54a002c2e2 requirements: Upgrade pyflakes to 2.0.0.
We fix a few errors that only the new version finds.
2018-05-24 11:31:36 -07:00
Steve Howell 1b57e568ff Add "zform" functionality to the web client.
A "zform" knows how to render data that follows our
schema for widget messages with form elements like
buttons and choices.

This code won't be triggered until a subsequent
server-side commit takes widget_content from
API callers such as the trivial chat bot and
creates submessages for us.
2018-05-24 09:30:22 -07:00
Steve Howell 435719c65b Add a schema checking concept to the web app.
This starts the concept of a schema checker, similar to
zerver/lib/validator.py on the server.  We can use this
to validate incoming data.  Our server should filter most
of our incoming data, but it's useful to have client-side
checking to defend against things like upgrade
regressions (i.e. what if we change the name of the field
on the server side without updating all client uses).
2018-05-24 09:30:22 -07:00
Steve Howell d018995409 node tests: Make the failure message more obvious.
I mistakenly pushed a PR when my tests failed.  I ran with
the coverage option, so I saw this brightly colored summary
report that distracted me from the failure message.

This adds a couple newlines and some all caps.
2018-05-24 09:30:22 -07:00
Sivakar Sithamparanathan 311ddefd32 node tests: Change timezone for node tests.
The timezone environment variable was set to UTC initially. It was
changed to something other than UTC so that any local vs UTC
conversion issues will manifest in the tests.

Fixes: #5105.
2018-05-23 13:29:19 -07:00
Aditya Bansal 8cfb437677 provision: Don't run create_realm_internal_bots management command.
We essentially stop running create_realm_internal_bots during
every provisioing and move its operations to run from populate db.
In fact to speed things up a bit we actually make populate db call the
funcs which create_realm_internal_bots calls behind the scenes.

Fixes: #9467.
2018-05-23 13:34:52 +05:30
Aditya Bansal a27c9a08dd generate-fixtures: Export DJANGO_SETTINGS_MODULE=zproject.test_settings.
This is required because the --settings=zproject.test_settings param
doesn't work with migrate or the dumpdata management commands. Thus
untill now if one ran just this tool ended up with test database not
properly setup. We never noticed this because test-backend ran this
tool again (after exporting DJANGO_SETTINGS_MODULE) thus making the
tool work this time.
2018-05-23 11:32:40 +05:30
Sampriti Panda 51c71fd3fd provision: Add libfontconfig1 to bionic apt dependencies. 2018-05-22 22:37:04 -07:00
Shubham Dhama ec4e5e73ab org settings: Make time limit input elements inline to dropdown.
(To make the capitalization pass we added `N` to ignored phrases.)
2018-05-22 19:32:44 -07:00
Greg Price 5e89c33e7f tools: Add push-to-pull-request tool.
I've often done this by hand -- basically typed out the last line,
with the variables found from looking at the PR page in a browser.
Seems nicer for both us maintainers and the contributor, in particular
because the PR gets marked as merged, instead of closed.  But it's a
bit of a pain, and I do it maybe half the time or less; plus it's kind
of a subtle GitHub feature, and as a result I think other maintainers
of Zulip repos do this approximately never.

I've always figured this couldn't be hard to automate; today I decided
to take the 45 minutes to look up how, write out the script, QA it,
write up a nice usage message and some comments, and commit it. :)
2018-05-22 18:33:51 -07:00
Armaan Ahluwalia 1525e92058 logging/errors/webpack: Improve error stack traces with blueslip.
This commit improves the output that blueslip produces while
showing error stack traces on the front-end. This is done by
using a library called error-stack-parser to format the stack
traces.

This commit also edits the webpack config to use a different
devtool setting since the previous one did not support sourcemaps
within stack traces. It also removes a plugin that was obviated
by this change.
2018-05-22 14:36:41 -07:00
Eeshan Garg e0ef831993 webhooks: Migrate to UnexpectedWebhookEventType.
This improves test coverage for a lot of our webhooks that relied
on ad-hoc methods to handle unexpected event types.

Note that I have deliberately skipped github_legacy, it isn't
advertised and is officially deprecated.

Also, I have refrained from making further changes to Trello, I
believe further improvements to test coverage should be covered
in separate per-webhook commits/PRs.
2018-05-22 08:30:19 -07:00
Eeshan Garg 3ed20589f2 webhooks: Add generic exception for unexpected webhook events.
UnexpectedWebhookEventType is a generic exception that we may
now raise when we encounter a webhook event that is new or one
that we simply aren't aware of.
2018-05-22 08:30:19 -07:00
Michael J. Sullivan c81b276c1c mypy: Add basic support for invoking dmypy to run-mypy. 2018-05-21 22:41:08 -07:00
Aditya Bansal e41d32a15a reminders: Add various design improvements for the UI/reminder msg. 2018-05-21 09:07:37 -07:00
Tim Abbott c6fda61227 block_internet: Make compatible with httpretty library.
If httpretty is enabled, we can let it handle HTTP requests.
2018-05-20 23:31:42 -07:00
Tim Abbott a170bd70af provision: Don't block installing with Ubuntu bionic.
Now that we have tsearch_extras packages uploaded, this mostly works.

There's a few issues being debugged in #9460; they should be fixed
soon, and regardless, merging this will make it easier to develop.
2018-05-20 21:21:11 -07:00
Tim Abbott 06ed55e45a webpack: Add ALLOWED_HOSTS list for zulipdev.com and friends.
This makes it possible to again use the *.zulipdev.com domains in the
development environment.

Ideally, we'd also read REALM_HOSTS to make this more flexible.
2018-05-20 18:12:28 -07:00
Max Nussenbaum 0ff2051982 portico: Add tour section to homepage.
This adds a tour of Zulip to the bottom of the homepage.

In order to get the carousel nave, we use Bootstrap 2 from a CDN on
this page; this isn't ideal in the medium term, but upgrading
Bootstrap across the project is too much work for now.
2018-05-20 15:04:23 -07:00
Tim Abbott 4f8e09d5af provision: Fix buggy management of apt_dependencies_hash.
Apparently, we were incorrectly appending each new hash onto the end
of the file, basically resulting in every run of provision being
treated as a miss for this cache.

Fixing this saves about 4s (over 1/3) of the no-op provision time.
2018-05-20 14:22:32 -07:00
Tim Abbott 235002a549 provision: Don't install lsb-release if already installed.
This early bootstrapping step should be rearely required, and it saves
about 0.5s in our no-op provision time.
2018-05-20 14:22:32 -07:00
Shubham Padia 0824308a7a tools: Allow optional arguments after file arguments in test_backend.
Fixes #9233.
Uses nargs='*' instead of nargs='argparse.REMAINDER'.
nargs='argparse.REMAINDER' gathers remaining terms as arguments
even if it is an option e.g --coverage, while '*' gathers all the
command-line arguments until the next option is encountered.
2018-05-18 17:08:40 +05:30
Tim Abbott 0dcf873370 third: Rename thirdparty-fonts.css for better clarity.
Now it's clear that this is just legacy fontawesome styles.
2018-05-17 04:58:30 -07:00
Aman Ghebreweldi b05a702285 third: Use font awesome css file from NPM package. 2018-05-17 04:58:30 -07:00
Steve Howell b48f052b0c Add basic widgets framework (JS side). 2018-05-16 15:13:33 -07:00
Steve Howell 1f0ffdc415 Add widget.py to enable widgets on the server side.
The only slash command implemented in this initial
version is an extremely crippled version of a
"/stats" slash command that reports that you are
running 1 server.
2018-05-16 15:13:33 -07:00
Joshua Pan a67ba617d6 minor: Fix typo in tools/lint. 2018-05-16 13:19:28 -04:00
Tim Abbott 00861b910d test_presence: Add tests for seconds_usage_between.
This isn't widely used, but is a nice way to verify the logic further.
2018-05-15 18:37:49 -07:00
Tim Abbott e39a3e07d7 coverage: Mark zerver/lib/debug.py as not needing coverage. 2018-05-15 17:19:14 -07:00
Tim Abbott fd24eac8bb coverage: Exclude zerver/lib/parallel.py from coverage. 2018-05-15 17:19:14 -07:00
Tim Abbott c46149efcb migrate: Remove obsolete act_on_message_ranges.
This has a cool structure, but it's written against the long-dead
South API, and we can always pull it out of the Git history if we want
to use this approach in the future.
2018-05-15 17:19:14 -07:00
Tim Abbott 3e04aa99fa fonts: Use source-sans-pro CSS from upstream. 2018-05-15 16:22:44 -07:00
Greg Price 6dbd90835b mypy: Move remaining exclusion list to config file.
This leaves the wrapper script with very little left to do!

The main thing left is finding scripts by searching for shebang lines;
mypy itself would happily do the search for importable Python files.
2018-05-15 18:14:33 -04:00
Tim Abbott badfec2b49 coverage: Don't include settings files in test coverage. 2018-05-15 13:55:01 -07:00
Tim Abbott 191e6dc13a create_user: Remove user_profile_id option.
This hasn't been used in years, had no test coverage, and doesn't have
a clear use case.
2018-05-15 13:51:56 -07:00
RobbieClarken f81b936727 zerver/tests: Require 100% test coverage of zerver/lib/upload.py.
The last line here is impossible to test.

Fixes #4489.
2018-05-15 10:52:20 -07:00
Aman Ghebreweldi 83ee8211a8 third: Get sorttable package from npm.
We didn't have any local changes, so there's no need to have it
vendored in the repository.
2018-05-15 09:25:48 -07:00
Steve Howell 4f0b3fc1b6 test-js-with-node: Exit more cleanly for errors.
We don't need a Python stack trace when the node
piece fails, so just give a non-distracting error
message.
2018-05-15 08:24:44 -07:00
Joshua Pan df84e1d7eb tests: Reach 100% coverage for zerver/liv/management.py.
Cleaned up add_user_list_args(). The "help" and
"all_users_help" have all default values. As noted in
an earlier commit, "all_users_help" is always passed in,
so we can get rid of "all_users_arg". We keep the default
for "all_users_help" so we don't have to change variable order
in function definition.
2018-05-14 10:46:21 -07:00
Darshan Markandaiah 607cab2a53 tests: Add 100% test coverage to zerver/apps.py.
Tweaked by tabbott to rename to test_cache.py and remove the
sender_name argument (Since it was kinda confusing).
2018-05-14 10:36:10 -07:00
Vishnu Ks b73603a97c droplets: Do shutdown command along with clear history.
Otherwise shutdown command would be still present in the
history.
2018-05-14 08:12:43 -07:00
Vishnu Ks 861001f7b3 droplets: Update snapshot id.
Upgraded zulip and python-zulip-api repos.
Installed flask in python-zulip-api.
2018-05-14 08:12:43 -07:00
Tim Abbott f5ea661c1f tools: Upgrade pycodestyle.
We have to add a few rules that we're ignoring for now.
2018-05-14 06:15:13 -07:00
Aditya Bansal 64678b459c linter: Add rule to lint against use of typing.Text and promote str.
This commit is the last in a series of commits which migrated our
entire codebase to use str instead of typing.Text.

Fixes: #9203.
2018-05-13 17:17:32 -07:00
Aditya Bansal addfde7374 tools: Change use of typing.Text to str. 2018-05-13 17:17:32 -07:00
Joshua Pan 073ecaac66 provision: Give concrete NFS error message on older OSX versions. 2018-05-13 08:47:19 -07:00
Tim Abbott ff2157c787 coverage: Exclude api_test_helpers.py from backend test coverage.
These files are tested by a different test suite, and in practice have
100% coverage; no need to double-measure it.
2018-05-09 20:49:13 -07:00
Tim Abbott 7e2841b358 bulk_create: Mark bulk_create_streams as nocoverage.
It's only used in `populate_db`, so it gets tested that way.
2018-05-09 20:49:12 -07:00
Steve Howell 27b7461e0a Revert "js: Implement DynamicText class."
This reverts commit 6e7305f784.

We never ending up using this class.
2018-05-08 12:13:55 -07:00
Vishnu Ks 49b3cb9da5 droplets: Remove the step to activate virtualenv manually.
I didn't had to activate this manually during the
last few droplet upgrades.
2018-05-07 08:42:18 -07:00
Vishnu Ks c684333051 droplets: Fix the command for clearing bash history.
history -c only clears the history from memory and
would be populated back during the next SSH session.
2018-05-07 08:42:18 -07:00
Vishnu Ks 18314e57f8 droplets: Add origin remote to python-zulip-api as well. 2018-05-07 08:42:18 -07:00
Vishnu Ks 13f6cbeefd droplets: Change droplet region from SFO1 to NYC3.
Since the region of base.zulipdev.org is
NYC3 we had to add SFO1 as an additional
region each time a snapshot of base droplet
is created. This is required as droplets
can be created in SFO1 only if there is
an image present in that region. Adding
of droplet image to SFO1 takes a lot of
time as well as cost 2X as we are storing
2 images. It's better to just create new
droplets in NYC3 instead. Alternatively we
can create a new base droplet in SFO1 if
we want all the droplets to be created in
SFO1.
2018-05-07 08:42:18 -07:00
Vishnu Ks d6c5635550 droplets: Update snapshot id. 2018-05-07 08:42:18 -07:00
Greg Price 9f3052b0ef mypy: Move remaining options on type-check semantics to config file.
This puts all of this config in one place, and also needs a lot fewer
lines to describe it; which, combined, makes it a lot clearer what our
normal config actually is.  (I'd been looking at this script for a few
minutes without realizing that we have `--disallow-untyped-defs` *on*
by default, not off.)

Experimenting with different values is still easy; just comment the
line in the config.
2018-05-06 19:33:55 -07:00
Greg Price 6b1e76c1b1 mypy: Move cache-dir to config file. 2018-05-06 19:33:55 -07:00
Greg Price 724e849e2b mypy: Move follow-imports to config file.
It's cleaner to have this configuration in a nice declarative
config file than embedded in a script.
2018-05-06 19:33:55 -07:00
Greg Price ff178bb27a mypy: Drop now-redundant `-i` option.
This used to be a synonym for `--incremental`.  Since mypy 0.590,
incremental mode is the default, and the flag is ignored; so we
can happily drop it.
2018-05-06 19:33:55 -07:00
Tim Abbott 41841221ee scripts: Remove obsolete zesty configuration.
Zesty already reached end-of-life, so we'll never support it.

And in one place, we add support for bionic.
2018-05-05 11:41:57 -07:00
Tim Abbott d178c53a10 provision: Add official support for provisioning directly on Debian. 2018-05-05 11:00:47 -07:00
Tim Abbott 8ea8bfe285 puppet: Add basic configuration for Ubuntu bionic. 2018-05-05 10:49:09 -07:00
Tim Abbott b193330474 provision: Add tsearch_extras package for stretch. 2018-05-05 10:49:09 -07:00
Tim Abbott 78f3cff151 provision: Fix unescaped reference to vagrant.
Previously, this would actually try to run `vagrant ssh`, rather than
printing it :(.
2018-05-05 10:48:37 -07:00
Tim Abbott 9d6233a457 capitalization: Add an exclude rule for zulip_org_id. 2018-05-03 22:37:24 -07:00
Armaan Ahluwalia cca10beb78 css/webpack: Moved archive styles to webpack bundle.
This commit moves the stylesheets under the archive bundle in
the Django pipeline to being compiled by webpack instead. It
also removes a remaining call to a portico stylesheet that no
longer exists.
2018-05-03 19:15:49 -07:00
Armaan Ahluwalia 54bf2a6231 css/webpack: Transition landing-page.css to webpack.
This commit transitions landing-page.css from the Django pipeline
to being compiled by webpack as landing-page.scss under the
'landing-page' and 'integration' bundles.
2018-05-03 19:15:23 -07:00
Armaan Ahluwalia 1c016e990d css/webpack: Transition common.css to webpack.
This commit transitions common.css from the Django pipeline
to being compiled by webpack under the common bundle.
2018-05-03 19:15:23 -07:00
Tim Abbott bd110ccb3c test-migrations: Add django_two_factor migrations to exclude list.
We don't control the names for these migrations.
2018-05-03 12:23:25 -07:00
Priyank Patel 1e1b72f6c8 webpack: Transition stats.css to use webpack.
The stats.scss file is added to activity bundle.
2018-05-03 08:08:23 -07:00
Priyank Patel c780bc33ba webpack: Transition activity.css to use webpack.
The activity.scss file is added to activity bundle.
2018-05-03 07:53:24 -07:00
Armaan Ahluwalia 5f7b47e20c css: Transition 'app.css' to SCSS.
This commit transitions all styles in app.css in the Django pipeline
to being compiled by webpack in an app-styles bundle, and renames the
various files to now be processed as SCSS.

To implement this transition, we move the old CSS file refernces in
settings.py and replace them with a bundle declared in
`webpack.assets.json` and includedn in the index.html template

Tweaked by tabbott to keep the list of files in `app.css` in
`webpack.assets.json`, and to preserve the ordering from the old
`settings.py`.
2018-05-02 17:13:16 -07:00
Armaan Ahluwalia 64dadae697 webpack: Add css-hot-loader to remove flash on unstyled content.
This commit removes the flash on unstyled content while in dev
mode that was caused by the use of style-loader. Instead it
enables mini-css-extract-plugin in dev in combination with
css-hot-loader which enables HMR for development.

This is because mini-css-extract-plugin does not currently support
HMR out of the box. It also adds a SourceMapDevtoolPlugin to enable
sourcemaps with css since mini-css breaks sourcemaps when used in
combination with the cheap-module-evel-source-map setting.

Related issues:
https://github.com/webpack-contrib/mini-css-extract-plugin/issues/34
https://github.com/webpack-contrib/mini-css-extract-plugin/issues/29
2018-05-02 16:55:58 -07:00
Aditya Bansal a62efd55df linter: Make duplicate html tag id detection work with archives.
We modify check-templates to check for duplicate id's in archive
templates and app templates separately. This means we are allowing
app and archive templates to potentially use same id's. This is
needed because we intend to re use some js from the main app and
having same id's help achieve that.
Note: We haven't up until this point actually added archive
templates. This commit is more of a preparatory commit for merging
the basic archive infra.
2018-05-02 15:23:33 -07:00
Armaan Ahluwalia 9f80418d12 webpack: Add support for png file imports in file-loader.
Adds support for importing png files using file-loader in webpack.
Changes the name of the output directory to be files instead of
fonts for better readability.
2018-05-02 09:45:04 -07:00
Armaan Ahluwalia fce6882eb9 webpack: Move the styles consumed by 5xx.html to webpack.
This commit removes the need for portico.css to be generated
by the Django pipeline and makes the error page use the css
file compiled by webpack instead.
2018-05-02 09:45:01 -07:00
Tim Abbott a4ff917789 emails: Fix spelling of "Unknown IP" and tag for translation. 2018-04-30 12:04:39 -07:00
Priyank Patel 50b13219a3 webpack: Combine both js and css into one portico bundle.
Combines, both portico js and css into one bundle. This for now solve
the issue of an empty js bundle being generated by webpack for the
portico-styles stylesheet.
2018-04-30 10:23:39 -07:00
Tim Abbott 7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
Steve Howell 605a90ce36 node tests: Reach 100% coverage for scroll_util.js. 2018-04-28 11:15:14 -07:00
Steve Howell 65d8eb3189 buddy list: Extract user_search.js.
This was a bit more than moving code.  I extracted the
following things:

        $widget (and three helper methods)
        $input
        text()
        empty()
        expand_column
        close_widget
        activity.clear_highlight

There was a minor bug before this commit, where we were inconsistent
about trimming spaces.  The introduction of text() and empty() should
prevent bugs where users type the space bar into search.
2018-04-28 11:15:14 -07:00
Armaan Ahluwalia fb0a421b8c webpack: Silence most webpack output for tests.
This commit adds a --quiet argument to tools/webpack which removes
the verbose output from webpack and replaces it with showing only
errors. It also makes tools/run-dev --tests use this argument while
running webpack for testing.

Tweaked by tabbott to clean up the code a bit.
2018-04-28 09:32:10 -07:00
Tim Abbott 699c4381f2 test-api: Add a PROVISION_VERSION --force option.
This brings this in line with the Casper and other tests.
2018-04-27 17:21:12 -07:00
Vishnu Ks e3314be114 emails: Add translation tags to confirm_new_email. 2018-04-27 11:59:36 -07:00
Armaan Ahluwalia 93ac40105f CSS: Move portico styles to webpack compilation.
static/styles/scss/portico.scss is now compiled by webpack
and supports SCSS syntax.

Changed the server-side templates to render the portico-styles
bundle instead of directly requiring the portico stylesheet. This
allows webpack to handle stylesheet compilation and minification.

We use the mini-css-extract-plugin to extract out css from the
includes in webpack and let webpacks production mode handle
minification. Currently we're not able to use it for dev mode
because it does not support HMR so we use style-loader instead.
Once the plugin supports HMR we can go on to use it for both
dev and prod.

The downside of this is that when reloading pages in the development
environment, there's an annoying flash of unstyled content :(.

It is now possible to make a change in any of the styles included
by static/styles/scss/portico.scss and see the code reload live
in the browser. This is because style-loader which we currently
use has the module.accept code built-in.
2018-04-27 09:04:50 -07:00
Armaan Ahluwalia f20671a509 webpack: Fix Hot Module Reloading in webpack.
This commit fixes hot module replacement in webpack. To do this
we open port 9994 used by webpack to communicate between browser
and devserver. The attempts to forward the proxy from 9991 failed
so the last resort was to open up the webpack port.
It also removes an uncessary plugin in the webpack config and moves
the --hot flag to tools/webpack.
2018-04-27 09:04:49 -07:00
Steve Howell 9ece6d2be4 lint: Exempt "opts." from addClass checks.
If we use something like "opts.highlight_class", it's probably
in a generic widget.
2018-04-26 08:42:47 -07:00
Tim Abbott 1ff909d971 coverage: Exclude Zephyr ccache logic from test coverage.
That code path is pretty constrained in how it's used, and is only for
the legacy Zephyr integration which we don't expect to spend effort
on again.
2018-04-25 22:41:31 -07:00
Tim Abbott 47bdf5ecba coverage: Exclude generate_test_data from testing.
That file is just used for generating manual testing data; it is
actually tested by our suite, just before coverage starts running.
2018-04-25 22:38:54 -07:00
Tim Abbott d5946de718 decorator: Add nocoverage comments for rate_limit decorator.
We've already got a bunch of other comments on work we need to do for
this decorator and an open issue that will ensure we at some point
rework this and add tests for it.  In the meantime, I'd like to lock
down the rest of decorator.py at 100% coverage.

Fixes #1000.
2018-04-25 22:37:12 -07:00
Priyank Patel bc454bab88 webpack: Disable host check for webpack-dev-server.
Webpack dev server by default does host checking for requests. so
in dev enviorment if the the request came for zulipdev.com it would not
send js files which caused dev envoirment to not work.
2018-04-24 14:14:20 -07:00
Emilio Schadt b6f5ea0fd2 docs: Add README.md to zulip/docs to prevent reading on GitHub.
This should help avoid confusing about broken links.

Tweaked by tabbott to fix formatting and the linter.
2018-04-24 10:50:39 -07:00
Vishnu Ks 7533796ea9 popover: Add an option to show user profile.
Fixes #8880
2018-04-23 16:56:24 -07:00
Tim Abbott 1703e23980 templates: Move all core app templates into a subdirectory.
This should make it easier to find the templates that are actually
part of the core webapp, instead of having them all mixed together
with the portico pages.
2018-04-23 16:46:37 -07:00
Tim Abbott 8fc04a074d test-backend: Update coverage excludes for new import_realm.py. 2018-04-23 16:26:01 -07:00
Armaan Ahluwalia 97ec5e0aaa webpack: Tweak config file for stats and faster recomp on HMR.
Update the config file to show slightly more information while
compiling webpack. Also decreased the time webpack waits before
recompiling in order to speed up HMR.
2018-04-23 15:49:33 -07:00
Armaan Ahluwalia c70d26224d webpack: Change devtool sourcemap strategy.
Changed the devtoool setting for development from 'eval' to
'cheap-module-eval-source-map' as it has better support for
breakpoints in Google Chrome and the difference is time is
negligible at the number and size of files currently being
consumed by webpack. This stragtegy can be reviewed in the
future as the size of files grows or Chrome adds better
support.
2018-04-23 15:49:33 -07:00
Armaan Ahluwalia fc7aa1a771 webpack: Upgrade webpack version 4.5.0.
Upgrade webpack to latest version at the time of authoring. This
involves upgrading webpack version and its loaders to compatible
versions. It also involved editing tools/webpack to use the
executable for webpack-cli instead because of a change in how the
webpack package wants you to handle shell execution.
It also fixes the confugration for TypeScript in the webpack config
as that was previously broken. Including TypeScript files in JS
files compiled by webpack now works.
2018-04-23 15:49:33 -07:00
Priyank Patel 29f04511c0 node_tests: Use nyc instead of deceprated istanbul package. 2018-04-23 15:27:16 -04:00
Tim Abbott 3f4f94d111 check-capitalization: Add "G Suite" to proper nouns list.
This fixes an error caught by CI.
2018-04-23 10:21:10 -07:00
Steve Howell 6d4855bd6a lint: Prevent accidental use of const.
This prevents us from using const in our JS code, with exceptions
for test code and the portico.  Hopefully this is just a temporary
rule until we make our pipelines with work with ES6.

I tried to prevent "let", but that was too noisy.

This adjusts the one false-negative case of using const in a comment.
2018-04-22 20:11:38 -07:00
Steve Howell 3f1930f9c5 buddy list: Extract buddy_data.js. 2018-04-22 20:08:08 -07:00
Steve Howell 536236d9b1 buddy list: Extract buddy_list.js. 2018-04-22 20:08:08 -07:00
Eeshan Garg ca5ea20ab7 doc tests: Add portico pages to tools/test-help-documentation.
Fixes #9117.
2018-04-21 21:51:32 -07:00
Preston Hansen 76d6c71595 tests: Move zerver/fixtures to zerver/tests/fixtures for clarity.
Fixes #9153.
2018-04-19 21:50:17 -07:00
Joshua Pan 99f07fe2e2 tests: Fully cover top_left_corner.js. 2018-04-19 14:56:55 -04:00
Eeshan Garg 19177a4aff webhooks: Move github_webhook/ to github/. 2018-04-19 11:00:55 -07:00
Eeshan Garg 48b8558c02 webhooks: Move github/ to github_legacy/ and remove docs. 2018-04-19 11:00:55 -07:00
Steve Howell f8fd169a7d node tests: Allow --coverage for individual files.
We now allow you to run --coverage on individual files.  This helps
when you want to make sure a file is being covered directly and not
just getting incidental coverage from higher level tests.

Before this commit, we were conflating wanting coverage reports with
wanting coverage checks.  For individual files, we now solve that by
simply eliminating the coverage checks.  This required some minor
refactoring to extract some functions.
2018-04-17 17:52:19 -07:00
Vishnu Ks cc93ac34a8 coverage: Add coverage to estimate_recent_messages.
With this message.py is fully covered and can be
removed from not_yet_fully_covered in test-backend.
2018-04-17 11:01:20 -07:00
Alyssa Wagenmaker d4e5777296 tests: Test user unsubscribing before soft deactivation.
Brings lib/soft_deactivation.py up to 100% test coverage.

Improves: #7089.
2018-04-16 11:28:08 -07:00
Tarun Kumar 0b62410f5e pm_pill: Achieve 100% node-test coverage for compose_pm_pill.js. 2018-04-13 11:59:57 -07:00
Tarun Kumar 51601fab44 muting: Achieve 100% node-test coverage for settings_muting.js. 2018-04-13 11:59:57 -07:00
Tim Abbott 105eed049e install-node: Fix leaking of $HOME.
This fixes a bug where provision was failing since our most recent
upgrade to yarn/nvm/node.

It turns out my original fix was the correct fix, but to the wrong
third-party tool: nvm, not yarn, was the offender.
2018-04-12 14:32:36 -07:00
Tim Abbott feef35bf25 linter: Add checks for sloppy use of .html().
Since jQuery's .html() can be a source of security bugs, we add a new
lint rule that tries to catch common problematic uses.
2018-04-12 09:50:14 -07:00
Shubham Padia 55619cbe70 browser-support: Add string.prototype.endswith polyfill.
String.prototype.endsWith is not supported in ie11.
Adds string.prototype.endswith package to dependencies and places
it at `common` entry point in webpack.assets.json.
2018-04-11 15:40:57 -07:00
Shubham Padia 57a494f94d browser-support: Add codepointat polyfill to `common` entry point.
Adds string.prototype.codepointat which was added as a polyfill
earlier to the project but was not added to `common` entry point.
2018-04-09 12:16:19 -07:00
Shubham Padia b906562f22 browser-support: Add string.prototype.startsWith polyfill.
Fixes #8944.
Adds string.prototype.startswith package to dependencies and places
it at `common` entry point in webpack.assets.json. As common.js is
loaded on all code paths first, there is no need to place this package
into other entry points.
2018-04-09 12:16:19 -07:00
Aditya Bansal b9f1acb300 linter: Enforce 2 space indents on tags spread over multiple lines.
We make some specific cases of tags use 2 space indents.
The case description:
* A tag with opening tag spread over multiple lines and closing tag
on the same line as of the closing angle bracket of the opening tag.
* A tag with opening tag spread over multiple lines and closing tag
not on the same line as of the closing angle bracket of the opening
tag.

Example:
Case 1:

Not linted:
<button type="button"
class="btn btn-primary btn-small">{{t "Yes" }}</button>

After linting:
<button type="button"
  class="btn btn-primary btn-small">{{t "Yes" }}</button>

Case 2:

Before linting:
<div class = "foo"
     id = "bar"
     role = "whatever">
     {{ bla }}
</div>

After linting:
<div class = "foo"
  id = "bar"
  role = "whatever">
    {{ bla }}
</div>
2018-04-07 20:08:44 -07:00
Aditya Bansal 550222dede linter: Make multiline handlebar singleton tags use 2 space indentation. 2018-04-07 20:08:38 -07:00
Aditya Bansal 2fe012ffff linter: Make html singleton tags use 2 space indentation. 2018-04-07 20:08:31 -07:00
Alyssa Wagenmaker df666c3dfc test-backend: Report fully covered files still in not_yet_fully_covered. 2018-04-06 17:29:18 -04:00
Ben Reeves 29a079ebbf test-backend: Remove fully covered files from not_yet_fully_covered. 2018-04-06 17:29:18 -04:00
Tim Abbott b0b134cb4c help: Clean up settings_html and subscriptions_html.
After some thinking, I don't think there's any actual value to doing
the ../ style relative links here, whereas there is actual harm from
the links being slightly broken in the current model.  We fix this by
just using /#settings as the URL.

Fixes #8978.
2018-04-05 14:48:26 -07:00
Tim Abbott ed5b374ffa help docs: Fix a sorta buggy URL test. 2018-04-05 13:07:34 -07:00
Steve Howell e145258fee node tests: Add 100% coverage for components.js. 2018-03-29 08:09:38 -07:00
neiljp (Neil Pilgrim) 090b47ed19 mypy: Add explicit Optional for default=None parameters in various files. 2018-03-28 12:31:51 -07:00
neiljp (Neil Pilgrim) 07971d3a66 linter: Add check that Optional models are tagged with null=True.
Exclude NullBooleanField explicitly.

Amend one line in models.py found from this linter change.
2018-03-27 10:21:25 -07:00
neiljp (Neil Pilgrim) 7d93f5726c linter: Add check that null=True lines are annotated as Optional.
Also add two missing annotations of this type that were missed
until linted.
2018-03-27 10:21:25 -07:00
Joshua Pan 0f58f20ad8 tests: Clean up test-js-with-node output on success. 2018-03-26 06:35:58 -04:00
Joshua Pan ed9eb3bdb5 tests: Make test-js-with-node handling smarter.
Added support for passing a filename without `.js` suffix.
This then fixed the issue of no complaints for invalid test
files. Now, throws an error for invalid test files.

Fixes #8579.
2018-03-26 06:35:58 -04:00
neiljp (Neil Pilgrim) 9a49812cde mypy: Extract default run-mypy options into mypy.ini. 2018-03-23 16:53:43 -07:00
neiljp (Neil Pilgrim) fb4f5c8570 tools: remove strict-optional command from run-mypy.
This is now controlled globally and on a per-file basis in mypy.ini.
2018-03-23 16:53:43 -07:00
neiljp (Neil Pilgrim) 70117f0e83 mypy: Add asserts of intermediate values in update-authors-json. 2018-03-23 13:07:39 -07:00
neiljp (Neil Pilgrim) f0e3fcb937 mypy: Amend test-backend:internet_guard to return NoReturn.
It is guaranteed to raise an exception, so this is reasonable,
and it satisfies mypy.
2018-03-23 11:39:54 -07:00
sinwar 675e844e6b team page: Add contributors of zulip-terminal.
It adds zulip terminal contributions to
team page using github apis.
2018-03-22 21:06:18 -07:00
Tim Abbott 08a6271b54 update-authors-json: Improve error handling for duplicate commits.
It's possible that a user with duplicate commits might not be in the
top 100 users for a given repo.
2018-03-22 14:58:13 -07:00
Gooca 2363c9955a ui scrollbar: Update perfect-scrollbar to 1.3.0.
Update perfect-scrollbar to fix stutter space-scrolling in #8544. Also
reworked deprecated `element.perfectScrollbar` to `new
PerfectScrollbar(element)`.  Lastly, updated provision version and
changed node module path to new path.

This also refactors perfect-scrollbar in help.js to work with updated
version of perfect-scrollbar. Because the update also changed
perfect-scrollbar's css selectors for all scrollbars in zulip, we
update those too.

Fixes #8544.
2018-03-20 21:59:11 -07:00
Tim Abbott 791953963b check-capitalization: Include banned words in overall output.
This locks down the ban on use of "Realm" in translated strings in the
codebase.
2018-03-16 16:59:02 -07:00
Tim Abbott 8e334b0f64 check-capitalization: Exclude certain uses of the word "realm".
Uses that are actually a variable name don't make sense to exclude.
2018-03-16 16:58:49 -07:00
neiljp (Neil Pilgrim) 966ca7015f mypy: Finalize migration of cache.py to python3 function annotation.
- Use forward declarations of some types from models.py to avoid cycles.
- Remove cache.py from linter rule exclude list to ensure it stays that way.
2018-03-16 11:29:12 -07:00
Vishnu Ks c385fcefa8 templates: Fix translation tag for disposable email tooltip. 2018-03-15 14:35:24 -07:00
neiljp (Neil Pilgrim) a3d8cc1f86 Mypy: Migrate tornado/descriptors.py to python3 function annotations.
Use quoted form of ClientDescriptor to break presumed circular dependency,
and modify import statement to shorten quotes.
2018-03-15 12:54:43 -07:00
neiljp (Neil Pilgrim) a7bfb09067 Mypy: Use models.py QuerySet annotation approach in stream_subscription.py.
Namely, annotate as best as possible, and add notes to indicate preference,
if QuerySet develops generic typing.

Note that the return values of functions with annotations changed in this
commit are used elsewhere as QuerySets, so the Sequence[T] approach used
for some functions in models.py is not applicable.
2018-03-15 12:54:43 -07:00
neiljp (Neil Pilgrim) ce4ac0d2cf Mypy: Ensure consistency of QuerySet return types in models.py.
Other functions took the form of returning Sequence[T] when the QuerySet
functionality is unused beyond the function, with T being the objects
filtered for in the function body; this commit follows that practice for the
one remaining python2 comment-annotated function, completing the transition
of models.py to py3.5 function annotations.

A note is also added to another function regarding a need to return a
QuerySet, and ideally a QuerySet[T] in line with the other functions, as and
when QuerySet becomes annotated as a generic.
2018-03-15 12:54:43 -07:00
Tarun Kumar c8d5cf7d6c user-groups: Achieve 100% node-test coverage of user_groups.js.
Fixes #8704.
2018-03-15 10:09:28 -07:00
Harshit Bansal 40958e0824 emoji: Switch to 64px 256 color indexed sprite sheets.
This commit switches our emoji infrastructure to use 256 color indexed
64px spritesheets. Earlier we were using non-indexed 32px spritesheets
which were blurry on high dpi displays. These indexed spritesheets not
only provide a crispier display but are also smaller in size.

This commit also removes the `emoji-datasource` package as a dependency
as all the data is now sourced from individual datasource packages.

Fixes: #7862.
2018-03-14 10:28:45 -07:00
neiljp (Neil Pilgrim) 43ff4ea1f7 tools: Ensure mypy is run on .pyi files. 2018-03-13 13:49:39 -07:00
Tim Abbott 8866c23b5e test-backend: Enforce 100% test coverage on models.py.
Following the exclusions and code deletions we just did, we can now
lock down 100% test coverage on zerver/models.py.
2018-03-12 11:52:39 -07:00
neiljp (Neil Pilgrim) ee2b8bfbfb linter: Require Py3 function annotations, with some exceptions.
Main exceptions are scripts/tools/puppet & tests.

Other current exclusions:
- api_test_helpers.py (avoid changing test code);
- cache.py, due to comments in file;
- models.py, due to failure on QuerySet[Message];
- stream_subscription.py, due to failure on QuerySet[Subscription];
- tornado/descriptors.py;
- views/streams.py, due to remaining FuncKwargPair issue;
- zthumbor/, since thumbor is in python2.

Tweaked by tabbott to partially document the stauts in comments.
2018-03-12 11:23:30 -07:00
Umair Khan 728cd04c4d check-capitalization: Check for banned words.
Fixes #8619
2018-03-11 15:34:47 -07:00
Steve Howell fef2840531 Add FetchStatus class.
We will use this in subsequent commits to replace the
`load_more_enabled` flag.
2018-03-09 14:20:58 -08:00
sinwar 6bed6ccdcf team page: Remove duplicate contribution counts.
It removes duplicate contribution count from zulip-server.

Fixes #7836.
2018-03-07 17:25:41 -08:00
Steve Howell 3e76d523c7 node tests: Add tests for user_pill.js.
This gets coverage to 100%.  (We already had a lot of indirect
coverage, but these tests should have more obvious failure
symptoms.)
2018-03-07 15:53:11 -08:00
Greg Price f22712e8e8 install: Go straight through by default.
This flips the experimental `--express` option to be the default.

We retain the old behavior, where the script exits before
`initialize-database`, as an option `--no-init-db`; it might be useful
in e.g. a migration scenario (from a Zulip install elsewhere, or
another chat system) where the admin wants to set up the database
separately.

The install instructions are adjusted to match, getting shorter by two
steps and a bunch of words.  I think this opens up opportunities to
refactor the text to simplify things further, too, but leaving that
for another commit.

Also tweak the "production" test suite to match.
2018-03-06 19:43:02 -08:00
Greg Price 8111848ac4 test-install: Do a dist-upgrade in prepare-base.
This keeps the base tree up to date, saving the time we'd spend
doing the same upgrades in each test install.
2018-03-06 19:43:02 -08:00
Tim Abbott f253f96211 lint: Add "Gravatar" to capitalized terms in linter. 2018-03-05 10:29:18 -08:00
Tim Abbott 4d8e18e58f settings: Move display_checkmark to settings_ui.js. 2018-03-04 18:26:27 -08:00
Tim Abbott 6a06e453c7 display_settings: Extend de-duplication of unnecessary logic.
We add a new option to the change_display_setting function to support
cases with a custom on-success or on-failure message.
2018-03-04 18:09:46 -08:00
Balaji2198 5aa7098c81 display_settings: Change success/failure feedback interface.
This replaces the cumbersome system we had for giving users feedback
on settings state changes in the display settings UI.

We expect this new system to be what we will attempt to migrate other
settings widgets to match over the coming weeks and months.  It also
provides the opportunity to significant refactor away a lot of the
code duplication in settings_display.js.

Thanks to Brock Whittaker for redoing the styling and improving the
code simplicity.

Fixes #7622.
2018-03-04 17:47:05 -08:00
Tim Abbott a3ca7ee1c7 lint: Add a few strings to capitalization linter.
My hope is that we'll be able to eliminate these strings from the UI
altogether soon, but this is needed to keep CI passing for now.
2018-03-04 16:34:00 -08:00
Robert Hönig 81ba7a1e40 Mark DigestWorker and PushNotificationsWorker as nocoverage.
These two classes are tricky to test, and nocoverage-ing them
allows us to mark queue_processors.py as fully covered. We
still want to cover these two workers at some point, but for
now, it's nice to enforce full coverage for any future changes
to queue_processors.py.

Fixes (sort of) #6542.
2018-03-04 13:31:33 -08:00
Tim Abbott 530e098916 node: Remove 100% coverage requirement for user groups.
This is a temporary change to get CI passing again; we lost some
coverage in aed8cc9062, which fixed some
major bugs in the feature.
2018-03-04 10:11:11 -08:00
YJDave f5a24efdd8 static/js: Add `settings_ui.js` file for common settings UI functions. 2018-03-03 08:38:54 -08:00
Eeshan Garg 59581f6a32 tools/lint: Require a space after every `:` in JSON files. 2018-03-02 14:10:53 -08:00
Tim Abbott 335c383bbb lint: Add some excludes to capitalization linter. 2018-03-02 11:55:46 -08:00
Tim Abbott 895b5642f1 commit-message-list: Don't print anything if no commits.
This fixes some output spam when running the linter at
upstream/master.
2018-03-02 11:51:08 -08:00
Tim Abbott 14764d015a settings: Tag cookie bot placeholder short name for translation. 2018-03-02 09:33:16 -08:00
Tim Abbott e18537d842 emoji: Add a translation tag for new_emoji placeholder. 2018-03-02 09:32:09 -08:00
Tim Abbott 9c4b0cc100 lint: Clean up translation tag lint for filter settings. 2018-03-02 09:30:42 -08:00
Tim Abbott f9210bace8 lint: Add a translation tag for marketing. 2018-03-02 09:28:57 -08:00
Joshua Pan 5777582013 lint: Tighten placeholder lint rule to exclude links.
Fixes #8420.
2018-03-02 09:24:09 -08:00
Tim Abbott 5ccd3aa71b i18n: Run process-mobile-i18n when syncing translations. 2018-03-02 08:59:00 -08:00
Tim Abbott 3be0b3dd91 mypy: Fix errors in linter.
We should figure out a better way to manage this issue.
2018-03-01 20:33:30 -08:00
Tim Abbott 8d00266fd6 lint: Add a linter rule banning use of sudo in most scripts.
This should help prevent against bugs where we accidentally introduce
use of sudo somewhere in the production installer or upgrade code path
(these used to happen all the time), which doesn't work on production
systems that don't have sudo setup.
2018-03-01 16:30:31 -08:00
Greg Price 72497d61c1 test-migrations: Clean up a bit. 2018-03-01 15:03:55 -08:00
Aditya Bansal bd0a5d5d69 css-minifier: Lint for '+' operator in calc().
Fixes: #8403.
2018-03-01 13:26:16 -08:00
Tim Abbott 6b31752f82 user_groups: Change placeholder group to "Marketing team".
This is much more something that would apply to likely customer
organizations.
2018-03-01 11:28:24 -08:00
Vishnu Ks 36f6ad76b4 admin ui: Set usergroup name input maxlength to 100.
Same as that of the maxlength defined in backend.
2018-03-01 11:27:26 -08:00
Greg Price 4475950ddf queue: Restore prematurely-cut upgrade path.
Revert c8f034e9a "queue: Remove missedmessage_email_senders code."
As the comment in the code says, it ensures a smooth upgrade path
from 1.7.x; we can delete it in master after 1.8.0 is released.
The removal commit was merged early due to a communication failure.
2018-02-28 11:15:53 -08:00
Umair Khan c8f034e9a0 queue: Remove missedmessage_email_senders code.
After 68513952fb, all emails are sent through email_senders queue.
This commit removes code related to the legacy queue.
2018-02-21 16:43:56 -08:00
Eeshan Garg be0a04f33e api docs: Test sample fixtures for unauthorized_errors_fatal.
This commit adds tests for the sample fixtures for when
unauthorized_errors_fatal is passed to client.add_subscriptions.
2018-02-20 14:54:12 -08:00
Eeshan Garg 06bf47d2f5 api docs: Test sample fixture for user not authorized error.
This commit adds tests for the fixture for when a user is not
authorized (perhaps because the query requires the use of admin
privileges) for a particular query.
2018-02-20 14:52:09 -08:00
Eeshan Garg 124c672a7c api docs: Test sample fixture for message edit permission error.
In templates/zerver/api/update-message.md, we have a sample fixture
for when a zulip.Client does not have the permission to update/edit
a particular message. This commit adds a test for that fixture.
Also, tools/test-api now also uses a non-admin client for this test,
which might come in handy in the future.
2018-02-20 14:52:09 -08:00
Steve Howell 1f6ddf0110 refactor: Extract transmit.js from compose.js.
We now isolate the code to transmit messages into transmit.js.
It is stable code that most folks doing UI work in compose.js don't
care about the details of, so it's just clutter there.  Also, we may
soon have other widgets than the compose box that send messages.

This change mostly preserves test coverage, although in some cases
we stub at a higher level for the compose path (this is a good thing).
Extracting out transmit.js allows us to lock down 100% coverage on that
file.
2018-02-20 09:29:26 -08:00
rht 70a41cc2eb tools/tests: Use Python 3 syntax for typing. 2018-02-19 10:17:03 -08:00
ihsavru 4103996a22 org-settings: Fix styling of Allowed Domains modal.
Fixes: #7628.

Cleaned up by Brock Whittaker and Node tests fixed by Steve Howell.
2018-02-16 16:00:21 -08:00
Aditya Bansal 6fce1d7834 template_parser: Add parsing support for self closing tags as per HTML5.
In this commit we add support for some tags which are also called
void-elements according to
http://w3c.github.io/html/syntax.html#void-elements to be parsed by
our template parser and get tagged as singleton_html_tags.

Fixes: #8387.
2018-02-15 17:47:43 -05:00
Greg Price b0b0777cd0 translation: Make new tagmessages script a bit easier to run.
Now executable! Just run `tools/tagmessages`.

Also, get the username and password from a `.transifexrc` file.

And hardcode the project slug to `zulip-test` rather than to `zulip`;
the Transifex API is bad at namespacing, so this makes it possible to
run this script on a test project (the only way we're currently using
it) even for people like me who can also upload to the real Zulip
project on Transifex.
2018-02-15 13:38:09 -08:00
Robert Hönig ebf79427ca translation: Add script to tag strings in Transifex.
For now, tags are only used for backend translations with django.
2018-02-15 13:38:09 -08:00
Tim Abbott be3d43bb46 lint: Add linter rule banning $.get and friends.
These are not allowed in our style guide.
2018-02-13 16:47:59 -08:00
Shubham Padia e1f943913a lint: Allow revert commit messages in gitlint.
Fixes #8145.
2018-02-13 09:21:01 -08:00
Aditya Bansal 35969edd66 deps: Replace libz-dev with zlib1g-dev since the former was renamed. 2018-02-12 14:40:26 -08:00
Aditya Bansal d5736ef65a provision.py: Remove duplicate install candidated from APT_DEPENDENCIES.
This is necessary, since we now have some dependencies in common
between the Thumbor and main Zulip dependency lists.
2018-02-12 14:40:26 -08:00
Rhea Parekh 3ebd30120c slack importer: Remove depreciated test.
This test was added in '/tests/test_slack_importer' in
6addf79edb.
2018-02-12 00:43:02 +05:30
Greg Price b374ea7357 mypy: Pass --show-traceback.
This saves a bit of debugging in the event that we hit a crash
inside mypy.
2018-02-09 19:42:49 -08:00
Eeshan Garg 929724e5e7 api docs: Add page for common error payloads.
We now have a separate page for common error payloads, for example,
the payload for when the client's API key is invalid. All error
payloads that are presented on this page will be tested similarly
to our other non-error sample fixtures.
2018-02-08 17:58:41 -08:00
Greg Price 86590dfdbe test-install: Add command destroy-all to clean up test containers.
This is just the one-liner I've been keeping in my shell history,
cleaned up a bit (newlines!) and with 28 lines of CLI boilerplate
added in front.
2018-02-08 17:29:41 -08:00
Greg Price 6e633f8e2f install: Use readlink -f rather than realpath.
It does exactly the same thing, though the name is less transparent; and
it simplifies the script by avoiding an extra, early `apt-get install`.
2018-02-08 17:22:02 -08:00
Greg Price 4c5326ce85 test-install: Factor out booted-yet-p polling loop, use in prepare-base.
Otherwise prepare-base is likely to fail when first run (but then
succeed when rerun, because the container is left running), because
the container isn't up yet when we try to operate in it.

Also clean up the placement of `set -e` vs `set -x`.
2018-02-08 16:34:49 -08:00
Greg Price fc9970e561 test-install: Add xenial support. 2018-02-08 16:34:49 -08:00
Tim Abbott 9249454518 update-authors-json: Fix handling deleted GitHub accounts.
Apparently, we've now had the first time one of our contributors had
their account deleted (at least, the author page for the contributor
who has 21 commits in python-zulip-api now 404s).
2018-02-08 12:17:20 -08:00
Umair Khan 764cd977d6 test-queue-worker-reload: Increase delay to 1.3s.
See 625939 for more information. In short, the purpose of this delay is
to give autoreload code enough time to touch every watched file at least
once before the change is made.
2018-02-06 10:35:20 -08:00
Umair Khan c415cc74d7 validate_stream_message_address_info: Add i18n tags.
Fixes #7076
2018-02-05 16:26:24 -08:00
rht 396a8b644d provision.py: Colorize error messages for fail condition. 2018-02-05 11:44:33 -08:00
Tim Abbott 0c4aa48895 provision: Clean up error message for missing virtualenv.
This new message should be clearer; it does come up occasionally.
2018-02-05 11:44:33 -08:00
rht e8a5f68980 provision: Extract color codes into standard variables.
This uses the standard color codes we have in the Python
`zulip_tools` library.

Tweaked by tabbott to clean up some bugs.
2018-02-05 11:44:33 -08:00
Greg Price e3c82fa3ec tools/test-all: Fix small desync with CI scripts.
This script and our CI scripts tools/travis/{backend,frontend} have
stayed pretty well in sync in the 6 months since 360c27ded made that
relationship explicit and easy to check!

Just one small exception; so fix that.
2018-02-01 11:38:51 -08:00
Greg Price 34f4c822cb ci: Cut out commented-out versions of tools/lint calls.
This may or may not be temporary, but either way, the other code is
there in source control, and the "why" of disabling gitlint is the
helpful bit for a comment.
2018-02-01 11:37:00 -08:00
Robert Hönig 53b4c793f6 setup_venvs.py: Enforce Python 2.7 for zulip-thumbor-venv setup.
zulip-thumbor-venv runs only with Python 2.7. This could cause
provision failures on setups where Python 2.7 is not the default.
2018-01-31 11:29:34 -08:00
Greg Price f995ad1ed5 circleci: Simplify Dockerfile templating a bit.
Injecting the generated-file warning into the settings dict felt a
little unnecessarily magical.  A warning like this is always going
to be at the top; the way it might differ between files is mainly
if the syntax for a comment varies, and in that case a simple
substitution like we're doing in this template wouldn't be enough
to express the difference anyway.  So, embrace the hardcoding.

Now, the template and the images.yml entry have a very simple
relationship: the keys in one are exactly the keys in the other.
That's good for people quickly and confidently understanding it.
2018-01-31 10:53:13 -08:00
Greg Price 8a0b5a29b9 circleci: Use the joy of `os.makedirs(..., exist_ok=True)`.
Since Python 3.2, we no longer need to write this little wrapper
all over our own code!  There was much rejoicing.
2018-01-31 10:53:13 -08:00
Vishnu Ks 91506f1bb3 circleci: Generate xenial dockerfiles along with trusty. 2018-01-31 10:53:13 -08:00
Vishnu Ks be328b2c7b circleci: Create script for generating Dockerfiles.
[greg: updated Dockerfile comment]
2018-01-31 10:53:13 -08:00
Eeshan Garg c158869096 Rename tools/lib/api_tests.py to zerver/lib/api_test_helpers.py.
Now that the Markdown extension defined in
zerver/lib/bugdown/api_generate_examples depended on code in the
tools/lib/* directory, it caused the production tests to fail since
the tools/ directory wouldn't exist in a production environment.
2018-01-31 07:30:54 -05:00
Eeshan Garg a1a69a0ac2 api/render-message: Make code examples and fixtures testable.
This commit uses the Markdown extension defined in
zerver/lib/bugdown/api_generate_example to generate the example
fixture and code example, so that both are tested in
tools/lib/api_tests.
2018-01-31 07:30:54 -05:00
Greg Price 8147897ac5 test-run-dev: Clean up some basic Python style. 2018-01-30 16:08:43 -08:00
Greg Price ff8e588340 test-run-dev: Delete commented-out code.
We don't disable code by commenting it out -- that leaves a mess.
We delete it.  Remembering what the code was is what source control
is for.

This fixes cd849bc3f "test-run-dev: Disable Nagios check."
from a few weeks ago.
2018-01-30 15:57:58 -08:00
Brock Whittaker 3b600d5591 /billing/: Rename "payment.html" => "billing.html".
This matches the URL path /billing/ to the filename "billing.html".
2018-01-29 17:24:28 -08:00
Aditya Bansal 0fcf0c5052 thumbor: Add thumbor on port 9995 in development.
For now, this does nothing in a production environment, but it should
simplify the process of doing testing on the Thumbor implementation,
by integrating a lot of dependency management logic.
2018-01-29 13:10:29 -08:00
Umair Khan 8700e8aeaf i18n: Retrieve mobile translations.
This commit also adds a tool to push translation sources to Transifex.
This tool makes sure that we don't push mobile source file. Mobile
source file is supposed to be handled from Zulip-Mobile repo.
2018-01-29 12:49:51 -08:00
Greg Price 9476a3a334 test-install: Give the host a direct view of the guest's /tmp/src/.
(This is a small fixup to the main change, which was accidentally
included in a previous commit:
  08bbd7e61 "settings: Slightly simplify EMAIL_BACKEND logic."
Oops.  See there for most of the changes described here.)

The installer works out of a release-tarball tree.  We typically want
to share this tree between successive test-install runs (with an rsync
or similar command to update source files of interest) because
rebuilding a release tree from scratch is slow.  But the installer
will munge the tree; so instead of directly bind-mounting the tree
into the container, we need to give it an overlay over the tree, as a
sandbox to play in.

Previously we used lxc-copy's `-m overlay=...` feature to do this,
mounting an overlay in the container.  But then sometimes in
development we want to reach in and edit some code in the tree,
e.g. before rerunning the installer after something failed.  Reaching
inside the container for this is a pain (`ssh` would add latency, and
I haven't installed sshd in the containers; and getting rsync to work
with `lxc-attach` was beyond what I could figure out in a few minutes
of fiddling); and editing the base tree often doesn't work.

So, create the overlay with our own `mount -t overlay`, and have
`lxc-copy` just bind-mount that in.  Now the host has direct access to
the same overlay which the guest is working from.

Also this makes it past time to help the user out in finding the fresh
names we've created: first the container, now this shared tree.  Print
those at the end, rather than make the user scroll to the top and find
the right `set -x` line to copy-paste from.
2018-01-29 10:27:11 -08:00
Aditya Bansal c714235922 keyboard_shortcuts.html: Clean up to use 4 space indentation. 2018-01-27 16:06:36 -08:00
Aditya Bansal fbd0e190d0 zerver/navbar.html: Clean up to use 4 space indentation. 2018-01-27 15:26:51 -08:00
Aditya Bansal 52c40668bc compose.html: Clean up to use 4 space indentation. 2018-01-27 15:25:40 -08:00
Tim Abbott 87beacd015 check-templates: Update comments on linter. 2018-01-27 15:17:28 -08:00
Aditya Bansal bc7614645d zerver/login.html: Clean up to use 4 space and valid indentation.
This is the last commit in the series of commits for completing the
project of cleaning up our html templates to have 4 space and
valid indentation.

Fixes: #1236.
2018-01-27 15:12:23 -08:00