Commit Graph

4160 Commits

Author SHA1 Message Date
Tim Abbott 97b256d1f0 css: Extract rendered_markdown.scss.
This moves our main CSS for rendered Zulip message content into an
external file, which may be reusable but in any case should make it
easier to find this content.
2019-07-31 12:08:17 -07:00
Wyatt Hoodes a109508e34 typing: Remove now-unnecessary conditional import.
As a result of dropping support for trusty, we can remove our old
pattern of putting `if False` before importing the typing module,
which was essential for Python 3.4 support, but not required and maybe
harmful on newer versions.

cron_file_helper
check_rabbitmq_consumers
hash_reqs
check_zephyr_mirror
check_personal_zephyr_mirrors
check_cron_file
zulip_tools
check_postgres_replication_lag
api_test_helpers
purge-old-deployments
setup_venv
node_cache
clean_venv_cache
clean_node_cache
clean_emoji_cache
pg_backup_and_purge
restore-backup
generate_secrets
zulip-ec2-configure-interfaces
diagnose
check_user_zephyr_mirror_liveness
2019-07-29 15:18:22 -07:00
Aman 2183a74040 zulint: Use zulint from the extracted repository.
zulint will be added as a "third-party" dependency in zulip from now
on.  See the new project at https://github.com/zulip/zulint for more
details.
2019-07-26 11:35:43 -07:00
Wyatt Hoodes a920f0396d webpack: Comment why NoReturn can't import normally. 2019-07-25 17:48:33 -07:00
Wyatt Hoodes 927feb615e provision: Comment why NoReturn can't import normally. 2019-07-25 17:48:20 -07:00
Wyatt Hoodes 3060bb2208 requirements: Upgrade to mypy 0.720.
We also add the option of the newly added `--warn_unreachable`
flag.
2019-07-25 17:41:10 -07:00
Anders Kaseorg a32663dc85 lint: Do not forbid ES6 const.
Our pipeline was upgraded in commit
f54a63e2f9 (#12838).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-25 16:12:48 -07:00
Anders Kaseorg 6d5a20ac62 requirements: Remove django-pipeline.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-24 17:40:31 -07:00
Tim Abbott c2bdccbaa8 provision: Use a shorter threshold for clean-unused-caches.
We've had a few reports of folks running out of disk when we're making
rapid changes to dependencies, and this should help prevent that.
2019-07-24 13:35:00 -07:00
Anders Kaseorg 43c9a26afb setup_venvs: Use system python3.
Otherwise python3 will be perpetually copied from virtualenv to
virtualenv and will never receive updates from the system.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-23 16:33:04 -07:00
Anders Kaseorg 82828bdba4 HTML validation: Remove invalid <button href> attribute.
For .start-button, Bootstrap carousel already supports <button
data-target> as a valid alternative to <button href>.  For
.call-to-action, the margin is decreased to exactly offset the lack of
margin collapsing with display: inline-block.  There should be no
visual change.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-23 16:16:22 -07:00
Eeshan Garg 8e10ab282a webhooks: Log unexpected webhook events separately.
This change serves to declutter webhook-errors.log, which is
filled with too many UnexpectedWebhookEventType exceptions.

Keeping UnexpectedWebhookEventType in zerver/lib/webhooks/common.py
led to a cyclic import when we tried to import the exception in
zerver/decorators.py, so this commit also moves this exception to
another appropriate module. Note that our webhooks still import
this exception via zerver/lib/webhooks/common.py.
2019-07-22 18:20:53 -07:00
Anders Kaseorg f54a63e2f9 webpack: Transpile JS code with Babel.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-22 17:55:32 -07:00
Anders Kaseorg ecfb7c6a7f lint: Add TypeScript compiler as a linter.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-22 17:55:18 -07:00
Anders Kaseorg 41209dd32e build-release-tarball: Run with zulip-py-venv symlink present.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-21 18:43:52 -07:00
Rishi Gupta 7d8d0b2284 settings: Update upgrade text and styling.
When we add Plus, the first sentence should change to "Available on Zulip
Standard and Plus".

I copied the styling of .tip out of expediency, but it's also possible that
long term we'll want only 1 tip-like box styling.

The hover styling is a bit random, but I tried to copy other hover styles I
found in settings.scss.

Note that this renames .upgrade_realm_plan_type_suggestion to .upgrade-tip.
2019-07-21 14:32:36 -07:00
Wyatt Hoodes e331a758c3 python: Migrate open statements to use with.
This is low priority, but it's nice to be consistently using the best
practice pattern.

Fixes: #12419.
2019-07-20 15:48:52 -07:00
Tim Abbott c931e76cf2 docs: Rewrite docs on writing API documentation.
This had gotten badly out of date, since it wasn't updated when we did
the big migration to the OpenAPI documentation system.

Fixes part of #12571.
2019-07-19 18:00:01 -07:00
Anders Kaseorg e80b57a18a provision_inner: Remove unused lsb_release run.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 16:36:11 -07:00
Anders Kaseorg c5a4b0501b provision: Start a new Python process after activating the virtualenv.
Mismatching imports from outside and inside the virtualenv in the same
process was causing segfaults after apparently benign changes to the
script!

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 12:16:55 -07:00
Anders Kaseorg a7eb3faf96 storage: Move the staticfiles.json hack to ZulipStorage.
There’s no reason to monkey-patch something that we were already
subclassing.

Removing the PRODUCTION conditional causes us to generate
staticfiles.json in the right place to begin with so we don’t need to
move it later.  It also allows Django to find staticfiles.json if
running the dev server with PIPELINE_ENABLED.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:47:51 -07:00
Anders Kaseorg 695b79c5d8 generate_zulip_bots_static_files: Write to static, not STATIC_ROOT.
Otherwise the files aren’t processed by collectstatic and don’t end up
in the staticfiles.json manifest.

Signed-off-by: Anders Kaseorg <andersk@zulipchat.com>
2019-07-19 11:33:08 -07:00
Anders Kaseorg 5f712b960f update-authors-json: Write to static, not STATIC_ROOT.
Otherwise the file isn’t processed by collectstatic and doesn’t end up
in the staticfiles.json manifest.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:33:08 -07:00
Anders Kaseorg 1d05a71b5d install, provision: Treat all nonzero exit codes as failure, not just 1.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:22:11 -07:00
Wyatt Hoodes 89df6e9425 check-templates: Add fix option to tools/check_templates.
`valid_indent_html` allows for replacing the incorrectly
indented file with the correct pretty-printed version
if `--fix` is passed to `tools/lint`.

Fixes #12641.
2019-07-19 11:12:11 -07:00
Anders Kaseorg ab89f40a66 generate-custom-icon-webfont: Replace with webpack webfonts-loader.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-18 12:00:00 -07:00
Anders Kaseorg 586aaddcc0 documentation_crawler: Stop treating all portico URLs as external.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-18 11:56:15 -07:00
Wyatt Hoodes 1418b8b05f test_server: Set LOCAL_UPLOADS_DIR in set_up_django.
For the Casper and API tests, we need to allow setting
LOCAL_UPLOADS_DIR, which is normally managed within test_runner.py.
2019-07-17 16:14:20 -07:00
Anders Kaseorg 54304b4008 webpack: Remove partial from handlebars-loader knownHelpers.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-17 16:07:17 -07:00
Anders Kaseorg b1aa304c4a templates: Suppress Handlebars automatic partial indentation.
Fixes #12795.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-17 16:07:17 -07:00
Sumanth V Rao 2c9f5e3980 tools: Add tool to update API field of local zuliprc file.
This tool can be used to update the API field of local
zuliprc files for dummy users of development server
(iago, prospero, etc) with the correct API key from database.

This tool can be run after provisioning (or similar tools) which change
the API keys in the database.
2019-07-17 16:00:21 -07:00
Tim Abbott 7c71e987ac openapi: Fix endpoints incorrectly tagged as documentation_pending. 2019-07-15 13:39:29 -07:00
Amal Rajan 09049f1678 provision: Add provision support for Ubuntu 19.04
This commit adds support for Ubuntu 19.04 in the development
environment.
2019-07-13 16:44:09 -07:00
Anders Kaseorg aa36e2f5e1 check-templates: Exclude static/icons/fonts/template.hbs.
This file was unchecked until the .handlebars ↦ .hbs rename, so this
is the easiest way to get tests passing again.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 22:52:41 -07:00
Anders Kaseorg 0c565f50be templates: Use upstream Handlebars partials syntax.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg db0b33842c templates: Replace templates.render with require calls.
This removes an unnecessary layer of indirection and allows webpack to
catch filename mistakes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg 3c3471b720 templates: Rename *.handlebars ↦ *.hbs and - ↦ _.
Tweaked by tabbott to avoid accidentally disabling the linter for
handlebars templates.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:03 -07:00
Wyatt Hoodes 0e3fddbc6e test_fixtures: Add logic for removing stale test directories.
Similarly to how stale database removal is handled, we add a check for
stale test run directories at the end of the `test-backend` script.
2019-07-12 17:42:18 -07:00
Wyatt Hoodes 16abd3d1e7 test-backend: Remove long-term clean up code.
With the refactored file structure approach, the clean up block at the
end of `test-backend` is no longer necessary.
2019-07-12 17:42:18 -07:00
Rishi Gupta 0f3c2748dd notification bot: Update stream announcement message. 2019-07-11 15:22:36 -07:00
Rishi Gupta 7de7b6872b notification bot: Modify initial stream creation message.
Discussion at
https://chat.zulip.org/#narrow/stream/137-feedback/topic/hello.20topic

The "by @**X**" part in the message content is in a later commit.
2019-07-11 14:16:56 -07:00
Anders Kaseorg 218c60ae86 third: Upgrade spectrum-colorpicker to 1.8.0 from NPM.
We had several patches to spectrum, but the only essential one 
(0ea770fc18) had already been fixed upstream,
and another was just handling jQuery deprecation warnings for not yet removed features.

See #12749 for details.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-10 10:07:34 -07:00
Wyatt Hoodes 886932fd23 provision: Update error output on failed `yarn install` reattempt. 2019-07-08 12:13:34 -07:00
Vishnu Ks 1f250a3fd2 casper: Add support for exporting results as XUnit XML file. 2019-07-07 22:30:01 -07:00
Vishnu Ks 4f65b50c44 requirements: Upgrade django-otp from 0.5.2 to 0.6.0.
This indirect dependency was upgraded separately since
it contained a migration with autogenerated name.
2019-07-07 22:28:54 -07:00
Anders Kaseorg 9f08513acb bundles: Rename commons.js to common.js.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 15:03:22 -07:00
Aman Agrawal 05aa3bcbc1 lint: Make `--fix` a default arg for zulint.
Since the check for `--fix` is in `zulint/command.py`,
we should keep it as an arg for zulint.
2019-07-03 14:57:31 -07:00
Aman Agrawal aa5a1cb509 lint: Make zulint independent of extra lint options in `./tools/lint`.
Check `options.full` in `pyflakes.py` instead of `linters.py` to
make zulint independent of `./tools/lint`.
2019-07-03 14:57:31 -07:00
Anders Kaseorg 1868e96850 tools/webpack: Don’t import Django.
It added about 150ms to the startup time of running webpack, for no
benefit.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 14:52:11 -07:00
Wyatt Hoodes 40227171e6 provision: Improve error messaging when attempting to use Ubuntu Trusty.
As part of dropping support, we add appropriate error messaging when a
user attempts to provision while using trusty.  If the user is running
in Vagrant we append information on how to proceed.
2019-07-03 14:32:25 -07:00
Anders Kaseorg 3ded6f53c3 storage: Transfer responsibility for hash-naming webpack files to webpack.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 14:16:17 -07:00
Anders Kaseorg b0a30aa01a storage: Remove RemoveUnminifiedFilesMixin.
This can be done much more easily with the right options to
collectstatic.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 14:16:07 -07:00
Anders Kaseorg 0cd6c75ab1 update-prod-static: Remove unused pre-webpack files.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 13:58:21 -07:00
Anders Kaseorg 079ddae4c8 minify-js: Remove; everything has been migrated to Webpack.
min/sockjs-0.3.4.min.js is not used.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 13:58:21 -07:00
Anders Kaseorg 5f3d36ae87 tex: Bundle KaTeX CLI using Webpack.
We don’t need a hacked copy anymore.  We run the installed version out
of node_modules in development, and a Webpack-bundled version of that
in production.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:50:03 -07:00
Anders Kaseorg 9f074fc071 webpack: Fix production setting of publicPath.
This is needed by webpack asynchronous require.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:49:38 -07:00
Anders Kaseorg 05663aa3fa webpack: Rectify "common" bundle with bundles/commons.js.
Moving bootstrap-typeahead from bundles/commons.js to bundles/app.js
and csrf.js from bundles/app.js to bundles/commons.js makes
bundles/commons.js equivalent to the "common" bundle, so we can
replace the latter with the former.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:38:26 -07:00
Anders Kaseorg 1b94733953 webpack: Remove resolve.modules override.
The minimal syntactic sugar it might provide isn’t worth the
unexpected side effects (including side effects on third party
modules).

For now, we allow zrequire to emulate the previous syntax in the Node
test suite, even though stealing part of the NPM namespace is
confusing.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:38:26 -07:00
Thomas Ip 8c199fd44c webpack: Use handlebars-loader to handle frontend templates.
And remove the compile-handlebars-templates system.
2019-07-02 16:23:29 -07:00
Anders Kaseorg fc3305b623 webpack: Trim resolve.extensions.
We don’t have any .tsx files, and nobody expects to be able to omit
the extension when importing .json, .scss, or .css files.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:09:29 -07:00
Anders Kaseorg 7b2a605978 provision: Sort system dependencies deterministically.
set iteration order is randomized in Python ≥ 3.3.  That might or
might not have had the potential for causing rare probabilistic bugs,
but if nothing else, it made build logs harder to compare.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 15:04:44 -07:00
Anders Kaseorg bbb56df6b0 i18n: Move static/locale back to locale.
As of commit cff40c557b (#9300), these
files are no longer served directly to the browser.  Disentangle them
from the static asset pipeline so we can refactor it without worrying
about them.

This has the side effect of eliminating the accidental duplication of
translation data via hash-naming in our release tarballs.

This reverts commit b546391f0b (#1148).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 14:57:55 -07:00
Anders Kaseorg a1cf77ca54 test-help-documentation: Bind vnu.jar to 127.0.0.1.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-28 18:18:29 -07:00
Anders Kaseorg 2d50dcf7cc test-help-documentation: Validate HTML with vnu.jar.
The VNU_IGNORE whitelist lets in some crazy-invalid preexisting HTML,
but hopefully this will stop the problem from getting much larger.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-27 14:53:21 -07:00
Anders Kaseorg f407a12ba2 test-documentation: Validate HTML with vnu.jar.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-27 14:53:21 -07:00
Aman Agrawal 7a3e8c4868 zulint: Add option to list groups using `--list-groups`. 2019-06-27 14:42:33 -07:00
Aman Agrawal 4424867adc zulint: Show errors in `--verbose` in a more readable format. 2019-06-27 14:42:33 -07:00
Aman Agrawal e275457387 zulint: Avoid changing the color of next line by using ENDC.
ENDC was not being applied due to missing `{}` in the end.
2019-06-27 14:42:33 -07:00
Anders Kaseorg 23cd064c86 webpack: Elide node_modules when importing JS modules.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 16:49:32 -07:00
Anders Kaseorg caecd1c2ad install: Disable installation and provisioning on Ubuntu 14.04 Trusty.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 15:58:53 -07:00
Anders Kaseorg bfa82a70c8 ci: Remove Ubuntu 14.04 (trusty) from CI.
Tweaked by tabbott to improve our CI docs discussion of these.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 11:01:13 -07:00
Anders Kaseorg b4204e7cfd Remove legacy (pre-Vagrant) Docker development environment setup.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 10:58:02 -07:00
Anders Kaseorg bac63f5a3e documentation_crawler: Reimplement crawl_with_status so it wraps crawl.
This way we inherit more of the upstream command’s behavior.
Importantly, this means we pass everything in `opts.spargs` to the
spider, not just `opts.spargs.skip_external`.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-24 23:20:17 -07:00
Anders Kaseorg 0f30c8c4ba dev-vagrant-docker: Upgrade docker-systemctl-replacement.
This fixes the 45 second boot delay bug.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-24 22:06:32 -07:00
Thomas Ip 80f972d42c webpack: Use correct types in config.
Cache-loader is used as an item in the use member so the correct
type should be RuleSetUseItem not RuleSetRule.
See: DefinitelyTyped/DefinitelyTyped/webpack/index.d.ts#L498
2019-06-24 17:58:51 -07:00
Thomas Ip 637b2b784a dependencies: Upgrade mini-css-extract-plugin to 0.6.0 and use HMR.
Given mini-css-extract-plugin can now do hot module replacement,
this commit also removed css-hot-loader. Not upgrading to 0.7.0
as that cause webpack to crash.
2019-06-24 17:58:51 -07:00
Yashashvi Dave 5c88475378 static/js/stream_data: Deduplicate notification setting updates.
Commit also add `stream_data.js` to full test covered
enforced files for `test-js-with-node` tests.
2019-06-24 14:46:45 -07:00
Thomas Ip a2de8ab44f lint: Add --fix option to ./tools/lint.
Linters with the fix option are stylelint, eslint and puppet-lint.
2019-06-24 14:13:04 -07:00
Wyatt Hoodes 6670b15234 provision: Clean up `var/` for the move to `var/<uuid>/test-backend`. 2019-06-24 12:42:00 -07:00
Wyatt Hoodes b1fe1ef42b test-backend: Clean up leak data import files after test-suite run.
This is a simple, non-intrusive way of removing the bulk of the
clutter from `var/<uuid>/test-backend` after running `test-backend`.

Ideally, we'll replace this logic with proper tearDown methods.
2019-06-24 12:39:56 -07:00
Aman Agrawal 8fc7c89874 zulint: Add `--verbose` arg to print good/bad lines in error.
This argument probably still needs some iteration, but it's already
useful.
2019-06-23 22:24:46 -07:00
Aman Agrawal 8eee19a160 lint: Add descriptions for all the linters. 2019-06-23 22:23:15 -07:00
Aman Agrawal 6d42b2392b zulint: Print description of linter with it in `--list`. 2019-06-23 22:23:15 -07:00
Aman Agrawal e236d39ef1 zulint: Add option to specify description of linter functions.
Extract description of function from __doc__ if used as wrapper.
2019-06-23 22:23:15 -07:00
Aman Agrawal e9ff9e34b3 lint: Use --groups to specify specific groups to run.
This helps generalize the use of groups inside zulint.
Introduce list_files to return `by_lang` files dict.
Add feature to create custom groups.
Make custom groups for backend and frontend files.
2019-06-23 22:23:15 -07:00
Aman Agrawal b5b3d9bd5f zulint: Refactor duplicate error printing code into print_error. 2019-06-23 21:59:00 -07:00
Aman Agrawal 7b7e887386 lint: Pass args to LinterConfig when initializing it.
This makes it easy to use args inside LinterConfig.
2019-06-23 21:55:02 -07:00
Aman Agrawal 0b12ec1417 lint: Initialize logger inside zulint.
This also helps move verbose_timing logic inside zulint.
2019-06-23 21:55:02 -07:00
Aman Agrawal b6d4438726 lint: Pass `args` instead of passing individual attributes to do_lint.
Pass the args to linter_config.do_lint.
2019-06-23 21:55:02 -07:00
Aman Agrawal b2b49089fd tools: Extract get_provisioning_status check logic.
Move get_provisioning_status check logic into
assert_provisioning_status_ok and use it instead of duplicating the
check code.
2019-06-23 21:55:02 -07:00
Tim Abbott f8b0727e5a test-backend: Check for venv before importing modules like ujson.
Previously, we were giving the wrong error message when running
test-backend not inside the Zulip venv if ujson wasn't installed on
the machine.
2019-06-23 16:12:21 -07:00
Aman Agrawal aa217efdbe lint: Add rule to ban rgb(a) in css/scss files, use hsl(a) instead. 2019-06-20 11:34:59 -07:00
Tim Abbott 4818011199 zulint: Add a basic README. 2019-06-18 12:25:46 -07:00
Aman Agrawal 6d040d330c lint: Add --list arg which lists all the registered linters. 2019-06-18 11:32:04 -07:00
Aman Agrawal 6457c1f7c6 lint: Generalize --skip arg and move logic to zulint/command. 2019-06-18 11:32:04 -07:00
Aman Agrawal 995b357cde lint: Add --only agr to run only the specified linters. 2019-06-18 11:32:04 -07:00
Aman Agrawal cf8653945f lint: Rename commit_messages to gitlint.
Use `gitlint` name to register gitlint linter since it is the commonly
used name for it.
2019-06-18 11:32:04 -07:00
Aman Agrawal 6b73926e93 lint: Add --skip arg to replace --no-gitlint/mypy.
Use --skip=gitlint,mypy instead of --no-gitlint/mypy.
2019-06-18 11:32:04 -07:00
Aman Agrawal 426a222b7e lint: Rename --verbose to --verbose-timing. 2019-06-18 11:32:04 -07:00
Aman Agrawal dcd46f1c11 lint: Move shebang_rules out of `RuleList` class to generalize it.
shebang_rules was moved to custom_check.py. Also add shebang_rules only
to those rules which need it.
2019-06-18 11:27:59 -07:00
Aman Agrawal db25c0c2ca lint: Combine functions in custom_rules into RuleList class.
This makes linting rules in zulint more general. Make necessary
changes in tools/lint and tools/custom_check.py to run with the new
RuleList class.

Modify tests for `RuleList` class. Tests only include minor changes to
test with the new class.
2019-06-18 11:27:50 -07:00
Tim Abbott 9760c8cfc9 tools: Fix postgres-init-*-db cache handling.
Previously, it didn't properly update the stamp files that determine
our caching behavior, so if one ran test-backend afterwards, nothing
would happen.

A secondary issue that this commit does not fix is that provision will
end up rerunning the whole thing.
2019-06-17 16:24:13 -07:00
Tim Abbott efe5e53118 docs: Update list of links to not check in tests. 2019-06-17 12:24:22 -07:00
Thomas Ip 0253ef12c6 popovers: Encode brackets in URI to avoid conflict with markdown links.
This fixes an issue where one could end up with a `(` in the markdown
syntax for a link after copy-pasting this, which doesn't work in
markdown.

Fixes #12579.
2019-06-16 15:18:33 -07:00
Vishnu Ks fdcae3b0b7 provision: Add cd /srv/zulip to .bashrc only if /srv/zulip exists. 2019-06-14 15:49:56 -07:00
Vishnu Ks d703e3638e droplets: Update snapshot id.
The new snapshot has zulip/zulip commit till
ac0d60f577. I have also included
the custom motd from b594708915
in base image.
2019-06-14 15:49:56 -07:00
Wyatt Hoodes 6b033c7909 test-backend: Add steps to deal with potential database leaks.
A function was written in `test_fixtures.py` to drop a test database
template if the corresponding database id doesn't belong to a file.
Alongside this fact, every file that is written is removed after 60
minutes.  Meaning any potential database template can never exist
longer than one hour.

This follow-up work was added to deal with the potential race
conditions when running `test-backend`.  Ensuring that all templates
are properly dealt with.

Essentially rewritten by tabbott for cleanliness.

Fixes the remainder of #12426.
2019-06-14 15:23:25 -07:00
Vishnu Ks 44d16d76c1 activity: Extract most js from realm_summary_table to activity.js. 2019-06-14 10:20:59 -07:00
Vishnu Ks 14e582fb59 support: Add functionality to copy admin emails.
Also renamed a bunch of functions in test_views for better
readability.
2019-06-14 10:19:50 -07:00
Puneeth Chaganti 6f4414e3f0 version: Fail gracefully if git describe cannot get version.
When running the `./tools/cache-zulip-git-version` script on Travis, the script
fails because Travis gets a shallow clone of the repository, and not a full
clone. This commit changes the script to fail gracefully, if we are unable to
get the version information using `git describe`.

When the command fails, it still writes an empty `zulip-git-version` and that
has not been changed to keep creation of the release tarball simple, and
avoiding a check for whether the file has any content. The code that sets
`ZULIP_VERSION` checks whether the contents of the `zulip-git-version` file are
empty, before setting `ZULIP_VERSION`. So, the version should never be set to an
empty string.
2019-06-14 13:47:27 +05:30
Tim Abbott 05e0e99b6e install-aws-server: Use our settings.py if available. 2019-06-13 14:39:25 -07:00
Tim Abbott 2d1c2d4802 install-aws-server: Stop using ssh -t without a real terminal.
More modern Linux versions like Bionic will block this, and what we
actually want to do is just run the code in our <<EOF block via bash,
so we should do that explicitly.
2019-06-13 14:39:25 -07:00
Alexandra Ciobica fbfd5d577a integrations: Move the css for the integrations page into a separate file.
The integrations page had css in both `landing-page.scss` and
`portico.scss`.  With this commit, the styles are mostly unified into
a single separate file.
2019-06-12 17:42:22 -07:00
Vaibhav 9096affe77 css: Move stylesheets not included in the app to another directory.
This moves all the stylesheets like stats, billing etc. to another
directory called `static/styles/portico/`, matching the directory
structure of our JavaScript.
2019-06-12 16:59:52 -07:00
ppreethi 86840adda5 version: Show number of commits and commit sha in ZULIP_VERSION.
We use `git describe --tags` to get information about the number of commit since
the last major version, and the sha of the current HEAD. This is added to the
ZULIP_VERSION when a deploy is done from `git`.

Modified heavily by punchagan to:
* to use git describe instead of `git log` and `wc`
* use a separate script to run the git describe command
* write the file with version info to var/ and remove it from the repo

Fixes #4685.
2019-06-07 13:39:12 -07:00
Vishnu Ks 8261c394b9 webpack: Include channel.js to email-log bundle.
During the refactoring in c27d927663
$.post was replaced with channel.post without including
channel.js in email-log webpack bundle.
2019-06-07 11:34:17 -07:00
Tim Abbott 9be3f30340 webpack: Don't import pyinotify in production context.
We only added the dependency for the development environment, but were
imported it unconditionally.
2019-06-04 18:10:58 -07:00
Tim Abbott 57a748ad63 node: Remove stream_ui_updates.js from modules expecting coverage.
This module is a UI module, which are harder to test, and in any case,
doesn't actually have any tests.
2019-06-04 16:45:54 -07:00
Priyank Patel 01f8c89294 webpack: Use cache-loader for various loaders.
Profiling shows that using cache-loader saves ~6-7 seconds of time take
by webpack-dev-server on subsequent runs. The overhaul this adds when
nothing is cached (when running first time) is around 1-2 seconds. We don't
use cache loader for ts-loader since webpack docs says it will slow it down
and file-loader since it just copies files over and caching it would just
was disk space.

This is the second merge of this commit.  It fixes the issue with the
previous one by placingn cache-loader after mini-css-loader because it
just extracts css and caching that will make file-loader not run which
in turn makes developement enviorment break.
2019-06-04 16:43:04 -07:00
Tim Abbott 5d0e144d62 Revert "webpack: Use cache-loader for various loaders."
This reverts commit eb53b5e8de.

This appeared to not have the right logic yet for handling branch
switches.

See https://chat.zulip.org/#narrow/stream/3-backend/topic/frontend.20hot.20reloading/near/749259 for details.
2019-06-03 22:11:47 -07:00
Priyank Patel eb53b5e8de webpack: Use cache-loader for various loaders.
Profiling shows that using cache-loader saves ~6-7 seconds of time take
by webpack-dev-server on subsequent runs. The overhaul this adds when
nothing is cached (when running first time) is around 1-2 seconds. We don't
use cache loader for ts-loader since webpack docs says it will slow it down
and file-loader since it just copies files over and caching it would just
was disk space.

Profiling data:

-------- Master ---------

~/zulip (master) $ tools/webpack --watch | ts -s '%.S' # master
03.995825 ℹ 「wds」: Project is running at http://127.0.0.1:9994/
03.996161 ℹ 「wds」: webpack output is served from /webpack/
03.996289 ℹ 「wds」: Content not from webpack is served from ...
19.284477 ℹ 「wdm」:
19.285371 ℹ 「wdm」: Compiled successfully.

-------- cache-loader ---------

~/zulip (cache-loader)$ tools/webpack --watch | ts -s '%.S'
04.107913 ℹ 「wds」: Project is running at http://127.0.0.1:9994/
04.108646 ℹ 「wds」: webpack output is served from /webpack/
04.109068 ℹ 「wds」: Content not from webpack is served from ...
12.633782 ℹ 「wdm」:
12.634083 ℹ 「wdm」: Compiled successfully.
2019-06-03 20:15:51 -07:00
Priyank Patel 4bb6c29d59 webpack: Use webpack's type defination.
Before we used to defined our own type Loader which was
partly incorrect because the use property can only
be string which is incorrect. We use the RuleSetRule type
provided by webpack instead.
2019-06-03 20:15:51 -07:00
Thomas Ip 537014ee47 dependencies: Upgrade typescript to 3.5.1. 2019-06-03 12:24:00 -07:00
Anders Kaseorg 5c734f7048 dev-vagrant-docker: Clear redis-server ExecStart before overriding it.
Real systemd requires this.  docker-systemctl-replacement currently
doesn’t but maybe it will later.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-01 16:55:56 -07:00
Anders Kaseorg faa4cf629a webpack: Close potential race condition leaking webpack-dev-server process.
This exchanges a race condition where webpack-dev-server might not be
stopped on a poorly timed KeyboardInterrupt for a less bad race
condition where we might get an UnboundLocalError.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-31 17:08:48 -07:00
Priyank Patel 2a864ebad3 webpack: Restart webpack-dev-server on config file changes.
This should make the run-dev.py user experience a lot nicer when
switching branches away from a branch that is at least as new as this
commit, since we won't need to manually restart run-dev.py to restart
webpack.

Fixes #9042.
2019-05-31 16:10:47 -07:00
YashRE42 5ff27c130a info-overlay: Fix "message formating" styling.
This reverts the temporary solution from 04d9d1d and introduces a
better, more permanent solution for the same bug.
2019-05-30 16:18:54 -07:00
Vaibhav 3462db3ecc node: Add test for updating bot_owner_id and enable user_events test suite.
Reverts the commit 4442509ec3
2019-05-30 15:43:18 -07:00
Anders Kaseorg d9f12cfb49 documentation_crawler: They’re called fragments, not permalinks.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-30 10:39:47 -07:00
Anders Kaseorg 4f28b88746 documentation_crawler: Set a 15 second download timeout.
By default it just hangs for as long as the server keeps the
connection open.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-30 10:39:47 -07:00
Anders Kaseorg c59747360a documentation_crawler: Set the default log level to warning.
This doesn’t seem to add any noise in the normal case, but if anything
shows up here we might want to see it.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-30 10:39:47 -07:00
Anders Kaseorg 2f547eaee4 documentation_crawler: Improve error handling.
* Remove the custom has_error logic in favor of checking whether any
  errors were logged, which gives us a much better chance at catching
  unanticipated exceptions.
* Use our error_callback for the initial requests of start_urls too.
* Clean up mypy types.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-30 10:39:47 -07:00
Anders Kaseorg 4c60e994c8 test-documentation: Fail on any nonzero exit code.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-30 10:39:47 -07:00
Anders Kaseorg 0ca9c260ae test-documentation: Use getopt; don’t silently ignore multiple arguments.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-30 10:39:47 -07:00
Aman Agrawal a1fadc28ba lint: Make `custom_rules.py` pass all lint checks.
The functions here weren't checked by all linters when they were in
 `custom_check.py`. Hence, modified to pass all lint checks.
2019-05-29 16:53:59 -07:00
Aman Agrawal 90c3578a88 lint: Move `custom_check_file` from `custom_check.py` to `custom_rules.py`. 2019-05-29 16:53:59 -07:00
Aman Agrawal cf038f0880 lint: Allow shebang_rules to use `include_only/exclude`.
Make shebang rules generic to be later extracted out of the
`custom_check_file` function.
2019-05-29 16:53:59 -07:00
Aman Agrawal 42efb60665 lint: Extract general funcs from custom_check.py to zulint/custom_rules.py. 2019-05-29 16:53:59 -07:00
Aman Agrawal 337060de01 lint: Move runnable code form `pyflakes.py` to `zulint/linters`.
Extract runnable code from `linter_lib/pyflakes.py` and move it to
`zulint/linters`. This keeps zulip specific pyflakes config separate
form zulint.
2019-05-29 16:53:59 -07:00
Anders Kaseorg 630f0fa761 Vagrantfile: Add UBUNTU_MIRROR configuration option.
The default http://archive.ubuntu.com/ubuntu/ is pretty slow in some
locations.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 16:19:39 -07:00
Anders Kaseorg 03cfe10da7 coveragerc: Omit blub.
The jedi package exec()s some code in the context of the fake module
blub, causing errors when generating the coverage report.  See
https://github.com/davidhalter/jedi/issues/1122.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 15:04:47 -07:00
Tim Abbott 095ff28277 test-backend: Avoid dropping/rebuiling zulip_test database.
The test-backend parallel test runner system doesn't actually use the
zulip_test database; instead, it creates its own databases off the
zulip_test_template database.

We were accidentally running `tools/generate_fixtures` even when there
are no changes, because this function is shared with the
tools/lib/test_server.py codebase, which needs us to do the work of
creating a test database for it off the zulip_test_template database.

Fixing this saves about 1.5s / 4s of the runtime of a single test.
2019-05-28 16:38:45 -07:00
Tim Abbott e0afdb675f test_fixtures: Rename and document update_test_databases_if_required.
The previous name was confusing because `generate_fixtures` isn't
actually a good name in the first place.
2019-05-28 16:38:45 -07:00
Tim Abbott 7b6f219663 test-backend: Flush stdout when starting testing.
This makes the output more properly ordered.
2019-05-28 16:38:45 -07:00
Tim Abbott 4442509ec3 node: Disable a test suite that's missing a line of coverage. 2019-05-28 16:36:30 -07:00
Anders Kaseorg f21d3be5fb dev-vagrant-docker: Copy the host user’s uid for the vagrant user.
This allows /srv/zulip to be writable from the guest even if the host
user’s uid is not 1000.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 19:21:51 -07:00
Anders Kaseorg 5231c8e575 dev-vagrant-docker: Unminimize the minimized Ubuntu cloud image.
This restores man pages and other documentation that have been
stripped from the default Ubuntu cloud image and installs
ubuntu-minimal and ubuntu-standard.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 12:53:11 -07:00
Rohitt Vashishtha eb44b7d69d linter: Do not lint scripts parsed with tail (like dev-motd). 2019-05-27 12:40:02 -07:00
Anders Kaseorg b594708915 Vagrantfile: Be nicer to /etc/update-motd.d.
Some of its information is helpful.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:33:06 -07:00
Anders Kaseorg 407564086d provision: Move apt-get update from retry handler to setup-apt-repo.
This avoids unnecessarily alarming error messages if the apt cache is
missing.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:07:38 -07:00
Anders Kaseorg ae524b677d ci: Move backend and production tests to Ubuntu 16.04 (xenial).
This is preparation for dropping support for Trusty in CI.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:07:15 -07:00
Anders Kaseorg 684ebc2a5e Vagrantfile: Support docker provider.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-24 15:43:23 -07:00
Hemanth V. Alluri c47521401f devtools: Move the integrations devtool JS to the portico folder. 2019-05-22 21:28:11 -07:00
Mateusz Mandera 58da8fd5f1 test-backend: Remove email_mirror from not_yet_fully_covered.
The preceding commits get the email mirror to 100% coverage, so we can
now remove zerver/lib/email_mirror.py from the list of not-fully-covered
files.
2019-05-20 19:35:32 -07:00
Tim Abbott 136a0da30b provision: Update unsupported architecture logging messages.
We don't use zulip-devel@ anymore, and also add a comment that it's
probably not a huge project to support e.g. arm.
2019-05-20 15:23:40 -07:00
Rishi Gupta fbc2239f73 keyboard shortcuts: Update description for S. 2019-05-20 15:10:11 -07:00
Kanishk Kakar 77ab9a0eb0 docs: Update desktop repo URL. 2019-05-20 11:01:11 -07:00
Anders Kaseorg 141088586b Completely replace perfect-scrollbar with SimpleBar.
perfect-scrollbar replaces both the appearance and the behavior of the
scrollbar, and its emulated behavior will never feel native on most
platforms.  SimpleBar customizes the appearance while preserving the
native behavior.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-17 12:06:51 -07:00
Vishnu Ks e7507a6eb1 tools: Avoid upgrading stripe automatically.
Upgrades to the stripe library can sometimes break semantics for our
billing system, and so we should make sure to use our documented 
testing process before doing them.
2019-05-17 11:36:35 -07:00
Vishnu Ks 56259c7acd lint: Catch exit(1) in management commands.
Using sys.exit(1) in a management command makes it impossible
to unit test the code in question.  The correct approach to
do the same thing in Django management commands is to raise
CommandError.
2019-05-17 11:34:00 -07:00
Tim Abbott 68255fe1df lint: Fix exclusion for development directory.
I forgot about running `test-tools` when making linter changes.
2019-05-15 13:40:33 -07:00
Tim Abbott 7f2fcea794 lint: Ban i18n usage under zerver/views/development. 2019-05-15 13:11:04 -07:00
Hemanth V. Alluri bae8295c52 devtools: Add integrations dev panel.
This commit adds a new developer tool: The "integrations dev panel"
which will serve as a replacement for the send_webhook_fixture_message
management command as a way to test integrations with much greater ease.
2019-05-15 13:07:44 -07:00
Vishnu Ks 06983298ba export: Add support for exporting realm with member consent.
This lets us handle directly in our tooling the user experience that
we document for exporting a realm with member consent (before, it
required unpleasant manual work).
2019-05-15 12:35:32 -07:00
Tim Abbott 5ec8f6e812 test_migrations: Disable migrations test.
Our migrations test suite only really works when no future migrations
conflict with it.
2019-05-12 22:06:17 -07:00
Tim Abbott 7c94d350d4 ci: Run test-backend with limited parallelism.
See the comment for details; this should fix timeouts we've been
seeing in CI.
2019-05-08 18:26:26 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
Wyatt Hoodes ececf0a209 provision.py: Add a conditional to remove '.eslintcache'.
It was discovered that the '.eslintcache' file was causing eslint to
throw a TypeError after a recent update/addition to the dependencies.
It makes sense to remove this file as part of the provisioning process
to avoid such exceptions.
2019-05-08 17:06:17 -07:00
Vishnu Ks 6c58603eaf support: Add support for scrubbing realm. 2019-05-06 20:12:54 -07:00
Mayank Madan 99414e2d96 provision: Add support for Debian 10 Buster. 2019-05-05 17:57:19 -07:00
Raymond Akornor e01d3be1ba tools: Remove check-urls linter.
This commit removes `tools/check-urls`. It was added as
a useful tool in preparation for the Django 1.10 migration.
Since we completed that migration, it is no longer needed.

Fixes #12180.
2019-05-05 17:47:41 -07:00
Wyatt Hoodes 21c67ea1c2 test-backend: Change --parallel default to use `cpu_count`.
The number of processes to run the backend tests is currently a
hardcoded value, this commit transistions the default to be based on the
number of logical CPUs available.
2019-05-02 10:48:20 -07:00
Yashashvi Dave 3649a9f15c subs: Add `stream_ui_updates.js` module for managing ui elements.
This commit adds `stream_ui_updates.js` module. This module
will includes functions which will update different ui elements
(i.e. subscription button, subscriber count).
2019-04-30 14:36:52 -07:00
Eeshan Garg a73e8109b7 webhooks: Remove the legacy GitHub integration.
The github-services model for how GitHub would send requests to this
legacy integration is no longer available since earlier in 2019.
Removing this integration also allows us to finally remove
authenticated_api_view, the legacy authentication model from 2013 that
had been used for this integration (and other features long since
upgraded).

A few functions that were used by the Beanstalk webhook are moved into
that webhook's implementation directly.
2019-04-27 15:13:44 -07:00
Vishnu Ks 16394e53a3 tools: Make dependency upgrader upgrade indirect dependencies separately. 2019-04-26 16:22:00 -07:00
Anders Kaseorg 122ad5be58 lint: Lint against calls to _() on computed strings.
These don't actually work from an i18n perspective.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:18 -07:00
Anders Kaseorg 44dd7c2d95 lint: Strengthen lint checks for string % non-tuple.
This is really a job for an AST parser rather than a pile of regexes;
among other issues, these will still miss violations that span
multiple lines.  But, you know, I tried.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:22:51 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Rafid Aslam d983611919 template_parser: Add support for jinja2 whitespace markers.
Fixes #11516.
2019-04-23 11:52:13 -07:00
Anders Kaseorg 5290519a62 scripts: Always use ON_ERROR_STOP=1 when running psql.
Also use psql -e (--echo-queries) in scripts that use ‘set -x’, so
errors can be traced to a specific query from the output.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:19 -07:00
Anders Kaseorg efd22fd2ae postgres-init-dev-db: Catch exception when recreating extant users.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:18 -07:00
Anders Kaseorg 25798c8605 linter_lib: Remove sudo exemption for terminate-psql-sessions.
terminate-psql-sessions no longer uses sudo as of #11771.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:17 -07:00
Anders Kaseorg 12e2e0d658 tools/webpack: Run webpack in the foreground.
`tools/run-dev.py` already backgrounds `tools/webpack` (and deals with
cleaning it up on exit), so there’s no need for `tools/webpack` to
also background the actual `webpack` process.  But when running
`tools/webpack` by itself, it’s annoying to clean up the backgrounded
process manually.

Run `webpack` in the foreground, using `os.execvp` so we don’t waste
memory on an intermediate wrapper process.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 13:59:11 -07:00
Anders Kaseorg dd3ccbc897 webpack: Fix node_modules path resolution.
Webpack applies special logic to relative paths provided in
`resolve.modules`, and this logic is expected to be used for
`node_modules`.  One case where this is important is when
`node_modules/foo` wants to import a different version of package
`bar` than the one at `node_modules/bar`, and so yarn gives it its own
copy at `node_modules/foo/node_modules/bar`.

It would probably be better to avoid screwing with `resolve.modules`
at all, but this at least brings us one step closer to the default of
just `["node_modules"]`.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 13:49:23 -07:00
Anders Kaseorg 9fc0aa7ffa scripts: Fix exec invocation for in-process virtualenv activation.
activate_this.py has always documented that it should be exec()ed with
locals = globals, and in virtualenv 16.0.0 it raises a NameError
otherwise.

As a simplified demonstration of the weird things that can go wrong
when locals ≠ globals:

    >>> exec('a = 1; print([a])', {}, {})
    [1]
    >>> exec('a = 1; print([a for b in [1]])', {}, {})
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1, in <module>
      File "<string>", line 1, in <listcomp>
    NameError: name 'a' is not defined
    >>> exec('a = 1; print([a for b in [1]])', {})
    [1]

Top-level assignments go into locals, but from inside a new scope like
a list comprehension, they’re read out of globals, which doesn’t work.

Fixes #12030.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-16 17:42:44 -07:00
Tim Abbott 31f6ace084 lint: Fix typo in eslint command line. 2019-04-13 22:45:07 -07:00
Thomas Ip 90478fc4b2 typescript: Fix violations of new typescript rules. 2019-04-13 11:42:48 -07:00
Thomas Ip 02cb85a8a9 typescript: Use ESLint instead of TSLint. 2019-04-13 11:42:47 -07:00
Thomas Ip ea9a74fe24 typescript: Fix violations of existing eslint rules. 2019-04-13 11:38:19 -07:00
Anders Kaseorg 53879c4673 scripts: Rename DEFAULT_USER to POSTGRES_USER.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-12 17:27:23 -07:00
Marco Burstein ce7d2fde70 tools: Add TypeScript to the dependency visualizer.
The dependency visualizer currently only supports JavaScript files,
such as in the `get_js_edges` function, where only the ".js" extension
is supported. Update the visualizer to support ".ts" files as well and
to output modules without their extensions.
2019-04-12 11:14:42 -07:00
Marco Burstein 57ffc3fe74 tests: Update JS Node tests to support TypeScript.
Currently, the `test-js-with-node` tests append ".js" to filenames
without an extension. Since Typescript is now also supported, it can
produce results such as "dict.ts.js". To remedy this, check for ".ts"
files as well.
2019-04-12 11:14:42 -07:00
sumanthvrao c5de6d0e18 spiders: Set error status for exceptions generated by spider.
Spider raises exceptions when errors like FileNotFound
are detected. However, these did not set error state
before exiting causing spider to fail silently.

This patch sets the status causing exceptions to exit with
non-zero exit status.
2019-04-08 17:04:29 -07:00
Abhinav Singh c27d927663 refactor: Remove inline javascript code in email_log.html.
All the inline javascript code present in email_log.html(which is
rendered when the user visits "/emails" in development mode) is
transferred to a new file: email_log.js in portico/ directory.

Fixes #11608.
2019-04-05 17:28:23 -07:00
Anders Kaseorg 11f18042fe run-dev-queue-processors: Remove
As of #367, `tools/run-dev-queue-processors` has evolved into nothing
more than an unnecessarily elaborate wrapper around `manage.py
process_queue --all`.  Remove it (mostly to make it marginally easier
to Tab-complete `tools/run-dev.py`, if I’m being honest).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-05 15:18:38 -07:00
Thomas Ip b8e1b8d6ff coverage: Bring dict.ts line coverage back to 100%.
The delete operator could throw a TypeError when attempting to
remove a non-configurable property, which is rare in practice since
they can only be created using `Object.defineProperty()` and
`Object.freeze()`. We also never uses the output of `del()` anyway.
2019-03-30 17:12:50 -07:00
Thomas Ip bec31c6791 coverage: Get Istanbul to recognise typescript modules. 2019-03-30 17:12:50 -07:00
Abhinav Singh d52e3b0d70 refactor: Remove inline javascript code in dev_login.html.
A new javascript file "dev-login.js" is created in static/js/portico/
and the inline javascipt code present in dev_login.html is transferred
to that file. An empty div element is added in dev_login.html with
unique data-page-id attribute to make it more easy to find in which
page we are, while working with the javascript code.
2019-03-25 15:14:24 -07:00
Thomas Ip a2872c107e typescript: Move TS files into JS directory.
This is just a code reorganization to avoid making it difficult to
find things as we migrate more file to TypeScript.
2019-03-25 12:11:37 -07:00
Rhythm Sharma e1212c04d1 emoji: Add a few useful aliases. 2019-03-17 13:38:31 -07:00
Tim Abbott 50dc317466 notifications: Rename notifications.py to email_notifications.py.
This library is entirely about email notifications specifically, and
this rename should help make the codebase more readable.
2019-03-15 11:02:17 -07:00
Steve Howell ba0418c989 stream_data: Remove untested useless code.
This code was actually dead, due to the .empty() check above.  Fixing
this lets us restore 100% coverage.
2019-03-15 09:27:41 -07:00
Raymond Akornor ea112828dc mypy: Use python3 type syntax in test-backend. 2019-03-15 09:22:30 -07:00
Raymond Akornor 193de819b8 test-backend: Refactor test-backend to be more pythonic. 2019-03-15 09:22:30 -07:00
Tim Abbott 76891f6a02 node: Disable a failing coverage check. 2019-03-14 22:22:06 -07:00
Tim Abbott cdf987fb14 push-translations: Run manage.py makemessages first.
This saves an extra step in pushing translations to transifex, with no
real negative side effects.
2019-03-13 14:11:27 -07:00
Vishnu Ks 8eeb8280b4 activity: Create interface for doing support operations.
This should grow into a tool that makes it much easier to do common
organization management tasks without using a manage.py shell.
2019-03-11 12:01:11 -07:00
Mayank Madan 264c2e9803 provision: Add provision support for Ubuntu 18.10.
Note that this is only support for Ubuntu 18.10 in the development
environment.
2019-03-09 16:01:46 -08:00
vinitS101 4523c8d7ee typo: Change "generage" to "generate". 2019-03-09 15:49:38 -08:00
Tim Abbott 0ab8cd9ca3 provision: Refactor handling of dependencies for building pgroonga.
This lets us leave build-pgroonga platform-agnostic.
2019-03-08 15:12:56 -08:00
Tim Abbott 5c20ee998c build-tsearch-extras: Add support for Debian/Ubuntu. 2019-03-08 15:12:56 -08:00
Tim Abbott 32d26c8aad provision: Clean up building search extensions from source.
This generalizes the provision logic for deciding whether to build our
tsearch_extras and pgroonga search extensions from source to support
Ubuntu cosmic as well (and evenutally, other future platforms).
2019-03-08 15:12:56 -08:00
Tim Abbott 8469c7774f scripts: Remove create-zulip-admin.
This was used only by ancient versions of the Docker project.
2019-03-04 15:33:51 -08:00
Tim Abbott 500b5cdaf1 tools: Fix missing comma in collectstatic configuration.
It appears that this code did the right thing despite being written
wrong, probably due to whatever `manage.py collectstatic` does in its
argument parsing.  But in any case, we should make the code read how
it's intended.
2019-03-04 08:31:12 -08:00
Rohitt Vashishtha 3d427d02cc scripts/zulip_tools: Use run_as_root instead of subprocess.check_call. 2019-03-01 11:21:16 -08:00
Rohitt Vashishtha ac48925977 scripts: Use run_as_root instead of run([sudo, ...]). 2019-03-01 11:21:16 -08:00
Wyatt Hoodes 5231b27dea popovers: Add a clear status option in user popover.
Accomplished by adding a function to clear the status message with
an empty string. The html is then updated to reflect changes without a
refresh.

Currently, it's a small hassle to clear a status message.  This option
makes things a bit easier.

Fixes #11630.
2019-02-26 14:40:17 -08:00
Anders Kaseorg 9faa009f66 lint: Prohibit unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-22 16:54:50 -08:00
Anders Kaseorg f1ec67c614 python: Mark intentionally unused imports with noop statements.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-22 16:54:47 -08:00
Wyatt Hoodes 2580965284 test-backend: Rename '--process' option to '--parallel'.
This change is being made to simply match up our naming conventions
with django's test framework.
2019-02-20 13:25:49 -08:00
Raymond Akornor 4dc7f5354d tests: Optimize test-backend performance.
This optimizes test-backend by skipping webhook
tests when run in default mode.

Tweaked by tabbott to extend the documentation and update the CI
commands.
2019-02-19 14:50:41 -08:00
Tim Abbott fe4a0d0fcd capitalization: Fix night logo lint errors. 2019-02-18 15:53:55 -08:00
Steve Howell e67cf30dfd private messages: Add user circles to top left.
This is mostly adding markup, calling some convenient
functions in buddy_data.js, and adjusting CSS.

To make the circles update dynamically, I mostly
orchestrate this though activity.js for now.  It's
possible we'll want to adjust that eventually to
happen through something like a `presence_events`
dispatcher, but that's essentially what
a good part of `activity.js` does now.
2019-02-18 14:22:37 -08:00
Anders Kaseorg dec8c7dba4 provision: Run generate_secrets earlier.
Commit 7d12e2019d (#10994) broke fresh
provisions by importing zproject.settings before we were done
modifying settings.  Fix it by moving the generate_secrets invocation
to the earliest reasonable place.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-08 10:13:18 -08:00
Tim Abbott a2373a63cd capitalization: Add GCM to list of proper nouns.
This fixes failures in master from the recent GCM merge.
2019-02-08 10:11:55 -08:00
Anders Kaseorg 8fdb62a6bc tools: Create DEPLOY_ROOT/zulip-py3-venv symlink in development too.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-07 17:12:33 -08:00
Vishnu Ks b62bd83083 invites: Add generate multiuse invite button. 2019-02-07 15:41:00 -08:00
Rishi Gupta 36472413e4 hotspots: Add hotspot for gear menu. 2019-02-05 12:19:21 -08:00
Vishnu Ks 4b1229c072 billing: Add node tests for upgrade.js. 2019-02-04 22:37:05 -08:00
Anders Kaseorg ee8ff4df66 tools: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:10:31 -08:00
Hemanth V. Alluri 73d26c8b28 streams: Render and store the stream description from the backend.
This commit does the following three things:
    1. Update stream model to accomodate rendered description.
    2. Render and save the stream rendered description on update.
    3. Render and save stream descriptions on creation.

Further, the stream's rendered description is also sent whenever the
stream's description is being sent.

This is preparatory work for eliminating the use of the
non-authoritative marked.js markdown parser for stream descriptions.
2019-02-01 22:24:18 -08:00
Anders Kaseorg 893e1475bb setup_virtualenv: Do not activate the virtualenv on creation.
Instead, manually activate it in the one place where this
functionality was used (tools/lib/provision.py).  This way we avoid
trying to activate the Python 2 thumbor virtualenv from Python 3.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-31 17:55:04 -08:00
Greg Price 6613f4c22c test-install/install: Print usage when run without arguments.
The `-z "$INSTALLER"` was intended to do this -- but we don't get that
far, because the `shift` fails.
2019-01-31 16:15:51 -08:00
Steve Howell c665f809f2 run-dev: Make address-in-use errors more obvious.
This seems to be a common enough pitfall to justify
a bit of extra handling.  Example output:

    $ ./tools/run-dev.py
    Clearing memcached ...
    Flushing memcached...
    OK
    Starting Zulip services on ports: web proxy: ...
    Note: only port 9991 is exposed to the host in a Vagrant environment.

    ERROR: You probably have another server running!!!

    Traceback (most recent call last):
      File "./tools/run-dev.py", line 421, in <module>
        app.listen(proxy_port, address=options.interface)
      File "/srv/zulip-py3-venv/lib/python3.5/...
        server.listen(port, address)
      File "/srv/zulip-py3-venv/lib/python3.5/...
        sockets = bind_sockets(port, address=address)
      File "/srv/zulip-py3-venv/lib/python3.5/...
        sock.bind(sockaddr)
    OSError: [Errno 98] Address already in use
    Terminated
2019-01-29 16:03:47 -08:00
Steve Howell 1228e541d7 dispatch test: Restore 100% line coverage (user_status).
We now have 100% coverage on server_events_dispatch.js.

The last piece were was covering "status_text" updates
in the "user_status" event.
2019-01-26 10:48:57 -08:00
Steve Howell bacf896228 poll widget: Clean up code and add edit controls.
NOTE: If you revert this commit, you want to revert
the immediately prior commit as well.  The history
is that Ishan made some improvements to the widget,
but there were some minor bugs.  I decided not
to squash the commits together so that the git
history is clear who did what.  (In particular, I
want questions about the JS code to come to me if
somebody does `git blame`.)

Anyway...

This is a fairly significant rewrite of the polling
widget, where I clean up the overall structure of
the code (including things from before the prior
fix) and try to polish the prior commit a bit as
well.

There are a few new features:

    * We tell "other" users to wait for the poll
      to start (if there's no question yet).
    * We tip the author to say "/poll foo" (as
      needed).
    * We add edit controls for the question.
    * We don't allow new choices until there's
      a question.
2019-01-22 10:27:39 -08:00
Harshit Bansal 2944075f70 lint: Respect `--force` flag while running mypy. 2019-01-17 10:34:43 -08:00
Anders Kaseorg 25d6107a4f docs: Avoid hardcoded /tmp paths in miscellaneous documentation.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:16:12 -08:00
Anders Kaseorg 79d888223f profiling: Use mkstemp for profile.data filename.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:08:48 -08:00
Anders Kaseorg f3c9b87c30 test-locked-requirements: Avoid hardcoded paths in /var/tmp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:05:51 -08:00
Anders Kaseorg b4e1403cf9 test-install: Avoid hardcoded paths in /tmp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:05:51 -08:00
Anders Kaseorg c17ddceeae test-emoji-name-scripts: Avoid hardcoded paths in /var/tmp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:05:50 -08:00
Anders Kaseorg df7b63cd5e tools: Remove check-css.
As of commit 0042694e24 (#10017), this
is unused.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:05:50 -08:00
Anders Kaseorg f099fd9e28 pretty_print: Replace hardcoded path in /var/tmp with a pipe.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:00:48 -08:00
Anders Kaseorg 6678a95b20 test-queue-worker-reload: Use a pipe rather than polling a log file.
This avoids unnecessary polling delays, quadratic string operations,
and hardcoded paths in /tmp.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:00:48 -08:00
Anders Kaseorg 19429c3ad7 test-run-dev: Use a pipe rather than polling a log file.
This avoids unnecessary polling delays, quadratic string operations,
and hardcoded paths in /tmp.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:00:48 -08:00
Tim Abbott e29533df0f emails: Fix duplicate <html> blocks in missed-message emails.
Apparently, the missed-message templates have a slightly different
structure from our other email templates, which triggered a latent,
subtle bug in inline-email-css's effort to remove duplicate <html>
blocks from emails that had been generated by premailer.  Fix this
bug, and add appropriate assertions to prevent similar issues in the
future.

Fixes #11249.
2019-01-11 14:23:30 -08:00
Anders Kaseorg 2ad1a27b28 create-production-venv: Remove Python 2 support.
We still create a Python 2 virtualenv for thumbor but that’s
separate (/srv/zulip-thumbor-venv from
scripts/lib/create-thumbor-venv).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-09 17:32:05 -08:00
Vishnu Ks c263674507 billing: Add node test for billing.js. 2019-01-08 10:47:35 -08:00
rht 15763f8545 provision: Include DISTRIB_FAMILY in parse_lsb_release output. 2019-01-07 18:52:09 -08:00
Vishnu Ks 13de959121 tools: Add script to automate python dependency upgrade. 2019-01-07 10:25:35 -08:00
Marco Burstein 9ddadd39f4 compose: Add support for using Zoom as the video chat provider.
This adds Zoom call properties to the `Realm` model, creates endpoints
for creating calls, adds a frontend and tests.

Fixes #10979.
2019-01-07 10:00:02 -08:00
Mateusz Mandera 36714342a3 lint: Allow usage of the word subject in test_email_mirror.py. 2019-01-05 15:59:19 -08:00
Tim Abbott 99721755e8 node: Temporarily remove composebox_typeahead from fully covered. 2019-01-05 15:31:30 -08:00
Tim Abbott 453b65bde3 provision: Clean up comments and unused variables.
This has some changes inspired by #8514.
2019-01-04 15:16:19 -08:00
rht 7e1520b01f provision: Ensure `python3` exists in PATH for CentOS. 2019-01-04 14:11:03 -08:00
Vishnu Ks d94b0da3c8 billing: Separate billing.js into multiple modules. 2019-01-04 23:52:00 +05:30
Vishnu Ks 81e6a022e6 emails: Inline CSS in missed_message.
As part of this change, we port into the .messages class the work in
4e8e7348da to change overflow-y to auto,
not scroll (skipping that would result in a regression).
2019-01-03 17:55:29 -08:00
Vishnu Ks 4e1c0589c2 emails: Refactor if condition from style tag in missed_message_email. 2019-01-03 17:55:29 -08:00
Tim Abbott 4df618f584 portico: Order jquery.validate before signup.js.
This is important, because the signup.js module uses jquery.validate.
2019-01-03 15:22:57 -08:00
Steve Howell d8d703af45 frontend: Add basic user_status module.
So far this processes page_params, but it's otherwise
an unused internal API.
2019-01-02 09:16:31 -08:00
Steve Howell 4d5ce0b285 lint: Exclude "subject" from JS files (except util.js).
We also exempt the frontend tests for now.
2019-01-01 20:52:07 -08:00
Steve Howell b4bc6b4445 emails: Fix subject -> email_subject in send_email.py. 2019-01-01 20:49:00 -08:00
Steve Howell 4da28efc90 emails: Fix subject -> email_subject in error_notify.py. 2019-01-01 20:48:59 -08:00
Steve Howell 16328732d4 refactor: Remove "subject" from stream_topic.py. 2019-01-01 20:48:56 -08:00
rht d3139266c8 setup_venv: Uninstall typing on Python >3.4.
Otherwise this causes an error
```
AttributeError: type object 'Callable' has no attribute '_abc_registry'
```
on 3.7. While the error is specific to 3.7, it is safer to uninstall
typing for all the versions that don't require a pip-provided typing
library.
2019-01-01 19:34:48 -08:00
rht c190959b58 provision: Move CentOS & Fedora venv deps construction to setup_venv.py.
They will be reused in production setup.
2019-01-01 19:34:46 -08:00
rht 17ffbdabe2 provision: Use virtualenv instead of python-virtualenv on Fedora.
The package name changed at some point.
2019-01-01 19:34:24 -08:00
rht 75134ef614 provision: Use generic python3 on Fedora dependencies.
It might be cleaner to add new variables in setup_venv.py, but we can
do that later.
2019-01-01 19:34:11 -08:00
Tim Abbott 5a8ad84381 test-documentation: Exclude the 500ing Citizen Code of Conduct.
We might want to eventually remove this, but I'm holding out hope that
it'll be fixed.
2018-12-29 15:30:19 -08:00
Vishnu Ks 6f5b1903c4 docs: Update code-reviewing docs to reflect CI changes. 2018-12-29 15:28:26 -08:00
Steve Howell 3fc8597119 popovers: Avoid passing message to actions popover template.
We instead get the specific fields from message
that we use.  This is particularly helpful
for subject -> topic migration; we no longer
have to account for "subject" fields in
client-side templates.
2018-12-29 14:19:18 -08:00
Sumanth V Rao ccd0badeed test-documentation: Simplify --skip-external-links code path.
The crawler used to be called directly for checking external links.
Now the scrapy command calls the crawl_with_status wrapper.
Crawl_with_status has been modified to pass the external parameter in
the previous commit, so we can now use this simpler approach.
2018-12-29 09:09:18 -08:00
Sumanth V Rao 776b981dde test-help-documentation: Skips checking external links.
This option causes test-help-documentation to verify internal links only.
This prevents documentation from flaking in CI due to links from
external websites having failed in the documentation.

Based on conversation:
https://chat.zulip.org/#narrow/stream/43-automated-testing/subject/quora.20404/near/674997
2018-12-29 20:15:30 +05:30
Rishi Gupta e7220fd71f billing: Do subscription management in-house instead of with Stripe Billing.
This is a major rewrite of the billing system. It moves subscription
information off of stripe Subscriptions and into a local CustomerPlan
table.

To keep this manageable, it leaves several things unimplemented
(downgrading, etc), and a variety of other TODOs in the code. There are also
some known regressions, e.g. error-handling on /upgrade is broken.
2018-12-22 13:39:30 -08:00
Rishi Gupta f8720fddd6 emails: Update text of notify_new_login. 2018-12-20 16:26:19 -08:00
Vishnu Ks f9e4d5d28e emails: Tag password_reset for translation. 2018-12-19 09:21:09 -08:00
Vishnu Ks 0960e654ab emails: Tag notify_new_login email for translation. 2018-12-19 09:21:09 -08:00
rht acbb174100 provision: Add RHEL 7 support. 2018-12-18 17:13:56 -08:00
rht 84c968ab36 provision: Include scripts/lib/setup-* and scripts/lib/build-* for sha1 sum check. 2018-12-18 16:56:27 -08:00
Tim Abbott dd946fff1f lint: Improve ui_report.error linter rule.
We now allow passing "" as the first argument.
2018-12-18 12:33:15 -08:00
rht a093d45064 provision: Check for pg_hba.conf existence for doing postgres initdb.
Apparently, an empty pg_datadir is initialized after postgresql-10 is
installed.
2018-12-18 12:07:45 -08:00
rht 3060ee9397 provision: Fix typo when checking that vendor is Fedora. 2018-12-18 12:07:03 -08:00
Anders Kaseorg 392175d6e8 Use #!/usr/bin/env for bash shebangs.
/bin/sh and /usr/bin/env are the only two binaries that NixOS provides
at a fixed path (outside a buildFHSUserEnv sandbox).

This discussion was split from #11004.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-17 17:21:08 -08:00
rht 5e12eb8d34 provision: Fix typo in list of services on RedHat. 2018-12-17 16:30:48 -08:00
rht 42cbf2071b provision: Skip RedHat postgresql initdb if already initialized.
We only need to initialize a postgres database once in the lifetime of
a system.
2018-12-17 16:29:50 -08:00
rht 295cbe7b9c provision: Add build-pgroonga and use it for Fedora. 2018-12-17 16:23:57 -08:00
rht 80ba105be5 provision: Abstract out SYSTEM_DEPENDENCIES. 2018-12-17 16:23:46 -08:00
rht b732fe819e provision: Add Fedora support. 2018-12-17 16:23:44 -08:00
Tim Abbott d469c14d11 docs: Advertise arbitrary org GitHub patterns for linkifiers.
Now that we support this, we should make it really obvious that we do.

Also, link to the Help Center article.
2018-12-17 12:58:52 -08:00
rht 2dc1dc8ec2 provision: Replace overwrite_symlink with `sudo ln -sf`.
overwrite_symlink doesn't work for a non-root user when accessing root
content.
2018-12-17 11:50:15 -08:00
Tim Abbott 2558f101af docs: Add documentation for `if False` mypy pattern in scripts.
This should help make it clear what's going on with these scripts.
2018-12-17 11:12:53 -08:00
Tim Abbott 209df75ffa lint: Enforce use of Python 2 style typing in zulint.
This is intended to be extracted as an external project that supports
Python 2+3.
2018-12-17 11:00:08 -08:00
Tim Abbott 35901cc955 lint: Fix a tooling test failure in last commit.
This is what I get for copy-pasting lint rule blocks.
2018-12-17 10:53:08 -08:00
Tim Abbott bce90a3340 lint: Add lint rule for scripts importing typing improperly.
This is a common bug that users might be tempated to introduce.

And also fix two instances of this bug that were present in our
codebase, including an important one in our upgrade code path.
2018-12-17 10:46:37 -08:00
Tim Abbott 6b69cc0b39 lint: Run mypy as part of main linter.
To support this, we add a pass_targets option to the main linter
library, because with current mypy, it's generally counterproductive
to pass the list of files in (can produce spurious errors; isn't
faster).
2018-12-16 21:52:53 -08:00
Tim Abbott 3003517430 docs: Move mypy documentation from contributing to testing.
The testing section is more appropriate, since it's fundamentally part
of our CI system.

While we're at it, fix the fact that we were linking to GitHub, not
ReadTheDocs, in the run-mypy output.
2018-12-16 21:52:53 -08:00
Tim Abbott 8637bdeb30 mypy: Provide a clear message when starting daemon.
Previously, when the mypy daemon was starting up, it was very unclear
from the linter context what was happening.
2018-12-16 21:52:53 -08:00
Tim Abbott f3c6d91e69 mypy: Switch default to daemon mode.
There isn't any real advantage to running the non-daemon mode at this
point.
2018-12-16 21:37:58 -08:00
Tim Abbott 7485cb2a50 widgets: Rename voting_widget to poll_widget.
This ensures greater consistency with our other widgets' naming
convention.
2018-12-16 19:46:48 -08:00
Tim Abbott adebe1bd4e js: Extract csrf.js and include in common bundle.
This should make it possible to use this AJAX setup code in logged-out
code as well, which is necessary to use blueslip from portico pages.
2018-12-16 16:18:42 -08:00
Tim Abbott ed9430f3a6 Revert "js: Extract csrf.js and include in common bundle."
This reverts commit dbffa66b6d.

This broke the logged-in app, due to some webpack issue.
2018-12-16 16:16:06 -08:00
Tim Abbott dbffa66b6d js: Extract csrf.js and include in common bundle.
This should make it possible to use this AJAX setup code in logged-out
code as well, which is necessary to use blueslip from portico pages.
2018-12-16 15:35:54 -08:00
Tim Abbott 9059c4d223 provision: Make apt/yum retry logic more coherent.
This changes a few things:
* Deplicates deps_to_install logic.
* Has a retry flag, under which we can guard the apt retry print statements.
* Makes the install_system_deps flow more parallel.
2018-12-16 13:41:12 -08:00
rht c6cde1cf5c provision: Rename install_apt_deps to a more generic install_system_deps. 2018-12-16 13:39:00 -08:00
rht 389c46b4e5 provision: Extract yum-specific part of install_apt_deps. 2018-12-16 13:39:00 -08:00
rht 2be3065249 provision.py: Abstract out postgres version for CentOS. 2018-12-16 13:26:28 -08:00
rht 43bedc0909 provision: Use vendored pg_hba.conf on CentOS. 2018-12-16 13:21:54 -08:00
Tim Abbott f2173bc2a7 check-capitalization: Add a few exclude rules for new strings. 2018-12-16 11:34:31 -08:00
rht 022e926d64 provision: Setup services for CentOS. 2018-12-16 10:14:51 -08:00
rht 9c776f4fcf provision: Provide tsearch stopwords path for CentOS. 2018-12-16 10:14:08 -08:00
rht 8b5a86b992 provision: Initialize Postgres db on CentOS. 2018-12-16 10:11:55 -08:00
rht 41a5f6a49d provision: Create symlink for pg_* on CentOS. 2018-12-16 10:11:54 -08:00
rht 10786375db provision: Do not use apt-specific retry when a failure happens. 2018-12-16 10:04:27 -08:00
rht 25a8dbef11 provision: Build tsearch_extras from source for CentOS support. 2018-12-16 10:04:27 -08:00
rht 448303b3f0 provision: Initialize setup-yum-repo. 2018-12-16 10:04:27 -08:00
rht 85b2627191 provision: Add experimental support for CentOS. 2018-12-16 10:04:27 -08:00
rht 79bcb4af47 provision: Remove unused comment. 2018-12-16 10:04:27 -08:00
Steve Howell d75ff80eb2 muting: Add stream_id to markup for mute settings.
We also prefer `attr` over `data` (it's more greppable).
2018-12-14 16:05:35 -08:00
Steve Howell bf6f5e7bc5 muting: Pass stream_id to muting_ui.mute and unmute.
We temporarily allow settings_muting to have
incomplete line coverage--we will fix this soon.
2018-12-14 16:02:48 -08:00
rht 501ae0c3d6
provision: Add centos7 postgresql dependencies. 2018-12-14 04:59:48 +00:00
rht e8c602ec58
provision: Rename APT_DEPENDENCIES -> SYSTEM_DEPENDENCIES. 2018-12-14 04:59:48 +00:00
rht d54fb5f40d
provision: Add venv dependencies for Centos 7. 2018-12-14 04:59:46 +00:00
Vishnu Ks 0fd6ff722b billing: Migrate /upgrade endpoint to JSON.
The fixture changes are because self.upgrade formerly used to cause a page load
of /billing, which in turn calls Customer.retrieve.

If we ran the full test suite with GENERATE_STRIPE_FIXTURES=True, we would
likely see several more Customer.retrieve.N.json's being deleted. But
keeping them there for now to keep the diff small.
2018-12-13 17:01:12 -08:00
Anders Kaseorg 149132348f lint: Update ShellCheck to 0.6.0.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-12 08:24:01 -08:00
rht a0b0ccf972 provision: Do not require lsb_release for centos. 2018-12-11 13:08:27 -08:00
rht c9d54f7854 puppet: Remove vendored puppetlabs apt and stdlibs dependencies.
This commit works by vendoring the couple functions we still use from
puppetlabs stdlib (join and range), but removing the rest of the
puppetlabs codebase, and of course cleaning up our linter rules in the
process.

Fixes #7423.
2018-12-11 13:03:26 -08:00
Vishnu Ks a86faecff5 tools: Rename tools/travis directory to tools/ci. 2018-12-10 11:07:26 -08:00
konraddeka e6be407e34 provision: Avoid spending time on update-authors-json.
This optimizes tools/provision by not running
`tools/update-authors-json --use-fixture` unless either the script
itself or its fixtures file (zerver/tests/fixtures/authors.json) was
changed.

Fixes #10991.
2018-12-09 22:21:51 -08:00
rht 9bfea2adba provision.py: Translate COMMON_APT_DEPENDENCIES to yum. 2018-12-09 21:54:02 -08:00
Sumanth V Rao 7d12e2019d provision: Generate static bots files using a function call.
This shaves about 350ms from the no-op provision runtime.

Fixes #10990.
2018-12-09 21:44:45 -08:00
Sumanth V Rao cae718f184 provision: Rename generate_zulip_bots_static_files to .py.
This should make it possible to call this script as a function without
starting up a new Python interpreter.
2018-12-09 21:42:40 -08:00
Tim Abbott 9e48b2f17f provision: Update comment for why we need puppet in development.
This should help avoid confusion like #10972.
2018-12-06 11:13:04 -08:00
Tim Abbott 9de1bd44e2 compatibility: Clean up linter logic for too-old client. 2018-12-05 15:04:21 -08:00
Greg Price fb7bfbe9ab compatibility: Require version 16.2.96+ for our Android app.
This release is from 2018-08-22, a little over 100 days ago.

It was the first release with the important fix so that when the
server advises it to stop displaying a notification because the user
has read the message (as the SEND_REMOVE_PUSH_NOTIFICATIONS server
setting enables), the app doesn't instead replace the notification
with a broken one reading "null".  We have that setting running now
on chat.zulip.org, and intend to roll it out more broadly soon.

The `# take 0` thing is a slightly absurd workaround for the fact
that our funky out-of-line way of marking lines to ignore doesn't
work right if there are multiple such lines in a given file that
are equal modulo leading and trailing whitespace.
2018-12-05 14:58:13 -08:00
Tim Abbott 6689913f77 ci: Enable test-documentation --skip-external-links in CI.
Fixes #10942.
2018-12-05 12:39:59 -08:00
Sumanth V Rao e1f33e0f6b test-documentation: Add --skip-external-links option.
This option causes test-documentation to only verify internal links
that we control and are important to be correct.  This prevents
test-documentation from flaking in CI due to issues with the dozens of
third-party blocks that we link to from various parts of our
documentation.

Tweaked by tabbott for comment clarity, and to also include
github.com/zulip links.

Fixes #10942.
2018-12-05 12:39:59 -08:00
Tim Abbott f5f320d5fc mypy: Print out link to documentation in error output.
This should help make details about how mypy works discoverable for
developers who are new to it.
2018-12-05 11:23:55 -08:00
Tim Abbott f694d01a1b test-backend: Expand desired files with full test coverage.
This should help catch a few classes of files that we weren't
correctly considering for potentially having full test coverage.
2018-12-05 11:01:55 -08:00
Steve Howell 67fba69b0c hashchange: Go to home page for bogus narrows.
This was the original intent of the code, and I think
it's the right behavior.
2018-12-04 17:16:38 -08:00
Steve Howell bcb142e68e hashchange: Move parse_narrow to hash_util.js.
The goal here was to enforce 100% coverage on
parse_narrow, but the code has an unreachable line
and is overly tolerant of bogus urls.  This will
be fixed in the next commit.
2018-12-04 17:16:32 -08:00
Anders Kaseorg 9ba860b4f2 Avoid double redirects to /login and then to /login/.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-04 16:12:58 -08:00
Tim Abbott 86536059b4 tools: Fix linter tests after SCSS rename.
This was introduced in 815388b023.
2018-12-04 09:43:56 -08:00
Vishnu Ks 788b98d041 portico: Add page for redirecting to a realm subdomain. 2018-12-04 09:35:35 -08:00
Tim Abbott 815388b023 lint: Fix CSS linter rules not running.
Apparently, our CSS linter rules haven't been running since we
migrated to SCSS.
2018-12-04 09:35:34 -08:00
Vishnu Ks 6de6b0ed31 emails: Extract CSS to email.css in digest.html. 2018-12-03 17:43:53 -08:00
Tim Abbott 07adda84da Revert "ci: Re-enable testing documentation links."
This reverts commit d8e5e2d960.

See https://github.com/zulip/zulip/issues/10942 for a plan for making
this something we can re-enable.
2018-12-02 18:34:35 -08:00
Tim Abbott 1303f8df20 docs: Fix missing quotes in `su zulip -c` documentation.
This fixes an actual user-facing issue in our mobile push
notifications documentation (where we were incorrectly failing to
quote the argument to `./manage.py register_server` making it not
work), as well as preventing future similar issues from occurring
again via a linter rule.
2018-11-30 12:12:27 -08:00
Tim Abbott a3b38d8402 docs: Document nginx configuration for development SSL.
This is a useful technique for developing OAuth integrations with Zulip.
2018-11-29 20:45:02 -08:00
Steve Howell ba243416e3 stream_color: Make stream color assignment more efficient.
We now let color_data keep its own state for
unused_colors, so that we longer have to pass in
a large list of unused_colors every time we want
to assign a new stream color.

This mostly matters at startup, where we might
be cycling through 5000 streams.  We claim all
the unused colors up front.

Each operation now has an upper bound of expensiveness,
where the worst case scenario is basically popping
off the first element of a list of <= 24 colors.

The algorithm is now deterministic, too, to make
it easier to test.  It's unclear whether random color
assignment ever had much benefit, and it made unit
testing the algorithm difficult.  Now we have 100%
line coverage.

Fixes part of #10902.
2018-11-29 13:00:25 -08:00
Anders Kaseorg 0bfcd2294e tools/update-prod-static: Avoid shelling out for mkdir, cp, mv, rm.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Anders Kaseorg 19bdf54f33 tools/setup/emoji/build_emoji: Avoid shelling out for touch, rm, mkdir, cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Tim Abbott d8e5e2d960 ci: Re-enable testing documentation links.
We've had a few unpleasant bugs with real documentation links being
broken, so we're going to make this more aggressive for now.

I think we instead want a more subtle option for suppressing failures
in some places but not others.
2018-11-28 09:52:04 -08:00
Tim Abbott 723fb60476 docs: Fix up some broken links output spam. 2018-11-28 09:49:39 -08:00
Rishi Gupta dd0126ff1b settings: Improve error message when deactivating the last user.
This PR was originally started by Rishi Gupta (see #10383).
2018-11-27 12:49:43 -08:00
Tim Abbott adf27aae4c python: Remove now-unnecessary str_utils library.
This library was absolutely essential as part of our Python 2->3
migration process, but all of its calls should be either no-ops or
encode/decode operations.

Note also that the library has been wrong since the incorrect
refactoring in 1f9244e060.

Fixes #10807.
2018-11-27 11:57:54 -08:00
Tim Abbott 58a6dafa68 lint: Exclude outbox from linter checks on "subject". 2018-11-27 11:05:25 -08:00
Tim Abbott d5f53164b4 docs: Suppress markdown line length linter errors. 2018-11-18 18:56:44 -08:00
Steve Howell eb4d279bbc Extract get_turtle_message().
This seems like kind of a silly function to extract
to topic.py, but it will theoretically help us sweep
"subject" if we change the DB.

It had test coverage.
2018-11-14 23:24:06 -08:00
Steve Howell cc33e4cd0c digest: Eliminate unneeded queries for hot convos.
We can easily keep track of messages by bucket from the
original loop through messages.
2018-11-14 23:24:06 -08:00
Steve Howell 5a539d44b0 subject -> topic: Fix notification tests.
The URLs changed here are artificial, so we can convert
subject -> topic without breaking anything.
2018-11-14 23:24:06 -08:00
Steve Howell 27d79352da topic -> subject: Extract get_topic_from_message_info().
This changes files where it's safe to just assume caller
may use either "topic" or "subject", and we prefer "topic"
but support "subject".
2018-11-14 23:24:06 -08:00
Steve Howell 44cbcdf00c minor: Remove txt file from lint exception.
It's already ignored because it's not a .py file.
2018-11-14 23:24:06 -08:00
Steve Howell 2505a2f3cc lint: Build exceptions for subject. 2018-11-14 23:24:06 -08:00
Steve Howell a3460d1ef6 lint: Remove obsolete exemptions.
These two files have been cleaned up since the original
lint rule exemption.
2018-11-14 23:24:06 -08:00
Steve Howell c69c40a55a lint: Extract check_file_for_pattern(). 2018-11-14 23:24:06 -08:00
Steve Howell fd0b249a29 lint: Calculate strip_rule outside the loop.
This adds up over a million lines. :)
2018-11-14 23:24:06 -08:00
Steve Howell f386c64759 lint: Speed up by doing fewer regex compiles.
Even though you'd think these regexes would be
cached, compiling the regex outside of looping
through lines makes a difference.

My timings are 8.4s -> 6.0s.  (You need to hack
on the linter to isolate the custom checks.)
2018-11-14 23:24:06 -08:00
Steve Howell 08e44aac0d lint: Extract get_rules_applying_to_fn(). 2018-11-14 23:24:06 -08:00
Steve Howell eda138343b lint: Extract get_line_info_from_file(). 2018-11-14 23:24:06 -08:00
Steve Howell 6d88210110 lint: Extract check_file_for_long_lines().
In passing this removes some unnecessary string
decoding that's a relic of python2.
2018-11-14 23:24:06 -08:00
Steve Howell a942732d83 lint: Get rid of unused skip_rules.
We also go to the modern annotation style for mypy.
2018-11-12 16:00:52 -08:00
Steve Howell 3eb2b2be5f lint: Change lastLine logic slightly. 2018-11-12 15:47:11 -08:00
Steve Howell a86e7fbd9f subject -> topic: Fix message tests. 2018-11-12 15:47:11 -08:00
Steve Howell c90085b28e lint: Extend "subject" prohibition to zerver/tests.
There are several exemptions here, so I triaged
them into various buckets.
2018-11-12 15:47:11 -08:00
Steve Howell d86dd165da gitter/slack/hipchat: Remove "subject" from conversions.
We (lexically) remove "subject" from the conversion code.  The
`build_message` helper calls `set_topic_name` under the hood,
so things still have "subject" in the JSON.

There was good code coverage on `build_message`.
2018-11-12 15:47:11 -08:00
Steve Howell ea98a44db3 webhooks: Replace SUBJECT_WITH_* with TOPIC_WITH_*. 2018-11-12 15:47:11 -08:00
Steve Howell 97062c4017 lint: Prevent use of "subject" in zerver/views. 2018-11-12 15:47:11 -08:00
Eeshan Garg 10570d508f tools/lint: Fix JSON lint pattern for spaces after ':'.
Previously, a string ending in "... 😄" was reported as an
error and the linter complained that there should be a space
after the last ':'. This commit changes the pattern so that the
linter only checks for colons that are preceded by an opening
double-quote (").
2018-11-08 22:07:56 -08:00
Steve Howell cd4d447d90 Minor: change message.subject -> message.topic_name().
This also allows us to lock down
push_notifications.py from using "subject"
in the future.
2018-11-08 16:34:30 +00:00
Steve Howell ff9a6c5ced minor: Rename subject -> topic_name in bugdown. 2018-11-08 16:21:14 +00:00
Steve Howell 70c16ddbcc lint: Prevent "subject" more aggressively.
We now prevent adding "subject" to any code in
zerver/lib, unless you specifically exempt it.

The new set called `FILES_WITH_LEGACY_SUJECT`
is also has comments that give a roadmap of what
to fix.
2018-11-08 15:33:36 +00:00
Steve Howell e802be0fde Remove "subject" in notifications.py.
These last few are all pretty safe.
2018-11-08 15:12:10 +00:00
Steve Howell 226a2826b5 Remove "subject" from zerver/lib/message.py.
We also lock this down file in the linter, so
that you can no longer introduce "subject"
to `message.py`.
2018-11-08 14:45:16 +00:00
Steve Howell 6b6001c46c Remove "subject" from test_classes.py.
We now use "topic" lingo.
2018-11-07 10:03:53 -08:00
Steve Howell 79d5e36ca3 Extract topic_match_sa() helper.
We'll also use this in zerver/views/messages.py, but
that's a bigger change.
2018-11-07 10:03:53 -08:00
Steve Howell df743e8948 lint: Start to deprecate "subject" references. 2018-11-07 10:03:53 -08:00
Rishi Gupta 69e9a8f196 user docs: Add a few portico URLs to documentation crawler test. 2018-11-01 11:59:50 -07:00
Tim Abbott d9887759b4 plans: Fix broken link from FAQ to import docs and add tests. 2018-10-31 11:44:21 -07:00
zegervdv 6dd2218c50 provision: Pass proxy settings through sudo via environment.
This makes provision not fail when installing node behind a proxy.

See #10505 for details.
2018-10-29 13:30:45 -07:00
Vishnu Ks 0cf4a75fb5 coverage: Enforce full test coverage for corporate/views.py. 2018-10-24 19:39:23 -07:00
Anders Kaseorg c3bd293eaf Add shellcheck as a linter.
Fixes #10162.

Tweaked by tabbott to cover documentation as well.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:32:26 -07:00
Anders Kaseorg 49ae9c1e44 pre-commit: Fix shellcheck warnings.
In tools/pre-commit line 18:
if [ -z "$VIRTUAL_ENV" ] && `which vagrant > /dev/null` && [ -e .vagrant ]; then
                            ^-- SC2092: Remove backticks to avoid executing output.
                            ^-- SC2006: Use $(..) instead of legacy `..`.
                             ^-- SC2230: which is non-standard. Use builtin 'command -v' instead.

In tools/pre-commit line 23:
    ./tools/lint --no-gitlint --force $changed_files || true
                                      ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:26:21 -07:00
Anders Kaseorg fc24d2e147 install-aws-server: Fix shellcheck warnings.
In tools/setup/install-aws-server line 25:
zulip_root=${ZULIP_ROOT:-$HOME/zulip}
^-- SC2034: zulip_root appears unused. Verify use (or export if used externally).

In tools/setup/install-aws-server line 40:
if [ -n "$zulip_confdir" ]; then
         ^-- SC2154: zulip_confdir is referenced but not assigned.

In tools/setup/install-aws-server line 55:
VIRTUALENV_NEEDED=$(if $(echo "$type" | grep -q app_frontend); then echo -n yes; else echo -n no; fi)
                       ^-- SC2091: Remove surrounding $() to avoid executing output.

In tools/setup/install-aws-server line 60:
SSH_OPTS=(-o HostKeyAlgorithms=ssh-rsa)
             ^-- SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

In tools/setup/install-aws-server line 69:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

In tools/setup/install-aws-server line 86:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:51:41 -07:00
Anders Kaseorg b15063155c postgres-init-dev-db: Fix shellcheck warnings.
In tools/setup/postgres-init-dev-db line 10:
ROOT_POSTGRES="sudo -i -u "$DEFAULT_USER" psql"
                           ^-- SC2027: The surrounding quotes actually unquote this. Remove or escape them.

In tools/setup/postgres-init-dev-db line 46:
    echo 'ERROR: Try `sudo service postgresql start`?'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

In tools/setup/postgres-init-dev-db line 64:
PGPASS_ESCAPED_PREFIX="*:\*:\*:$USERNAME:"
                         ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".
                            ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:48:57 -07:00
Anders Kaseorg 7f35ad916e django-template-graph: Fix shellcheck warnings.
In tools/django-template-graph line 10:
    for t in $(find -name '*.html' -printf '%P\n'); do
             ^-- SC2044: For loops over find output are fragile. Use find -exec or a while read loop.
               ^-- SC2185: Some finds don't have a default path. Specify '.' explicitly.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg c63d852b55 deploy-branch: Fix shellcheck warnings.
In tools/deploy-branch line 17:
[ $? -ne 0 ] && error_out "Unknown branch: $branch"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 23:
    if [ $? -eq 0 ]; then
         ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 35:
[ $? -ne 0 ] && error_out "Rebase onto origin/master failed"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

In tools/deploy-branch line 39:
[ $? -ne 0 ] && error_out "Push of master to origin/master failed"
  ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg 4a8074000c commit-msg: Fix shellcheck warnings.
In tools/commit-msg line 9:
if [ $(grep '^[^#]' .git/COMMIT_EDITMSG --count) -ne 0 ]; then
     ^-- SC2046: Quote this to prevent word splitting.

In tools/commit-msg line 10:
    lint_cmd="cd ~/zulip && cat \"$1\" | python -m gitlint.cli"
              ^-- SC2089: Quotes/backslashes will be treated literally. Use an array.

In tools/commit-msg line 11:
    if [ -z "$VIRTUAL_ENV" ] && `which vagrant > /dev/null` && [ -e .vagrant ]; then
                                ^-- SC2092: Remove backticks to avoid executing output.
                                ^-- SC2006: Use $(..) instead of legacy `..`.
                                 ^-- SC2230: which is non-standard. Use builtin 'command -v' instead.

In tools/commit-msg line 14:
        $lint_cmd
        ^-- SC2090: Quotes/backslashes in this variable will not be respected.

In tools/commit-msg line 17:
    if [ $? -ne 0 ]; then
         ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg 8b38f61176 clean-branches: Fix shellcheck warnings.
In tools/clean-branches line 33:
                echo -n "Deleting local branch $(echo "$ref" | sed 's!^refs/heads/!!')"
                                                 ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/clean-branches line 41:
                echo -n "Deleting local branch $(echo "$ref" | sed 's!^refs/heads/!!')"
                                                 ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/clean-branches line 49:
                remote_name="$(echo "$ref" | sed 's!^refs/remotes/origin/!!')"
                               ^-- SC2001: See if you can use ${variable//search/replace} instead.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg b5bfe77844 build-release-tarball: Fix shellcheck warnings.
In tools/build-release-tarball line 50:
for i in `cat "$TMPDIR/$prefix/tools/release-tarball-exclude.txt"`; do
         ^-- SC2013: To read lines rather than words, pipe/redirect to a 'while read' loop.
         ^-- SC2006: Use $(..) instead of legacy `..`.

In tools/build-release-tarball line 51:
    rm -r --interactive=never "$TMPDIR/$prefix/$i";
                              ^-- SC2115: Use "${var:?}" to ensure this never expands to / .

In tools/build-release-tarball line 97:
    echo; echo -ne "\033[33mRunning update-prod-static failed. "
                    ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/build-release-tarball line 98:
    echo -e "Check $TMPDIR/update-prod-static.log for more information.\033[0m"
                                                                       ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Anders Kaseorg a5280d7d02 build-docs: Fix shellcheck warnings.
In tools/build-docs line 3:
cd "$(dirname "$0")"/../docs
^-- SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:38:56 -07:00
Rishi Gupta d76f728dda api docs: Rename webhook-walkthrough to incoming-webhooks-walkthrough. 2018-10-16 21:07:00 -07:00
Eeshan Garg 0292ffe04b webhooks/taiga: Add test for when a task is moved between stories.
This commit adds a test for the payload that is generated when
a Task is moved from one user story to another on Taiga's Sprint
Taskboard UI.

This commit also gets up this webhook's test coverage up to 100%.
2018-10-16 13:06:20 -07:00
Aditya Bansal 75ae94e459 font-awesome: Drop support for legacy font awesome icons.
We drop support for usage of `icon-vector` as base class when
including icons from font awesome icons package.
Now on, only icons as specified in font awesome v4.7.0 can be used
in the code base.
2018-10-15 20:14:55 +05:30
Aditya Bansal 7eeed1413c icons: Remove base class from comments and test templates. 2018-10-15 19:55:18 +05:30
Steve Howell b7f764aa29 settings: Add confirm_dialog module.
This module makes it really easy to create are-you-sure
dialogs for dangerous operations.

Basically it's one function with five parameters.  You
give three chunks of HTML, a callback function, and
a parent container.

The first use of this will be in settings_user_groups,
coming up in a couple commits.
2018-10-12 10:37:06 -07:00
Eeshan Garg ba929508e2 webhooks/ifttt: Get test coverage up to 100%.
IFTTT allows custom templating for their payloads, so the onus is
on the user to ensure that their custom templates conform to the
expectations outlined in our IFTTT webhook docs. For that reason,
these payloads weren't generated, but were manually edited.
2018-10-04 12:16:06 -07:00
Eeshan Garg 316f9aa78b webhooks/hellosign: Rewrite the integration from scratch.
After discovering a couple of bugs, I decided to thoroughly test
and rewrite this integration from scratch. The older code wasn't
generating coherent messages.

This also commit gets this integration up to 100% test coverage.
2018-10-04 12:16:06 -07:00
Eeshan Garg 5039f6dfb5 webhooks/gitlab: Get test coverage up to 100%.
Test coverage was improved by removing an unused function and
removing some code (written by me) that was actually handling
Test Hook event types incorrectly.
2018-10-04 12:16:06 -07:00
Eeshan Garg 6e2e2b9125 webhooks/github: Test commit status payloads with target_url.
It was a painful amount of work to generate the actual payload.
Since the only difference was a small build URL, I manually
edited the payload and used that for testing.

This commit gets our GitHub webhook up to 100% test coverage.
2018-10-04 12:16:06 -07:00
Eeshan Garg b40aec3a09 webhooks/freshdesk: Improve test coverage.
Note that Freshdesk allows custom templating for outgoing payloads
in their webhook UI. Therefore, the payloads added in this commit
did not have to be official payloads from Freshdesk.
2018-10-04 12:16:06 -07:00
Eeshan Garg d8101ca139 webhooks/bitbucket2: Improve test coverage.
The lack of coverage was due to:

* An unused function that was never used anywhere.
* get_commit_status_changed_body was using a regex where it didn't
  really need to use one. And there was an if statement that
  assumed that the payload might NOT contain the URL to the commit.
  However, I checked the payload and there shouldn't be any instances
  where a commit event is generated but there is no URL to the commit.
* get_push_tag_body had an `else` condition that really can't happen
  in any payload. I verified this by checking the BitBucket webhook
  docs.
2018-10-04 12:16:06 -07:00
Eeshan Garg 185a023745 webhooks/beanstalk: Improve test coverage.
We shouldn't just ignore exceptions when encoding the incoming
auth credentials. Even if the incoming credentials are properly
encoded, it is better to know when that is the case or if
something else fails.
2018-10-04 12:16:06 -07:00
Harshit Bansal fc7e654d7b emoji: Remove single-digit names for number emojis.
Names like `:1:`, `:2:` etc don't seem to be of much benefit rather
they seem to get rendered at places not intended.
2018-10-02 11:30:11 -07:00
Vishnu Ks 59865797ac billing: Move billing tests from zilencer to corporate. 2018-10-01 15:44:02 -07:00
Vishnu Ks 93b398bc0a billing: Move zilencer/lib/stripe to corporate. 2018-10-01 15:43:44 -07:00
Vishnu Ks daf3b46262 billing: Move billing related templates from zilencer to corporate. 2018-10-01 15:43:39 -07:00
Vishnu Ks 6914ee126c billing: Move billing-related views and urls to corporate. 2018-10-01 15:43:21 -07:00
Tim Abbott 00821d67bb travis: Remove no-longer-present packages from apt-mark hold list.
Apparently, Travis removed the Heroku bundle of packages from their
servers, which made the build start failing when trying to configure
apt to hold their versions (sigh).  This commit removes the
problematic packages.
2018-09-28 14:20:04 -07:00
Greg Price 3b646c5d28 push-to-pull-request: Update local tracking ref, if any.
The companion tool `tools/reset-to-pull-request` has a handy feature
to maintain a local ref tracking the PR: e.g., pr/1234 for PR 1234.
If this were a normal remote-tracking branch maintained by `git fetch`,
it'd get updated on `git push`.  Do the same thing here.

This helps keep a view like `gitk --all @` a bit tidier, by causing
merged PRs to stop pointing at side branches of the main history.
2018-09-25 13:15:02 -07:00
Vishnu Ks 5a6b2ebb1f billing: Add ability for users to change their card. 2018-09-24 19:25:26 -07:00
Anders Kaseorg 5e88b3013a zanitizer_config.pm.sample: keep_file → scrub_filename
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-09-23 20:42:27 -07:00
Eeshan Garg ecd4f821be user docs: Automatically activate correct tab for OS-specific instructions. 2018-09-18 13:49:34 -07:00
Vishnu Ks d6b548bd57 styles: Extract billing SCSS into its own stylesheet. 2018-09-11 00:08:38 -07:00
Steve Howell 06225d1424 tests: Clean up calls to tools/webpack.
Before this change, the way we loaded
webpack for various tools was brittle.

First, I addressed test-api and test-help-documentation.

These tools used to be unable to run standalone on a
clean provision, because they were (indirectly)
calling tools/webpack without the `--test` option.

The problem was a bit obscure, since running things
like `./tools/test-backend` or `./tools/test-all` in
your workflow would create `./var/webpack-stats-test.json`
for the broken tools (and then they would work).

The tools themselves weren't broken; they were the
only relying on the common `test_server_running` helper.
And even that helper wasn't broken; it was just that
`run-dev.py` wasn't respecting the `--test` option.

So I made it so that `./tools/run-dev` passes in `--test` to
`./tools/webpack`.

To confuse matters even more, for some reason Casper
uses `./webpack-stats-production.json` via various
hacks for its webpack configuration, so when I fixed
the other tests, it broke Casper.

Here is the Casper-related hack in zproject/test_settings.py,
which was in place before my change and remains
after it:

    if CASPER_TESTS:
        WEBPACK_FILE = 'webpack-stats-production.json'
    else:
        WEBPACK_FILE = os.path.join('var', 'webpack-stats-test.json')

I added similar logic in tools/webpack:

    if "CASPER_TESTS" in os.environ:
        build_for_prod_or_casper(args.quiet)

I also made the helper functions in `./tools/webpack` have
nicer names.

So, now tools should all be able to run standalone and not
rely on previous tools creating webpack stats files for
them and leaving them in the file system.  That's good.

Things are still a bit janky, though.  It's not completely
clear to me why `test-js-with-casper` should work off of
a different webpack configuration than the other tests.

For now most of the jankiness is around Casper, and we have
hacks in two different places, `zproject/test_settings.py` and
`tools/webpack` to force it to use the production stats
file instead of the "test" one, even though Casper uses
test-like settings for other things like which database
you're using.
2018-09-07 11:39:55 -04:00
Steve Howell 25ad755914 Fix usage for test-api. 2018-09-07 11:39:55 -04:00
Steve Howell 336c2cee20 Revert unused code for creating non-editable pills.
We don't use input.create_non_editable_pill() in our
code yet.  If we add this back, we'll want to have node
tests on it.

Removing this unused code brings us to 100% line
coverage for input_pill.js.

This directly reverts 5c11ab85 with the small addition
of adding input_pill to our list of fully covered
modules.
2018-08-31 16:26:05 -07:00
Harshit Bansal 88bfb9778b build_emoji: Add infrastructure for google-blob and twitter emojisets. 2018-08-31 11:52:24 -07:00
Marco Burstein 25e624eab1 team: Include contributions from legacy repositories in the total.
These repositories (`zulip-ios-legacy` and `zulip-android`) are
deprecated, and as such should not have their own tabs, but still
should be included in the total contributions count.
2018-08-28 16:45:00 -07:00
Tim Abbott 2c7f9ce0fc puppet: Fix puppet-lint warnings in various manifests.
Apparently, `puppet-lint` on Ubuntu trusty throws warnings for certain
quoting patterns that are OK in modern `puppet-lint`.  I believe the
old Zulip code was actually correct (i.e. the old `puppet-lint`
implementation was the problem), but it seems worth changing anyway to
suppress the warnings.

We also exclude more of puppet-apt from linting, since it's
third-party code.
2018-08-28 13:46:31 -07:00
Lyla Fischer 1efcdfdb5c templates: Merge Help and API doc main into documentation_main.
With minor fixes by eeshangarg.
2018-08-27 20:19:24 -02:30
Harshit Bansal e204fe3948 minor: Fix for failing tests. 2018-08-27 06:16:16 +00:00
Harshit Bansal d96d6a96f7 build_emoji: Add a missing check for image existence. 2018-08-26 22:16:23 -07:00
Harshit Bansal f5454dff51 build_emoji: Automatically calculate sprite dimensions.
Instead of using a hardcoded value for spritesheet dimensions,
automatically calculate it using `emoji_data`. This will free
us from updating it only emoji datasource update as well as
allow us to add google blob emojiset.
2018-08-26 22:16:23 -07:00
Harshit Bansal dbc3774c2f build_emoji: Refactor `get_emoji_code()` to not fail for old datasources.
If `non_qualified` field is not present then return the value of
`unified` field instead of failing.
2018-08-26 22:16:23 -07:00
Harshit Bansal ec2ebd1cc9 build_emoji: Change sprite CSS files to use hyphens in names.
This is more consistent with our other names in emoji infra.
2018-08-26 22:16:23 -07:00
Harshit Bansal 9bd56d0ce4 build_emoji: Change emoji sprite sheets to use hyphens in names.
Use of hyphens is more consistent with our other emoji infra names.
2018-08-26 22:16:23 -07:00
Harshit Bansal aa6f098496 build_emoji: Extract `setup_emoji_farm()`. 2018-08-26 22:16:23 -07:00
Harshit Bansal dc9ff928f3 build_emoji: Refactor `ensure_emoji_image()` to take paths as args. 2018-08-26 22:16:23 -07:00
Harshit Bansal ed694001da build_emoji: Rename `setup_emoji_farm()` to `setup_emoji_farms()`. 2018-08-26 22:16:23 -07:00
Steve Howell ca272ef49d tests: Test how we handle extra data for zform messages. 2018-08-24 10:00:25 -07:00
Steve Howell bee592b44e node tests: Bring list_cursor to 100% coverage.
We had some indirect coverage of this already, so
this mostly focuses on error cases and corner
cases.
2018-08-24 10:00:04 -07:00
Steve Howell 44756a326d tests: Test duplicate default stream/groups.
This also gets us to 100% line coverage
for actions.py.
2018-08-22 16:52:30 -07:00
Steve Howell 4318f75718 Remove `code` from JsonableError.__init__.
Since this class was built, folks have always chosen
to subclass JsonableError for situations where
the default of ErrorCode.BAD_REQUEST is insufficient.

So now we simplify the use cases, which also gets
us 100% coverage on this core module.
2018-08-22 16:51:40 -07:00
Tim Abbott b93d5bd5e8 test-js-with-node: Temporarily remove server_events_dispatch.js.
This fixes master failing tests since 2aeabf24a6.
2018-08-21 14:40:05 -07:00
Joshua Pan 793ec9e266 tests: Test all update_display_settings events. 2018-08-19 14:51:31 -04:00
xificurC 9e053c74cf scripts: Allow configuring a custom CA bundle for build process.
For building Zulip in an environment where a custom CA certificate is
required to access the public Internet, one needs to be able to
specify that CA certificate for all network access done by the Zulip
installer/build process.  This change allows configuring that via the
environment.
2018-08-13 15:45:28 -07:00
Tim Abbott 90828297e4 puppet-lint: Enforce double_quoted_strings check.
This makes our puppet codebase more consistent by using single-quoted
strings consistently.
2018-08-13 12:31:19 -07:00
Tim Abbott d0b51b70f4 puppet-lint: Enforce 2sp_soft_tables puppet-lint check.
This cleans up the puppet codebase's whitespace formatting to be more
consistent.
2018-08-13 12:31:16 -07:00
Tim Abbott b26e0a957d puppet-lint: Enforce arrow_alignment check.
This fixes all exceptions in our puppet codebase to this lint rule.
2018-08-13 12:30:57 -07:00
Aditya Bansal a3b8ab46b3 puppet-lint: Include in linter tool chain to run by default.
We start to use puppet-lint to lint puppet modules by default by
adding it to tools/lint (which controls our linter tool chain).

We also define a few puppet-lint rules to exclude.

Fixes: #9185.
2018-08-13 12:30:57 -07:00
Aditya Bansal 56d49980de puppet-lint: Add as a dependancy to install during provision process. 2018-08-13 12:30:57 -07:00
Harshit Bansal 29daf76420 emoji: Don't generate emoji farms corresponding to now banned emojisets.
This will help us in reducing the size of the release tarball
significantly. I have refrained from changing the `EMOJISETS`
constant in the `emoji_setup_utils.py` as that controls the
emojisets that we want to support. Since we want to re-enable
the feature of changing emojisets sometime again in the future
that variable should be kept as it is as it controls several
other things like emoji scripts that we use to generate emoji
names. Changing it might cause hard to catch bugs.
2018-08-13 12:09:06 -07:00
Harshit Bansal 36b977c382 emoji: Refactor `generate_sprite_css_files()` to take emojiset in args. 2018-08-13 12:09:06 -07:00
Harshit Bansal 67bbbbb007 emoji: Move logic for hard-coding octopus emoji.
This logic was incorrectly placed in `setup_old_emoji_farm()` while
it should have been placed in `setup_emoji_farm()`.
2018-08-13 12:09:06 -07:00
Harshit Bansal 1eb3825048 emoji: Remove now-unnecessary `get_remapped_emojis_map()`. 2018-08-13 12:08:14 -07:00
Harshit Bansal 99e1a81055 emoji: Fix some emoji images not loading in missed message emails.
`emoji-datasource` package v4.0.4 introduced the concept of qualified
and non-qualified emoji codes. As chat programs don't need to use
emoji representation selector, so we used migrated our infrastructure
to use non-qualified emoji codes. But we missed the fact that the
emoji file names in emoji farm are based on emoji data's 'unified'
field and the value of this field has changed. Consequently the image
file names must also have been changed. We used `emoji_code` while
converting the span tags to img tags while processing notifications.
But since now `emoji_code` refers to non-qualified code while image
file names are based on qualified code, we need to rename images
to correctly do the conversion. This commit just fixes this.
2018-08-13 12:03:16 -07:00
Steve Howell 413a0174f4 Extract a zephyr.py library.
Right now it only has one function, but the function
we removed never really belonged in actions.py, and
now we have better test coverage on actions.py, which
is an important module to get to 100%.
2018-08-11 14:51:26 -07:00
Max Nussenbaum 570b9515c6 billing: Add location for errors on billing page.
This adds a location for error messages to the billing page,
using the standard error styling.
2018-08-10 22:48:10 -07:00
Vishnu Ks 0c84260faa billing: Enforce full coverage in zilencer/tests. 2018-08-10 13:52:35 -07:00
Vishnu Ks 9946202148 coverage: Enforce full coverage for zilencer/lib/stripe.py. 2018-08-10 13:52:35 -07:00
Tim Abbott aaedec1fdb tools: Move a few i18n scripts to tools/i18n.
These are likely to only be run by Zulip maintainers.
2018-08-09 15:46:27 -07:00
Tim Abbott ba626dcad6 tools: Move optimize-svg to tools/setup/. 2018-08-09 15:39:57 -07:00
Tim Abbott c2e5cc99a4 tools: Move generate-custom-icon-webfont to a subdirectory. 2018-08-09 15:39:57 -07:00
Tim Abbott 2bb04fc720 tools: Remove send-github-payloads.
This ancient tool predates our practice of collecting test fixtures
for third-party integrations, which is a better general system for the
problem this solved.
2018-08-09 15:39:57 -07:00
Tim Abbott 74dc3e9a3e tools: Remove unused munge-postgres-logs.
This hasn't been used in several years.
2018-08-09 15:39:57 -07:00
Tim Abbott bd945333a3 tools: Remove replace-tarball-shebang.
This was a temporary helper tool for part of
tools/build-release-tarball during our Python 3 migration, which
finished some time ago.
2018-08-09 15:27:09 -07:00
Tim Abbott 78539e7819 tools: Remove unused deployment-lock-ctl.
This interacted with a deployment locking system that is no longer
part of the project.
2018-08-09 15:25:15 -07:00
Rhea Parekh 26300110ca import: Fix rendered_content in imported messages.
After the messages have been imported, set the rendered_content of the
messages instead of leaving its value to be 'None'.

This is important to ensure that:
(1) Performance for users is good after completing the import.
(2) The database's full-text indexes have all of the imported messages
(which only happens properly when Message rows have their
rendered_content field edited).

Fixes #9168.
2018-08-09 15:12:53 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Yago González a0d8cbc9cb test-backend: Fix typo. 2018-08-08 16:34:29 -07:00
Cynthia Lin ecbc58c586 settings: Fix color styling for nested inputs to work in night mode. 2018-08-08 11:29:48 -07:00
Rhea Parekh 30cc7354eb import: Move 'process_avatars' to import_util. 2018-08-07 16:45:40 -07:00
Rhea Parekh 1117455a90 import: Move 'ZerverFieldsT' and 'build_zerver_realm' to import_util. 2018-08-07 16:35:56 -07:00
Vishnu Ks 119d108da0 droplets: Update code and remove nvm settings from bashrc.
* Synced zulip repo to master and provisioned
* Synced python-zulip-api repo to master and provisioned.
* Removed unnecessary nvm config from bashrc
2018-08-07 13:38:18 -07:00
Tim Abbott 4300683bca lint: Fix mypy error in check_pyflakes annotations.
Introduced in 78a93b8d9e.
2018-08-07 09:59:14 -07:00
Tim Abbott e67fa36803 lint: Update config to split files more evenly. 2018-08-07 09:42:37 -07:00
Tim Abbott 78a93b8d9e lint: Refactor pyflakes to use more consistent style.
In particular, we no longer pass in the full `by_lang` object and
expect it to pull out the Python piece.
2018-08-07 09:42:37 -07:00
Tim Abbott 458c17868f zulint: Move pycodestyle runner code to library.
This way, only the Zulip project's specific configuration ends up
living in pep8.py.
2018-08-04 19:54:18 -07:00
Tim Abbott 3b53ddf779 zulint: Stop importing color codes from zulip_tools.py.
This is I think the final dependency that zulint/ has on the rest of
the project.
2018-08-04 19:53:53 -07:00
Tim Abbott c0beeebc3e zulint: Move printer.py module to tools/zulint.
At this point, tools/linter_lib is intended to be the directory of
Zulip-specific code.
2018-08-04 19:53:53 -07:00
Tim Abbott 55ba434ecb zulint: Extract LinterConfig class to zulint. 2018-08-04 19:53:53 -07:00
Tim Abbott f74bb8a9b2 zulint: Extract run_parallel and do_lint. 2018-08-04 19:53:53 -07:00
Tim Abbott acf8ec492d zulint: Extract common linter argument parsing logic.
The remaining code is Zulip-specific business logic.
2018-08-04 19:53:53 -07:00
Tim Abbott 1fcc2a6ea4 zulint: Move lister.py to tools/zulint.
This is preparatory refactoring work for being able to extract the
linter as an external project called "zulint".
2018-08-04 19:53:53 -07:00
Joshua Pan 5303281dfb tests: Fully cover hash_util.js. 2018-08-04 17:56:02 -07:00
Abhilash Verma fd9652a9d7 settings_emoji: Make strings more user-friendly. 2018-08-04 09:37:48 -07:00
Steve Howell c8898e1dc8 refactor: Move by_conversation_and_time_uri to hash_util.
This removes the 100% coverage on hash_util, but we are
pretty careful about not messing with this code.
2018-08-04 09:32:27 -07:00
Anders Kaseorg d79555e9be update-locked-requirements: Fix shellcheck warnings.
In tools/update-locked-requirements line 66:
compile_requirements requirements/prod.in $OUTPUT_BASE_DIR/prod.txt
                                          ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 67:
compile_requirements requirements/dev.in $OUTPUT_BASE_DIR/dev.txt
                                         ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 68:
compile_requirements requirements/mypy.in $OUTPUT_BASE_DIR/mypy.txt
                                          ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 69:
compile_requirements requirements/docs.in $OUTPUT_BASE_DIR/docs.txt
                                          ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/update-locked-requirements line 70:
compile_requirements requirements/thumbor.in $OUTPUT_BASE_DIR/thumbor.txt py2
                                             ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg 3e7bf9be9c production-helper: Fix shellcheck warnings.
In tools/travis/production-helper line 24:
if ! apt-get dist-upgrade -y $APT_OPTIONS; then
                             ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/travis/production-helper line 26:
    apt-get dist-upgrade -y $APT_OPTIONS
                            ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg e05a56aa8b test-migrations: Fix shellcheck warnings.
In tools/test-migrations line 18:
    echo "$new_auto_named_migrations" | sed 's/\[[x ]\] /  /'
    ^-- SC2001: See if you can use ${variable//search/replace} instead.

In tools/test-migrations line 27:
    echo 'ERROR: Migrations are not consistent with models!  Fix with `./tools/renumber-migrations`.'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg c4ed4bc021 test-install: Fix shellcheck warnings.
In tools/test-install/destroy-all line 31:
  | while read c
          ^-- SC2162: read without -r will mangle backslashes.

In tools/test-install/install line 57:
    installer_dir="$(readlink -f $INSTALLER)"
                                 ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/test-install/lxc-wait line 30:
for i in {1..60}; do
^-- SC2034: i appears unused. Verify use (or export if used externally).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg e1d770d451 test-documentation: Fix shellcheck warnings.
In tools/test-documentation line 6:
    echo -e "\e[${color_code}m${message}\e[0m" >&2
             ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".
                                        ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".

In tools/test-documentation line 41:
scrapy crawl_with_status documentation_crawler $loglevel
                                               ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg bb51503907 test-all-docker: Fix shellcheck warnings.
In tools/test-all-docker line 7:
source /home/zulip/.bash_profile
^-- SC1091: Not following: /home/zulip/.bash_profile: openBinaryFile: does not exist (No such file or directory)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg c93f53b463 test-all: Fix shellcheck warnings.
In tools/test-all line 7:
TEMP=`getopt -o f --long force -- "$@"`
     ^-- SC2006: Use $(..) instead of legacy `..`.

In tools/test-all line 24:
    echo "Running $@"
                  ^-- SC2145: Argument mixes string and array. Use * or separate argument.

In tools/test-all line 26:
        printf "\n\e[31;1mFAILED\e[0m $@\n"
               ^-- SC2059: Don't use variables in the printf format string. Use printf "..%s.." "$foo".
                ^-- SC1117: Backslash is literal in "\n". Prefer explicit escaping: "\\n".
                  ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".
                                ^-- SC1117: Backslash is literal in "\e". Prefer explicit escaping: "\\e".
                                      ^-- SC2145: Argument mixes string and array. Use * or separate argument.
                                        ^-- SC1117: Backslash is literal in "\n". Prefer explicit escaping: "\\n".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg e9656564c2 start-dockers: Fix shellcheck warnings.
In tools/start-dockers line 7:
source /home/zulip/.bash_profile
^-- SC1091: Not following: /home/zulip/.bash_profile: openBinaryFile: does not exist (No such file or directory)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg f4c4e857ff generate-test-credentials: Fix shellcheck warnings.
In tools/setup/generate-test-credentials line 10:
password=$(./manage.py print_initial_password "$email" | fgrep "$email" | awk '{ print $2 }')
                                                         ^-- SC2197: fgrep is non-standard and deprecated. Use grep -F instead.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg af64bff9d4 reset-to-pull-request: Fix shellcheck warnings.
In tools/reset-to-pull-request line 25:
    git fetch "$remote" +"pull/$request_id/head":"$target_ref"
                                                ^-- SC2140: Word is of the form "A"B"C" (B indicated). Did you mean "ABC" or "A\"B\"C"?

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg c669626666 provision: Fix shellcheck warnings.
In tools/provision line 13:
FAIL="\033[91m"
      ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/provision line 14:
WARNING="\033[93m"
         ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/provision line 15:
ENDC="\033[0m"
      ^-- SC1117: Backslash is literal in "\0". Prefer explicit escaping: "\\0".

In tools/provision line 19:
PARENT_PATH=$( cd "$(dirname "${BASH_SOURCE}")" ; pwd -P )
                              ^-- SC2128: Expanding an array without an index only gives the first element.

In tools/provision line 32:
if [ $failed = 1 ]; then
     ^-- SC2086: Double quote to prevent globbing and word splitting.

In tools/provision line 49:
    echo 'or just close this shell and start a new one (with Vagrant, `vagrant ssh`).'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg d7c7e8dd7c optimize-svg: Fix shellcheck warnings.
In tools/optimize-svg line 3:
if [ `node_modules/.bin/svgo -f static/images/integrations/logos | grep -o '\.[0-9]% = ' | wc -l` -ge 1 ]
     ^-- SC2046: Quote this to prevent word splitting.
     ^-- SC2006: Use $(..) instead of legacy `..`.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:27 -07:00
Anders Kaseorg cd451491ff find-unused-css: Fix shellcheck warnings.
In tools/find-unused-css line 5:
    if [ $(git grep "$n" | grep -v '^static/styles/zulip.css' | wc -l) -eq 0 ]; then
         ^-- SC2046: Quote this to prevent word splitting.
                           ^-- SC2126: Consider using grep -c instead of grep|wc -l.

In tools/find-unused-css line 6:
        echo $n
             ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg b920d19358 fetch-rebase-pull-request: Fix shellcheck warnings.
In tools/fetch-rebase-pull-request line 15:
git checkout -B "review-${request_id}" $remote/master
                                       ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Anders Kaseorg e7e25fb70b do-destroy-rebuild-test-database: Fix shellcheck warnings.
In tools/do-destroy-rebuild-test-database line 6:
"`dirname "$0"`/../tools/setup/generate-fixtures" --force
 ^-- SC2006: Use $(..) instead of legacy `..`.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-08-03 09:15:26 -07:00
Rhea Parekh ee37866687 import: Add gitter import file in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Rhea Parekh b8e1e8b31d import: Add slack import files in zerver/data_import directory. 2018-08-01 11:52:14 -07:00
Anders Kaseorg 510c97d861 scripts: Use shell quoting when displaying commands to be run.
This way, commands with arguments containing whitespace or
metacharacters are unambiguously readable.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:08 -07:00
Anders Kaseorg de5e3d3fbd tools/review: Don’t pretend to emulate shell=True either.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:08 -07:00
Anders Kaseorg 9a99d76296 tools: Remove pointless uses of shell=True.
Which, not coincidentally, is all of them.  Don’t use shell=True.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:06 -07:00
Steve Howell 95d136ca5e Extract search_util.js module.
We probably should have done this a while ago, even
though these functions are pretty tiny.  The goal here
is to make it easier to have more consistent search
semantics.

Our first use case is subs.js.  In this case we
are able to decouple a bit of generic string
matching from the subs-specific code.
2018-07-30 11:25:32 -07:00
Greg Price 4ba8015002 tools/reset...: Add experimental option to record the PR as a ref.
I often find myself looking manually through the reflog of `master` to
find a commit I previously reset to with tools/reset-to-pull-request .
Sometimes I want to see a previous version of a PR I'm reviewing a
revised version of; sometimes to look at two related PRs together.

So, here's a feature to automate that by saving each PR branch in its
own ref, with a name like `refs/remotes/pr/1234` -- or `pr/1234`, as
you'd normally refer to it.

To enable this, set the new config option:
  $ git config zulip.prPseudoRemote pr
(Or you can pick another name.)

The reason I hesitate to just make this the behavior for everyone
immediately is that the resulting `pr/1234` refs will naturally
accumulate and may clutter up the view -- and because with the
`refs/remotes/` style of name I've chosen, it requires a bit of
Git plumbing to clean them up.  (Use `git update-ref -d`.)
I'll play with it and iterate; comments welcome from other willing
early adopters.
2018-07-27 23:47:32 -07:00
Harshit Bansal cf5b2b4815 emoji: Change emoticon mapping for `:)`, `(:` and `:(`.
See discussion on CZO:
https://chat.zulip.org/#narrow/stream/101-design/subject/emoji.20picker/near/617811
2018-07-26 11:17:03 -07:00
Anders Kaseorg 19f37fb3d4 tools/test-queue-worker-reload: Avoid shelling out for touch.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Anders Kaseorg d3f8f74ac4 tools/test-locked-requirements: Avoid shelling out for cp, mkdir.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Anders Kaseorg dfed7da911 tools/lib/provision.py: Avoid shelling out for mkdir, touch.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-25 16:54:46 -07:00
Aditya Bansal 0042694e24 stylelint: Start using stylelint to lint our css.
Tweaked by tabbott to document it.

Fixes #8894.
2018-07-25 15:03:35 -07:00
Priyank Patel 8a012b9e7f Revert "webpack: Combine archive-style bundle into archive."
This reverts commit d1d9d63841, and fixes
the fonts in webapp being overwritten by bootstarap due to webpack bundling.
2018-07-24 16:00:26 -07:00
Greg Price 79dfe6e0ae push-to-pull-request: Fix shell usage in an error path.
Errors go to stderr, not stdout.

While we're here, match the message itself to the others in
this script.
2018-07-23 23:19:27 -07:00
Greg Price 26d0904436 push-to-pull-request: Give a nice error if `jq` missing. 2018-07-23 23:19:27 -07:00
Priyank Patel d1d9d63841 webpack: Combine archive-style bundle into archive. 2018-07-23 15:19:47 -07:00
Priyank Patel f8398238a9 webpack: Migrate archive bundle to webpack.
Tested by visiting localhost:9991/archive/streams/1/topics/denmark3.
2018-07-23 15:19:44 -07:00
Harshit Bansal c0b0fb7cce emoji: Move `EMOTICON_CONVERSIONS` mapping to build_emoji infra.
This commit closes a long pending issue which involved moving the
`EMOTICON_CONVERSION` mapping to build_emoji infrastructure so
that there is only one source of truth. This was pending from the
time when this feature was implemented.
2018-07-23 12:35:08 -07:00
Joshua Pan 565fd75661 tests: Test closing alert word status msg. 2018-07-23 14:53:49 -04:00
Tim Abbott 2918cc92f3 search: Duplicate HTML template content for search area.
This is preparation for being able to work on the search pills feature
without making any user-facing changes until we're ready to enable it.
2018-07-23 11:29:10 -07:00
Rhea Parekh 3a6921e54b gitter import: Add gitter data conversion script. 2018-07-23 08:36:30 -07:00
Harshit Bansal bf70955c0c emoji: Update `emoji-datasource` packages.
This commit updates the `emoji-datasource` packages to version 4.0.4.
This update brings following changes to emoji infra:

1: Fix for the bleeding sprite sheets.

2: The category of some emojis has been changed. Categorywise breakup of
   net gain or loss is as follows:
    Travel & Places: 58 (gain)
    Symbols: 47 (loss)
    Smileys & People: 52 (gain)
    Objects: 11 (loss)
    Food & Drink: 3 (gain)
    Animals and Nature: 46 (gain)
    Activities: 9 (loss)

3: There were some changes in the image farm of the package which were
   breaking our old emoji farm. I fixed them by modifying the remapped
   emoji map.

Fixes: #8235.
2018-07-23 08:04:58 -07:00
Harshit Bansal 6c28a60db0 emoji: Hardcode octopus emoji.
Google emojiset's octopus is really cute and whole Zulip community
loves it. So using a CSS hack, we hardcode octopus emoji to use image
from Google's emojiset only irrespective of the choosen emojiset.
2018-07-23 08:04:58 -07:00
Harshit Bansal 3f73826ffa emoji: Extract `get_emoji_code()`. 2018-07-23 08:04:58 -07:00
Harshit Bansal 69eaa2de67 emoji: Move `remapped_emojis` list to emoji_setup_utils.py. 2018-07-23 08:04:58 -07:00
Joshua Pan 015bc8b01e tests: Reach 100% coverage for keydown_util.
This commit pretty much just gets line coverage.
2018-07-23 08:42:11 -04:00
Joshua Pan 0596632457 lint: Exclude frontend_tests/node_tests from $().text rule. 2018-07-20 12:13:56 -07:00
Steve Howell b9c9dd9b5a lint: Turn off the CSS checker (officially).
Our CSS checker globs for .css files.  Since the
SCSS cutover, it has been a no-op, so there's no
sense launching it.  See #8894 for details on
future plans.
2018-07-20 10:03:28 -04:00
Anders Kaseorg 8698198452 tools/update-authors-json: Avoid shelling out for cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 6a8f0165ee tools/test-help-documentation: Avoid shelling out for mkdir.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg f01942e510 tools/test-emoji-name-scripts: Avoid shelling out for cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 07f5b81406 tools/minify-js: Avoid shelling out for mkdir, cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Anders Kaseorg 2d12b5b3d9 tools/diagnose: Avoid shelling out for pwd.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-19 10:43:37 -07:00
Tim Abbott 811ac718d6 build_emoji: Document a quirk of generate_sprite_css_files. 2018-07-13 21:19:26 +05:30
Harshit Bansal ec1ce08523 emoji_setup_utils: Migrate to python3 type annotations. 2018-07-13 21:19:26 +05:30
Harshit Bansal f6fe5d6266 build_emoji: Migrate to python3 type annotations. 2018-07-13 21:19:25 +05:30
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Tim Abbott 7a86e83a6f build_emoji: Use with for writing to files.
This is a nice code cleanup.
2018-07-13 21:07:08 +05:30
Tim Abbott 85243c525d build_emoji: Pass around emoji_map data structure less.
It wasn't used in either of these functions.
2018-07-13 21:04:37 +05:30
Harshit Bansal b9f974ba61 tools: Add a tool to validate emoji name listing scripts.
This tools emoji name import/export tooling by verifying that no
information gets modified in a round-trip conversion.
2018-07-13 20:53:25 +05:30
Harshit Bansal 628e868d1e tools: Add `export_emoji_names_to_csv`.
This tool is used for exporting `emoji_names.py` to a csv file.
2018-07-13 20:53:08 +05:30
Harshit Bansal 05f85eb94d emoji: Add emoji_names.py file.
Credits to @rishig, Alice Lai, and @rntharu for naming all the emoji.

Names are inspired by iamcal, gemoji, and unicode names, sources like
emojipedia and iemoji, google search results for articles about emoji,
and emoji usage on twitter.
2018-07-13 20:53:07 +05:30
Harshit Bansal ae17a15b29 tools: Add generate_emoji_names_table tool.
This tool generates a listing of the emojis as per the current cache
in use.
2018-07-13 20:53:07 +05:30
Harshit Bansal fef5b43e22 minor: Fix directory name label in 'import_emoji_names_from_csv'. 2018-07-13 20:53:02 +05:30
Max Nussenbaum 8fd900626d billing: Style billing page.
This adds styles to /billing.
2018-07-13 18:30:38 +05:30
Tim Abbott 3cc93fd721 setup_venv: Fix missing libssl-dev dependency.
We were already correctly including libssl-dev in Zulip's dependencies
in development environment provisioning, but (at least now) it's
needed to build certain Python packages like pycurl when building a
Zulip virtualenv in production.  I haven't investigated why we didn't
need this on Ubuntu, but one possible reason would be that some other
library in our dependencies list happens to depend on it on Ubuntu.

We fix this by moving the dependency over to the shared
VENV_DEPENDENCIES list.

Fixes part of #9946.
2018-07-13 18:00:38 +05:30
Harshit Bansal 8ada7cfe5b tools: Add import_emoji_names_from_csv tool. 2018-07-12 18:05:02 +05:30
Greg Price 36cf898589 mypy: Add a place to put type stubs.
This will allow us to begin to add our own stubs for external
libraries.  Writing stubs can be surprisingly little work to do, and
can have high leverage in keeping our type annotations high-quality.
2018-07-12 14:10:09 +05:30
Yago González a3d42d9901 test-api: Pass non-admin client to the test helpers. 2018-07-11 23:54:00 +05:30
Cynthia Lin 7d9281fa6c simplebar: Add simplebar dependency for help page scrolling. 2018-07-11 20:04:55 +05:30
Puneeth Chaganti 3dace27ed6 tools: Only files starting with same migration number conflict. 2018-07-10 21:09:34 +05:30
Shayan Toqraee 0757d022f5 messages: Add support for right-to-left messages.
This implements right-to-left message automatic detection support in
the compose box as well as the message feed.  Full unit tests and
support in the message-editing UI are for future work (as are
potentially more fancy things like supporting things like
right-to-left multi-word names for users/streams/etc.).

Fixes #3123.
2018-07-10 10:47:56 +05:30
Rishi Gupta 16334a1ba7 billing: Update billing system. 2018-07-09 14:33:07 +05:30
Aditya Bansal b6ba162769 generate-fixtures: Extract zulip_test_template creation as a function.
This will be helpful in the upcoming changes which will make use
of this extracted function to re-create zulip_test_template after
migrating zulip_test db so that we have latest schema in tests.
2018-07-09 09:11:48 +05:30
Armaan Ahluwalia ba6543b850 webpack: Fix missing exposed modules left out of transition.
This commit fixes some modules that were erroneously left out while
transitioning app.js to webpack. This commit exposes them using
expose-loader or setting them directly to window.
2018-07-07 08:00:34 +02:00
Shubham Dhama ced6f967bc voting_widget: Increase node test coverage to 100%. 2018-07-06 11:30:12 -04:00
Tim Abbott 0bd139af0c webpack: Fix missing expose for toMarkdown. 2018-07-05 13:24:19 +02:00
Armaan Ahluwalia 54d3d8e8b3 webpack: Transition app.js to be compiled by webpack.
This commit moves all files previously under the 'app' bundle in
the Django pipeline to being compiled by webpack under the 'app'
entry point. In the process, it moves assets under the app entry
to a file called app.js that consumes all relevant css and js files.

This commit also edits the webpack config to be able to expose certain
variables for third party libraries that are currently required by
some modules. This is bad coding form and should be refactored to
requiring whatever dependencies a module may have; we're just
deferring that to the future to simplify the series of transitions we
need to do here. The variable exposure is done using expose-loader in
webpack.

The app/index.html template is edited to override the newly introduced
'commonjs' block in the base template. This is done as a temporary
measure so as not to disrupt other pages on the app during the transition.

It also fixes the value of the 'this' context that was being inferred
as window by third party libraries. This is done using imports-loader
in the webpack config.  This is also messy and probably isn't how we
want things to work long term.
2018-07-05 11:03:08 +02:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Shubham Dhama b938523592 widgets: Increase node tests coverage to 100%. 2018-07-03 08:48:49 -04:00
Yashashvi Dave 2b478268f3 third: Add sortablejs library. 2018-07-01 02:05:00 -07:00
Vishnu Ks e1bf749d3c provision: Run build_pygments_data only if required. 2018-06-23 00:15:39 +05:30
Vishnu Ks 109fa85614 provision: Rename file_hash_updated to file_or_package_hash_updated.
Check for changes in package version as well along
with the files.
2018-06-22 23:40:31 +05:30
Vishnu Ks 431d9d6076 provision: Run generate-custom-icon-webfont only if required. 2018-06-22 23:40:31 +05:30
Shubham Padia 6853530eba search: Add search_pill.js with basic operations.
Adds search_pill.js to the static asset pipeline. The items
for search pill contain 2 keys, display_value and search_string.
Adding all the operator information i.e the operator, operand and
negated fields along with the search_string and description was tried out.
It was dropped because it didn't provide any advantage as one had to
always calculate the search_string and the description from the operator.
2018-06-22 09:05:36 -04:00
Shubham Dhama c167227ddc search: Add 100% coverage for search.js. 2018-06-22 07:58:06 -04:00
Greg Price 0462e85d02 tools: Make remote configurable in {reset,push}-to-pull-request.
The appropriate name for the remote pointing at the repo we maintain
may be `upstream` for most of our repos... but not when we're
downstream of someone else, e.g. for react-native.  So, make it easy
to configure per-repo.
2018-06-21 17:28:21 -07:00
Vishnu Ks 59169cc302 provision: Run inline-email-css only if required.
This saves a few hundred milliseconds on provision time.

Edited by tabbott to include email.css in the dependencies.

Fixes #9468.
2018-06-18 06:58:55 -07:00
Vishnu Ks 7b8e79ae48 provision: Refactor hashing of compilemessages into a function.
This allows it to be reused for other tools.

Edited by tabbott to remove the use of "compilemessages" in variable
names.
2018-06-18 06:55:36 -07:00
Raymond Akornor e82581e5bd provision: Close opened files automatically. 2018-06-12 13:38:58 -04:00
Aditya Bansal 1205e02c64 provisioning: Don't rebuild DB if running migrations is sufficient.
This results in a significant optimization in the performance of
re-provisioning Zulip if all that you're doing is rebasing onto a
newer version of master (which just adds new migrations).

The change carries some risk of generating unpleasant-to-debug
situations, because if we merge a buggy migration and then later fix
it, some clients may not have a properly migrated database (and also,
this changes how populate_db commutes with migrations).  But it seems
worth it, given how much time is currently wasted by not having this.

Fixes: #9512.
2018-06-06 13:43:59 -07:00
Aditya Bansal 65dc80fe9d test_fixtures: Add run_generate_fixtures_if_required function.
In this commit we are adding run_generate_fixtures_if_required,
a new function which is meant to de-duplicate a bit of code
between test-server and test-backend which is essentially
responsible for rebuilding the test database if that was required.
2018-06-06 13:37:35 -07:00
Aditya Bansal f7c11d1747 test_fixtures: Refactor to have template_database_status API.
In this commit we are essentially just refactoring the function
is_template_database_current to be called template_database_status
and adjusting the return values accordingly.
This is essentially a preparatory commit for the upcoming commits
which will essentially enable us to not throw away entire DB and
rebuild from scratch if only running migrations could do the job.
2018-06-06 13:37:35 -07:00
Tim Abbott e9312b921c tests: Disable test-queue-worker-reload test.
This test is low value (it's for testing a very specific feature/bug),
is slow, and apparently flakes in CI occasionally.
2018-06-05 09:24:03 -07:00
Tim Abbott 4cdcb7e0d3 webpack: Use more consistent bundle naming scheme.
This way, both our JS and CSS files using .chunkhash.ext as their
naming scheme.
2018-06-03 16:49:59 -07:00
Tim Abbott 7813376934 webpack: Use filenames based on hashes.
This should avoid us creating duplicate webpack bundles every time we
do a deployment, even if none of the files in the bundles themselves
have changed at all.
2018-06-03 16:49:43 -07:00
Tim Abbott 714ad67db1 storage: Move library code into zerver/lib.
There's no particularly good reason for this to have been a top-level
file.
2018-06-03 16:02:02 -07:00
Shubham Padia e32dd53cd9 lint: Check for occurrences of `.includes` except in `frontend_tests/`.
Adds a custom check to js_rules in `/tools/lint/lib/custom_check.py`.
2018-06-03 14:30:22 -07:00
Greg Price f686d55fa0 push-to-pull-request: Broaden one error message.
I discovered this case just now while testing other changes.
2018-06-01 16:52:18 -07:00
Greg Price 954c71e90c push-to-pull-request: Use `jq -r` to simplify a test slightly.
This option (aka `--raw-output`) prints a string as itself, rather
than JSON-encoded; which makes it fit a bit better in a shell script,
saving us a layer of quoting.
2018-06-01 16:52:18 -07:00
Greg Price 8e81ca0fb2 push-to-pull-request: Fix regex usage in parsing remote.
This replaces ad4617c95 with a different fix for the same issue:
instead of stripping the `.git` off separately, we can just correct
the regex, using `+?` to fix our stepping in a classic regex pitfall.
2018-06-01 16:52:18 -07:00
Tim Abbott 751c602a0b update-prod-static: Don't copy/minify templates and styles.
This is a performance optimization: Rather than copying these files
into the `prod-static` directory and then deleting them, we just don't
copy them over in the first place.

For styles, it might have once been the case that this did something,
but we've moved them all to being managed by webpack some time ago.

For the js directory, I think it was never useful to copy and then
delete them; these files were always compiled via tools/minify-js,
and the raw JS files weren't needed, anyway.
2018-06-01 16:39:35 -07:00
Tim Abbott cefdf6f6d4 update-authors-json: Pretty-print output with sorted keys.
This makes working with this file for debugging/etc. much nicer.
2018-06-01 16:39:04 -07:00
Tim Abbott 60eae53327 push-to-pull-request: Provide better output for invalid PR.
Previously, we were very confusingly showing the error message for
permissions when the issue was a not-found error.
2018-06-01 16:00:27 -07:00
Tim Abbott ad4617c953 push_to_pull_request: Fix handling of .git.
This was throwing the "you can't access the repo" error for this
reason.
2018-06-01 16:00:27 -07:00
Tim Abbott 95f1f1d363 mypy: Remove logic for linecoverage report. 2018-06-01 11:14:32 -07:00
Vishnu Ks 69f8b7e8df circleci: Add a base image for Ubuntu Bionic. 2018-05-31 18:51:32 -07:00
Tim Abbott 5b5d1f3e87 update-prod-static: Rewrite to use the run() scripting tool.
This provides nice `set -x` style logging of which commands are
running when.
2018-05-31 16:45:51 -07:00
Sampriti Panda e95f972e24 migrate: Add do_batch_update method for running batch updates.
* Refactor pgroonga_0002 migration to use new method.
2018-05-31 13:00:34 -07:00
Tim Abbott a2c9517f8d test_server: Don't continuously recompile handlebars templates.
This changes run-dev.py to ensure that we have in fact compiled
handlebars templates before running webpack, which is the right model.

Future work will likely include running the handlebars compiler from
webpack, and thus eliminating this extra process.
2018-05-30 20:16:06 -07:00
Tim Abbott 5a35507a7a success-http-headers: Add X-Frame-Options. 2018-05-30 09:41:50 -07:00
Tim Abbott 0fabff6dda setup_venv: Clean up VENV_DEPENDENCIES using parsed lsb_release.
This is mostly a cleanup, but it should also save 50ms in the runtime
of create-production-venv.
2018-05-29 10:57:36 -07:00
Raymond Akornor 5d39a0f0fc scripts: Replace calls to lsb release with our own parsing.
This improves the performance of these operations, by saving a ~50ms
Python process startup.  While not a major performance improvement, it
seems worth it, given how often these commands get run.

Fixes #9571.
2018-05-29 10:57:36 -07:00
Robert Hönig afa806a0ce linter: Add rule to favor Botserver over botserver or bot server. 2018-05-29 07:27:36 -07:00
Tim Abbott 2f571f9a45 webpack: Move font and pygments CSS to common.css.
First, it's silly that these weren't in common.css in the first place,
since that meant these were a bunch of duplicated code, but
additionally, that meant that these weren't available on the
`/activity` page (or other pages that don't include the portico styles).

Fixes #9561.
2018-05-27 23:19:40 -07:00
Yago González b4084d30d2 emails: Move user role logic for followup_day1 to notifications.py. 2018-05-26 07:29:58 -07:00
Yago González 5b0b9feb7d i18n: Atomize translation tags.
Makes the i18n strings in this file much easier to translate by splitting
them into smaller chunks (which avoids having a lot of code in the tagged
strings), and adds a string that was missing as well.
2018-05-26 07:06:43 -07:00
Tim Abbott 20aa3233b0 lint: Improve the check for realm->organization to support identifiers. 2018-05-25 12:03:28 -07:00
Tim Abbott 07b4da7659 Revert "node tests: Change timezone for node tests."
This reverts commit 311ddefd32.

This was causing tests to fail in some but not other environments.
2018-05-25 08:52:13 -07:00
Vishnu Ks abade460a9 tools: Add tool for listing outdated pip packages. 2018-05-24 15:16:02 -07:00
Aditya Bansal f9aa012486 linter: Fix issue with check-templates output.
We fix the issue of check-templates spitting out diff between
expected and found indentation of a file before mentioning the
error message and the file name. Basically stuff was being in the
wrong order despite the fact that in code stuff was happening in the
correct order ie, first print the error message along with the filename
and then the actual diff between expected and found file indentation.

Fixes: #9533.
2018-05-24 12:42:32 -07:00
Vishnu Ks 54a002c2e2 requirements: Upgrade pyflakes to 2.0.0.
We fix a few errors that only the new version finds.
2018-05-24 11:31:36 -07:00
Steve Howell 1b57e568ff Add "zform" functionality to the web client.
A "zform" knows how to render data that follows our
schema for widget messages with form elements like
buttons and choices.

This code won't be triggered until a subsequent
server-side commit takes widget_content from
API callers such as the trivial chat bot and
creates submessages for us.
2018-05-24 09:30:22 -07:00
Steve Howell 435719c65b Add a schema checking concept to the web app.
This starts the concept of a schema checker, similar to
zerver/lib/validator.py on the server.  We can use this
to validate incoming data.  Our server should filter most
of our incoming data, but it's useful to have client-side
checking to defend against things like upgrade
regressions (i.e. what if we change the name of the field
on the server side without updating all client uses).
2018-05-24 09:30:22 -07:00
Steve Howell d018995409 node tests: Make the failure message more obvious.
I mistakenly pushed a PR when my tests failed.  I ran with
the coverage option, so I saw this brightly colored summary
report that distracted me from the failure message.

This adds a couple newlines and some all caps.
2018-05-24 09:30:22 -07:00
Sivakar Sithamparanathan 311ddefd32 node tests: Change timezone for node tests.
The timezone environment variable was set to UTC initially. It was
changed to something other than UTC so that any local vs UTC
conversion issues will manifest in the tests.

Fixes: #5105.
2018-05-23 13:29:19 -07:00
Aditya Bansal 8cfb437677 provision: Don't run create_realm_internal_bots management command.
We essentially stop running create_realm_internal_bots during
every provisioing and move its operations to run from populate db.
In fact to speed things up a bit we actually make populate db call the
funcs which create_realm_internal_bots calls behind the scenes.

Fixes: #9467.
2018-05-23 13:34:52 +05:30
Aditya Bansal a27c9a08dd generate-fixtures: Export DJANGO_SETTINGS_MODULE=zproject.test_settings.
This is required because the --settings=zproject.test_settings param
doesn't work with migrate or the dumpdata management commands. Thus
untill now if one ran just this tool ended up with test database not
properly setup. We never noticed this because test-backend ran this
tool again (after exporting DJANGO_SETTINGS_MODULE) thus making the
tool work this time.
2018-05-23 11:32:40 +05:30
Sampriti Panda 51c71fd3fd provision: Add libfontconfig1 to bionic apt dependencies. 2018-05-22 22:37:04 -07:00
Shubham Dhama ec4e5e73ab org settings: Make time limit input elements inline to dropdown.
(To make the capitalization pass we added `N` to ignored phrases.)
2018-05-22 19:32:44 -07:00
Greg Price 5e89c33e7f tools: Add push-to-pull-request tool.
I've often done this by hand -- basically typed out the last line,
with the variables found from looking at the PR page in a browser.
Seems nicer for both us maintainers and the contributor, in particular
because the PR gets marked as merged, instead of closed.  But it's a
bit of a pain, and I do it maybe half the time or less; plus it's kind
of a subtle GitHub feature, and as a result I think other maintainers
of Zulip repos do this approximately never.

I've always figured this couldn't be hard to automate; today I decided
to take the 45 minutes to look up how, write out the script, QA it,
write up a nice usage message and some comments, and commit it. :)
2018-05-22 18:33:51 -07:00
Armaan Ahluwalia 1525e92058 logging/errors/webpack: Improve error stack traces with blueslip.
This commit improves the output that blueslip produces while
showing error stack traces on the front-end. This is done by
using a library called error-stack-parser to format the stack
traces.

This commit also edits the webpack config to use a different
devtool setting since the previous one did not support sourcemaps
within stack traces. It also removes a plugin that was obviated
by this change.
2018-05-22 14:36:41 -07:00
Eeshan Garg e0ef831993 webhooks: Migrate to UnexpectedWebhookEventType.
This improves test coverage for a lot of our webhooks that relied
on ad-hoc methods to handle unexpected event types.

Note that I have deliberately skipped github_legacy, it isn't
advertised and is officially deprecated.

Also, I have refrained from making further changes to Trello, I
believe further improvements to test coverage should be covered
in separate per-webhook commits/PRs.
2018-05-22 08:30:19 -07:00
Eeshan Garg 3ed20589f2 webhooks: Add generic exception for unexpected webhook events.
UnexpectedWebhookEventType is a generic exception that we may
now raise when we encounter a webhook event that is new or one
that we simply aren't aware of.
2018-05-22 08:30:19 -07:00
Michael J. Sullivan c81b276c1c mypy: Add basic support for invoking dmypy to run-mypy. 2018-05-21 22:41:08 -07:00
Aditya Bansal e41d32a15a reminders: Add various design improvements for the UI/reminder msg. 2018-05-21 09:07:37 -07:00
Tim Abbott c6fda61227 block_internet: Make compatible with httpretty library.
If httpretty is enabled, we can let it handle HTTP requests.
2018-05-20 23:31:42 -07:00
Tim Abbott a170bd70af provision: Don't block installing with Ubuntu bionic.
Now that we have tsearch_extras packages uploaded, this mostly works.

There's a few issues being debugged in #9460; they should be fixed
soon, and regardless, merging this will make it easier to develop.
2018-05-20 21:21:11 -07:00
Tim Abbott 06ed55e45a webpack: Add ALLOWED_HOSTS list for zulipdev.com and friends.
This makes it possible to again use the *.zulipdev.com domains in the
development environment.

Ideally, we'd also read REALM_HOSTS to make this more flexible.
2018-05-20 18:12:28 -07:00
Max Nussenbaum 0ff2051982 portico: Add tour section to homepage.
This adds a tour of Zulip to the bottom of the homepage.

In order to get the carousel nave, we use Bootstrap 2 from a CDN on
this page; this isn't ideal in the medium term, but upgrading
Bootstrap across the project is too much work for now.
2018-05-20 15:04:23 -07:00
Tim Abbott 4f8e09d5af provision: Fix buggy management of apt_dependencies_hash.
Apparently, we were incorrectly appending each new hash onto the end
of the file, basically resulting in every run of provision being
treated as a miss for this cache.

Fixing this saves about 4s (over 1/3) of the no-op provision time.
2018-05-20 14:22:32 -07:00
Tim Abbott 235002a549 provision: Don't install lsb-release if already installed.
This early bootstrapping step should be rearely required, and it saves
about 0.5s in our no-op provision time.
2018-05-20 14:22:32 -07:00
Shubham Padia 0824308a7a tools: Allow optional arguments after file arguments in test_backend.
Fixes #9233.
Uses nargs='*' instead of nargs='argparse.REMAINDER'.
nargs='argparse.REMAINDER' gathers remaining terms as arguments
even if it is an option e.g --coverage, while '*' gathers all the
command-line arguments until the next option is encountered.
2018-05-18 17:08:40 +05:30
Tim Abbott 0dcf873370 third: Rename thirdparty-fonts.css for better clarity.
Now it's clear that this is just legacy fontawesome styles.
2018-05-17 04:58:30 -07:00
Aman Ghebreweldi b05a702285 third: Use font awesome css file from NPM package. 2018-05-17 04:58:30 -07:00
Steve Howell b48f052b0c Add basic widgets framework (JS side). 2018-05-16 15:13:33 -07:00
Steve Howell 1f0ffdc415 Add widget.py to enable widgets on the server side.
The only slash command implemented in this initial
version is an extremely crippled version of a
"/stats" slash command that reports that you are
running 1 server.
2018-05-16 15:13:33 -07:00
Joshua Pan a67ba617d6 minor: Fix typo in tools/lint. 2018-05-16 13:19:28 -04:00
Tim Abbott 00861b910d test_presence: Add tests for seconds_usage_between.
This isn't widely used, but is a nice way to verify the logic further.
2018-05-15 18:37:49 -07:00
Tim Abbott e39a3e07d7 coverage: Mark zerver/lib/debug.py as not needing coverage. 2018-05-15 17:19:14 -07:00
Tim Abbott fd24eac8bb coverage: Exclude zerver/lib/parallel.py from coverage. 2018-05-15 17:19:14 -07:00
Tim Abbott c46149efcb migrate: Remove obsolete act_on_message_ranges.
This has a cool structure, but it's written against the long-dead
South API, and we can always pull it out of the Git history if we want
to use this approach in the future.
2018-05-15 17:19:14 -07:00
Tim Abbott 3e04aa99fa fonts: Use source-sans-pro CSS from upstream. 2018-05-15 16:22:44 -07:00
Greg Price 6dbd90835b mypy: Move remaining exclusion list to config file.
This leaves the wrapper script with very little left to do!

The main thing left is finding scripts by searching for shebang lines;
mypy itself would happily do the search for importable Python files.
2018-05-15 18:14:33 -04:00
Tim Abbott badfec2b49 coverage: Don't include settings files in test coverage. 2018-05-15 13:55:01 -07:00
Tim Abbott 191e6dc13a create_user: Remove user_profile_id option.
This hasn't been used in years, had no test coverage, and doesn't have
a clear use case.
2018-05-15 13:51:56 -07:00
RobbieClarken f81b936727 zerver/tests: Require 100% test coverage of zerver/lib/upload.py.
The last line here is impossible to test.

Fixes #4489.
2018-05-15 10:52:20 -07:00
Aman Ghebreweldi 83ee8211a8 third: Get sorttable package from npm.
We didn't have any local changes, so there's no need to have it
vendored in the repository.
2018-05-15 09:25:48 -07:00
Steve Howell 4f0b3fc1b6 test-js-with-node: Exit more cleanly for errors.
We don't need a Python stack trace when the node
piece fails, so just give a non-distracting error
message.
2018-05-15 08:24:44 -07:00
Joshua Pan df84e1d7eb tests: Reach 100% coverage for zerver/liv/management.py.
Cleaned up add_user_list_args(). The "help" and
"all_users_help" have all default values. As noted in
an earlier commit, "all_users_help" is always passed in,
so we can get rid of "all_users_arg". We keep the default
for "all_users_help" so we don't have to change variable order
in function definition.
2018-05-14 10:46:21 -07:00
Darshan Markandaiah 607cab2a53 tests: Add 100% test coverage to zerver/apps.py.
Tweaked by tabbott to rename to test_cache.py and remove the
sender_name argument (Since it was kinda confusing).
2018-05-14 10:36:10 -07:00
Vishnu Ks b73603a97c droplets: Do shutdown command along with clear history.
Otherwise shutdown command would be still present in the
history.
2018-05-14 08:12:43 -07:00
Vishnu Ks 861001f7b3 droplets: Update snapshot id.
Upgraded zulip and python-zulip-api repos.
Installed flask in python-zulip-api.
2018-05-14 08:12:43 -07:00
Tim Abbott f5ea661c1f tools: Upgrade pycodestyle.
We have to add a few rules that we're ignoring for now.
2018-05-14 06:15:13 -07:00
Aditya Bansal 64678b459c linter: Add rule to lint against use of typing.Text and promote str.
This commit is the last in a series of commits which migrated our
entire codebase to use str instead of typing.Text.

Fixes: #9203.
2018-05-13 17:17:32 -07:00
Aditya Bansal addfde7374 tools: Change use of typing.Text to str. 2018-05-13 17:17:32 -07:00
Joshua Pan 073ecaac66 provision: Give concrete NFS error message on older OSX versions. 2018-05-13 08:47:19 -07:00
Tim Abbott ff2157c787 coverage: Exclude api_test_helpers.py from backend test coverage.
These files are tested by a different test suite, and in practice have
100% coverage; no need to double-measure it.
2018-05-09 20:49:13 -07:00
Tim Abbott 7e2841b358 bulk_create: Mark bulk_create_streams as nocoverage.
It's only used in `populate_db`, so it gets tested that way.
2018-05-09 20:49:12 -07:00
Steve Howell 27b7461e0a Revert "js: Implement DynamicText class."
This reverts commit 6e7305f784.

We never ending up using this class.
2018-05-08 12:13:55 -07:00
Vishnu Ks 49b3cb9da5 droplets: Remove the step to activate virtualenv manually.
I didn't had to activate this manually during the
last few droplet upgrades.
2018-05-07 08:42:18 -07:00
Vishnu Ks c684333051 droplets: Fix the command for clearing bash history.
history -c only clears the history from memory and
would be populated back during the next SSH session.
2018-05-07 08:42:18 -07:00
Vishnu Ks 18314e57f8 droplets: Add origin remote to python-zulip-api as well. 2018-05-07 08:42:18 -07:00
Vishnu Ks 13f6cbeefd droplets: Change droplet region from SFO1 to NYC3.
Since the region of base.zulipdev.org is
NYC3 we had to add SFO1 as an additional
region each time a snapshot of base droplet
is created. This is required as droplets
can be created in SFO1 only if there is
an image present in that region. Adding
of droplet image to SFO1 takes a lot of
time as well as cost 2X as we are storing
2 images. It's better to just create new
droplets in NYC3 instead. Alternatively we
can create a new base droplet in SFO1 if
we want all the droplets to be created in
SFO1.
2018-05-07 08:42:18 -07:00
Vishnu Ks d6c5635550 droplets: Update snapshot id. 2018-05-07 08:42:18 -07:00
Greg Price 9f3052b0ef mypy: Move remaining options on type-check semantics to config file.
This puts all of this config in one place, and also needs a lot fewer
lines to describe it; which, combined, makes it a lot clearer what our
normal config actually is.  (I'd been looking at this script for a few
minutes without realizing that we have `--disallow-untyped-defs` *on*
by default, not off.)

Experimenting with different values is still easy; just comment the
line in the config.
2018-05-06 19:33:55 -07:00
Greg Price 6b1e76c1b1 mypy: Move cache-dir to config file. 2018-05-06 19:33:55 -07:00
Greg Price 724e849e2b mypy: Move follow-imports to config file.
It's cleaner to have this configuration in a nice declarative
config file than embedded in a script.
2018-05-06 19:33:55 -07:00
Greg Price ff178bb27a mypy: Drop now-redundant `-i` option.
This used to be a synonym for `--incremental`.  Since mypy 0.590,
incremental mode is the default, and the flag is ignored; so we
can happily drop it.
2018-05-06 19:33:55 -07:00
Tim Abbott 41841221ee scripts: Remove obsolete zesty configuration.
Zesty already reached end-of-life, so we'll never support it.

And in one place, we add support for bionic.
2018-05-05 11:41:57 -07:00
Tim Abbott d178c53a10 provision: Add official support for provisioning directly on Debian. 2018-05-05 11:00:47 -07:00
Tim Abbott 8ea8bfe285 puppet: Add basic configuration for Ubuntu bionic. 2018-05-05 10:49:09 -07:00
Tim Abbott b193330474 provision: Add tsearch_extras package for stretch. 2018-05-05 10:49:09 -07:00
Tim Abbott 78f3cff151 provision: Fix unescaped reference to vagrant.
Previously, this would actually try to run `vagrant ssh`, rather than
printing it :(.
2018-05-05 10:48:37 -07:00
Tim Abbott 9d6233a457 capitalization: Add an exclude rule for zulip_org_id. 2018-05-03 22:37:24 -07:00
Armaan Ahluwalia cca10beb78 css/webpack: Moved archive styles to webpack bundle.
This commit moves the stylesheets under the archive bundle in
the Django pipeline to being compiled by webpack instead. It
also removes a remaining call to a portico stylesheet that no
longer exists.
2018-05-03 19:15:49 -07:00
Armaan Ahluwalia 54bf2a6231 css/webpack: Transition landing-page.css to webpack.
This commit transitions landing-page.css from the Django pipeline
to being compiled by webpack as landing-page.scss under the
'landing-page' and 'integration' bundles.
2018-05-03 19:15:23 -07:00
Armaan Ahluwalia 1c016e990d css/webpack: Transition common.css to webpack.
This commit transitions common.css from the Django pipeline
to being compiled by webpack under the common bundle.
2018-05-03 19:15:23 -07:00
Tim Abbott bd110ccb3c test-migrations: Add django_two_factor migrations to exclude list.
We don't control the names for these migrations.
2018-05-03 12:23:25 -07:00
Priyank Patel 1e1b72f6c8 webpack: Transition stats.css to use webpack.
The stats.scss file is added to activity bundle.
2018-05-03 08:08:23 -07:00
Priyank Patel c780bc33ba webpack: Transition activity.css to use webpack.
The activity.scss file is added to activity bundle.
2018-05-03 07:53:24 -07:00
Armaan Ahluwalia 5f7b47e20c css: Transition 'app.css' to SCSS.
This commit transitions all styles in app.css in the Django pipeline
to being compiled by webpack in an app-styles bundle, and renames the
various files to now be processed as SCSS.

To implement this transition, we move the old CSS file refernces in
settings.py and replace them with a bundle declared in
`webpack.assets.json` and includedn in the index.html template

Tweaked by tabbott to keep the list of files in `app.css` in
`webpack.assets.json`, and to preserve the ordering from the old
`settings.py`.
2018-05-02 17:13:16 -07:00
Armaan Ahluwalia 64dadae697 webpack: Add css-hot-loader to remove flash on unstyled content.
This commit removes the flash on unstyled content while in dev
mode that was caused by the use of style-loader. Instead it
enables mini-css-extract-plugin in dev in combination with
css-hot-loader which enables HMR for development.

This is because mini-css-extract-plugin does not currently support
HMR out of the box. It also adds a SourceMapDevtoolPlugin to enable
sourcemaps with css since mini-css breaks sourcemaps when used in
combination with the cheap-module-evel-source-map setting.

Related issues:
https://github.com/webpack-contrib/mini-css-extract-plugin/issues/34
https://github.com/webpack-contrib/mini-css-extract-plugin/issues/29
2018-05-02 16:55:58 -07:00
Aditya Bansal a62efd55df linter: Make duplicate html tag id detection work with archives.
We modify check-templates to check for duplicate id's in archive
templates and app templates separately. This means we are allowing
app and archive templates to potentially use same id's. This is
needed because we intend to re use some js from the main app and
having same id's help achieve that.
Note: We haven't up until this point actually added archive
templates. This commit is more of a preparatory commit for merging
the basic archive infra.
2018-05-02 15:23:33 -07:00
Armaan Ahluwalia 9f80418d12 webpack: Add support for png file imports in file-loader.
Adds support for importing png files using file-loader in webpack.
Changes the name of the output directory to be files instead of
fonts for better readability.
2018-05-02 09:45:04 -07:00
Armaan Ahluwalia fce6882eb9 webpack: Move the styles consumed by 5xx.html to webpack.
This commit removes the need for portico.css to be generated
by the Django pipeline and makes the error page use the css
file compiled by webpack instead.
2018-05-02 09:45:01 -07:00
Tim Abbott a4ff917789 emails: Fix spelling of "Unknown IP" and tag for translation. 2018-04-30 12:04:39 -07:00
Priyank Patel 50b13219a3 webpack: Combine both js and css into one portico bundle.
Combines, both portico js and css into one bundle. This for now solve
the issue of an empty js bundle being generated by webpack for the
portico-styles stylesheet.
2018-04-30 10:23:39 -07:00
Tim Abbott 7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
Steve Howell 605a90ce36 node tests: Reach 100% coverage for scroll_util.js. 2018-04-28 11:15:14 -07:00
Steve Howell 65d8eb3189 buddy list: Extract user_search.js.
This was a bit more than moving code.  I extracted the
following things:

        $widget (and three helper methods)
        $input
        text()
        empty()
        expand_column
        close_widget
        activity.clear_highlight

There was a minor bug before this commit, where we were inconsistent
about trimming spaces.  The introduction of text() and empty() should
prevent bugs where users type the space bar into search.
2018-04-28 11:15:14 -07:00
Armaan Ahluwalia fb0a421b8c webpack: Silence most webpack output for tests.
This commit adds a --quiet argument to tools/webpack which removes
the verbose output from webpack and replaces it with showing only
errors. It also makes tools/run-dev --tests use this argument while
running webpack for testing.

Tweaked by tabbott to clean up the code a bit.
2018-04-28 09:32:10 -07:00
Tim Abbott 699c4381f2 test-api: Add a PROVISION_VERSION --force option.
This brings this in line with the Casper and other tests.
2018-04-27 17:21:12 -07:00
Vishnu Ks e3314be114 emails: Add translation tags to confirm_new_email. 2018-04-27 11:59:36 -07:00
Armaan Ahluwalia 93ac40105f CSS: Move portico styles to webpack compilation.
static/styles/scss/portico.scss is now compiled by webpack
and supports SCSS syntax.

Changed the server-side templates to render the portico-styles
bundle instead of directly requiring the portico stylesheet. This
allows webpack to handle stylesheet compilation and minification.

We use the mini-css-extract-plugin to extract out css from the
includes in webpack and let webpacks production mode handle
minification. Currently we're not able to use it for dev mode
because it does not support HMR so we use style-loader instead.
Once the plugin supports HMR we can go on to use it for both
dev and prod.

The downside of this is that when reloading pages in the development
environment, there's an annoying flash of unstyled content :(.

It is now possible to make a change in any of the styles included
by static/styles/scss/portico.scss and see the code reload live
in the browser. This is because style-loader which we currently
use has the module.accept code built-in.
2018-04-27 09:04:50 -07:00
Armaan Ahluwalia f20671a509 webpack: Fix Hot Module Reloading in webpack.
This commit fixes hot module replacement in webpack. To do this
we open port 9994 used by webpack to communicate between browser
and devserver. The attempts to forward the proxy from 9991 failed
so the last resort was to open up the webpack port.
It also removes an uncessary plugin in the webpack config and moves
the --hot flag to tools/webpack.
2018-04-27 09:04:49 -07:00
Steve Howell 9ece6d2be4 lint: Exempt "opts." from addClass checks.
If we use something like "opts.highlight_class", it's probably
in a generic widget.
2018-04-26 08:42:47 -07:00
Tim Abbott 1ff909d971 coverage: Exclude Zephyr ccache logic from test coverage.
That code path is pretty constrained in how it's used, and is only for
the legacy Zephyr integration which we don't expect to spend effort
on again.
2018-04-25 22:41:31 -07:00
Tim Abbott 47bdf5ecba coverage: Exclude generate_test_data from testing.
That file is just used for generating manual testing data; it is
actually tested by our suite, just before coverage starts running.
2018-04-25 22:38:54 -07:00
Tim Abbott d5946de718 decorator: Add nocoverage comments for rate_limit decorator.
We've already got a bunch of other comments on work we need to do for
this decorator and an open issue that will ensure we at some point
rework this and add tests for it.  In the meantime, I'd like to lock
down the rest of decorator.py at 100% coverage.

Fixes #1000.
2018-04-25 22:37:12 -07:00
Priyank Patel bc454bab88 webpack: Disable host check for webpack-dev-server.
Webpack dev server by default does host checking for requests. so
in dev enviorment if the the request came for zulipdev.com it would not
send js files which caused dev envoirment to not work.
2018-04-24 14:14:20 -07:00
Emilio Schadt b6f5ea0fd2 docs: Add README.md to zulip/docs to prevent reading on GitHub.
This should help avoid confusing about broken links.

Tweaked by tabbott to fix formatting and the linter.
2018-04-24 10:50:39 -07:00
Vishnu Ks 7533796ea9 popover: Add an option to show user profile.
Fixes #8880
2018-04-23 16:56:24 -07:00
Tim Abbott 1703e23980 templates: Move all core app templates into a subdirectory.
This should make it easier to find the templates that are actually
part of the core webapp, instead of having them all mixed together
with the portico pages.
2018-04-23 16:46:37 -07:00
Tim Abbott 8fc04a074d test-backend: Update coverage excludes for new import_realm.py. 2018-04-23 16:26:01 -07:00
Armaan Ahluwalia 97ec5e0aaa webpack: Tweak config file for stats and faster recomp on HMR.
Update the config file to show slightly more information while
compiling webpack. Also decreased the time webpack waits before
recompiling in order to speed up HMR.
2018-04-23 15:49:33 -07:00
Armaan Ahluwalia c70d26224d webpack: Change devtool sourcemap strategy.
Changed the devtoool setting for development from 'eval' to
'cheap-module-eval-source-map' as it has better support for
breakpoints in Google Chrome and the difference is time is
negligible at the number and size of files currently being
consumed by webpack. This stragtegy can be reviewed in the
future as the size of files grows or Chrome adds better
support.
2018-04-23 15:49:33 -07:00
Armaan Ahluwalia fc7aa1a771 webpack: Upgrade webpack version 4.5.0.
Upgrade webpack to latest version at the time of authoring. This
involves upgrading webpack version and its loaders to compatible
versions. It also involved editing tools/webpack to use the
executable for webpack-cli instead because of a change in how the
webpack package wants you to handle shell execution.
It also fixes the confugration for TypeScript in the webpack config
as that was previously broken. Including TypeScript files in JS
files compiled by webpack now works.
2018-04-23 15:49:33 -07:00
Priyank Patel 29f04511c0 node_tests: Use nyc instead of deceprated istanbul package. 2018-04-23 15:27:16 -04:00
Tim Abbott 3f4f94d111 check-capitalization: Add "G Suite" to proper nouns list.
This fixes an error caught by CI.
2018-04-23 10:21:10 -07:00
Steve Howell 6d4855bd6a lint: Prevent accidental use of const.
This prevents us from using const in our JS code, with exceptions
for test code and the portico.  Hopefully this is just a temporary
rule until we make our pipelines with work with ES6.

I tried to prevent "let", but that was too noisy.

This adjusts the one false-negative case of using const in a comment.
2018-04-22 20:11:38 -07:00
Steve Howell 3f1930f9c5 buddy list: Extract buddy_data.js. 2018-04-22 20:08:08 -07:00
Steve Howell 536236d9b1 buddy list: Extract buddy_list.js. 2018-04-22 20:08:08 -07:00
Eeshan Garg ca5ea20ab7 doc tests: Add portico pages to tools/test-help-documentation.
Fixes #9117.
2018-04-21 21:51:32 -07:00
Preston Hansen 76d6c71595 tests: Move zerver/fixtures to zerver/tests/fixtures for clarity.
Fixes #9153.
2018-04-19 21:50:17 -07:00
Joshua Pan 99f07fe2e2 tests: Fully cover top_left_corner.js. 2018-04-19 14:56:55 -04:00
Eeshan Garg 19177a4aff webhooks: Move github_webhook/ to github/. 2018-04-19 11:00:55 -07:00
Eeshan Garg 48b8558c02 webhooks: Move github/ to github_legacy/ and remove docs. 2018-04-19 11:00:55 -07:00
Steve Howell f8fd169a7d node tests: Allow --coverage for individual files.
We now allow you to run --coverage on individual files.  This helps
when you want to make sure a file is being covered directly and not
just getting incidental coverage from higher level tests.

Before this commit, we were conflating wanting coverage reports with
wanting coverage checks.  For individual files, we now solve that by
simply eliminating the coverage checks.  This required some minor
refactoring to extract some functions.
2018-04-17 17:52:19 -07:00
Vishnu Ks cc93ac34a8 coverage: Add coverage to estimate_recent_messages.
With this message.py is fully covered and can be
removed from not_yet_fully_covered in test-backend.
2018-04-17 11:01:20 -07:00
Alyssa Wagenmaker d4e5777296 tests: Test user unsubscribing before soft deactivation.
Brings lib/soft_deactivation.py up to 100% test coverage.

Improves: #7089.
2018-04-16 11:28:08 -07:00
Tarun Kumar 0b62410f5e pm_pill: Achieve 100% node-test coverage for compose_pm_pill.js. 2018-04-13 11:59:57 -07:00
Tarun Kumar 51601fab44 muting: Achieve 100% node-test coverage for settings_muting.js. 2018-04-13 11:59:57 -07:00
Tim Abbott 105eed049e install-node: Fix leaking of $HOME.
This fixes a bug where provision was failing since our most recent
upgrade to yarn/nvm/node.

It turns out my original fix was the correct fix, but to the wrong
third-party tool: nvm, not yarn, was the offender.
2018-04-12 14:32:36 -07:00
Tim Abbott feef35bf25 linter: Add checks for sloppy use of .html().
Since jQuery's .html() can be a source of security bugs, we add a new
lint rule that tries to catch common problematic uses.
2018-04-12 09:50:14 -07:00
Shubham Padia 55619cbe70 browser-support: Add string.prototype.endswith polyfill.
String.prototype.endsWith is not supported in ie11.
Adds string.prototype.endswith package to dependencies and places
it at `common` entry point in webpack.assets.json.
2018-04-11 15:40:57 -07:00
Shubham Padia 57a494f94d browser-support: Add codepointat polyfill to `common` entry point.
Adds string.prototype.codepointat which was added as a polyfill
earlier to the project but was not added to `common` entry point.
2018-04-09 12:16:19 -07:00
Shubham Padia b906562f22 browser-support: Add string.prototype.startsWith polyfill.
Fixes #8944.
Adds string.prototype.startswith package to dependencies and places
it at `common` entry point in webpack.assets.json. As common.js is
loaded on all code paths first, there is no need to place this package
into other entry points.
2018-04-09 12:16:19 -07:00
Aditya Bansal b9f1acb300 linter: Enforce 2 space indents on tags spread over multiple lines.
We make some specific cases of tags use 2 space indents.
The case description:
* A tag with opening tag spread over multiple lines and closing tag
on the same line as of the closing angle bracket of the opening tag.
* A tag with opening tag spread over multiple lines and closing tag
not on the same line as of the closing angle bracket of the opening
tag.

Example:
Case 1:

Not linted:
<button type="button"
class="btn btn-primary btn-small">{{t "Yes" }}</button>

After linting:
<button type="button"
  class="btn btn-primary btn-small">{{t "Yes" }}</button>

Case 2:

Before linting:
<div class = "foo"
     id = "bar"
     role = "whatever">
     {{ bla }}
</div>

After linting:
<div class = "foo"
  id = "bar"
  role = "whatever">
    {{ bla }}
</div>
2018-04-07 20:08:44 -07:00
Aditya Bansal 550222dede linter: Make multiline handlebar singleton tags use 2 space indentation. 2018-04-07 20:08:38 -07:00
Aditya Bansal 2fe012ffff linter: Make html singleton tags use 2 space indentation. 2018-04-07 20:08:31 -07:00
Alyssa Wagenmaker df666c3dfc test-backend: Report fully covered files still in not_yet_fully_covered. 2018-04-06 17:29:18 -04:00
Ben Reeves 29a079ebbf test-backend: Remove fully covered files from not_yet_fully_covered. 2018-04-06 17:29:18 -04:00
Tim Abbott b0b134cb4c help: Clean up settings_html and subscriptions_html.
After some thinking, I don't think there's any actual value to doing
the ../ style relative links here, whereas there is actual harm from
the links being slightly broken in the current model.  We fix this by
just using /#settings as the URL.

Fixes #8978.
2018-04-05 14:48:26 -07:00
Tim Abbott ed5b374ffa help docs: Fix a sorta buggy URL test. 2018-04-05 13:07:34 -07:00
Steve Howell e145258fee node tests: Add 100% coverage for components.js. 2018-03-29 08:09:38 -07:00
neiljp (Neil Pilgrim) 090b47ed19 mypy: Add explicit Optional for default=None parameters in various files. 2018-03-28 12:31:51 -07:00
neiljp (Neil Pilgrim) 07971d3a66 linter: Add check that Optional models are tagged with null=True.
Exclude NullBooleanField explicitly.

Amend one line in models.py found from this linter change.
2018-03-27 10:21:25 -07:00
neiljp (Neil Pilgrim) 7d93f5726c linter: Add check that null=True lines are annotated as Optional.
Also add two missing annotations of this type that were missed
until linted.
2018-03-27 10:21:25 -07:00
Joshua Pan 0f58f20ad8 tests: Clean up test-js-with-node output on success. 2018-03-26 06:35:58 -04:00
Joshua Pan ed9eb3bdb5 tests: Make test-js-with-node handling smarter.
Added support for passing a filename without `.js` suffix.
This then fixed the issue of no complaints for invalid test
files. Now, throws an error for invalid test files.

Fixes #8579.
2018-03-26 06:35:58 -04:00
neiljp (Neil Pilgrim) 9a49812cde mypy: Extract default run-mypy options into mypy.ini. 2018-03-23 16:53:43 -07:00
neiljp (Neil Pilgrim) fb4f5c8570 tools: remove strict-optional command from run-mypy.
This is now controlled globally and on a per-file basis in mypy.ini.
2018-03-23 16:53:43 -07:00
neiljp (Neil Pilgrim) 70117f0e83 mypy: Add asserts of intermediate values in update-authors-json. 2018-03-23 13:07:39 -07:00
neiljp (Neil Pilgrim) f0e3fcb937 mypy: Amend test-backend:internet_guard to return NoReturn.
It is guaranteed to raise an exception, so this is reasonable,
and it satisfies mypy.
2018-03-23 11:39:54 -07:00
sinwar 675e844e6b team page: Add contributors of zulip-terminal.
It adds zulip terminal contributions to
team page using github apis.
2018-03-22 21:06:18 -07:00
Tim Abbott 08a6271b54 update-authors-json: Improve error handling for duplicate commits.
It's possible that a user with duplicate commits might not be in the
top 100 users for a given repo.
2018-03-22 14:58:13 -07:00
Gooca 2363c9955a ui scrollbar: Update perfect-scrollbar to 1.3.0.
Update perfect-scrollbar to fix stutter space-scrolling in #8544. Also
reworked deprecated `element.perfectScrollbar` to `new
PerfectScrollbar(element)`.  Lastly, updated provision version and
changed node module path to new path.

This also refactors perfect-scrollbar in help.js to work with updated
version of perfect-scrollbar. Because the update also changed
perfect-scrollbar's css selectors for all scrollbars in zulip, we
update those too.

Fixes #8544.
2018-03-20 21:59:11 -07:00
Tim Abbott 791953963b check-capitalization: Include banned words in overall output.
This locks down the ban on use of "Realm" in translated strings in the
codebase.
2018-03-16 16:59:02 -07:00
Tim Abbott 8e334b0f64 check-capitalization: Exclude certain uses of the word "realm".
Uses that are actually a variable name don't make sense to exclude.
2018-03-16 16:58:49 -07:00
neiljp (Neil Pilgrim) 966ca7015f mypy: Finalize migration of cache.py to python3 function annotation.
- Use forward declarations of some types from models.py to avoid cycles.
- Remove cache.py from linter rule exclude list to ensure it stays that way.
2018-03-16 11:29:12 -07:00
Vishnu Ks c385fcefa8 templates: Fix translation tag for disposable email tooltip. 2018-03-15 14:35:24 -07:00
neiljp (Neil Pilgrim) a3d8cc1f86 Mypy: Migrate tornado/descriptors.py to python3 function annotations.
Use quoted form of ClientDescriptor to break presumed circular dependency,
and modify import statement to shorten quotes.
2018-03-15 12:54:43 -07:00
neiljp (Neil Pilgrim) a7bfb09067 Mypy: Use models.py QuerySet annotation approach in stream_subscription.py.
Namely, annotate as best as possible, and add notes to indicate preference,
if QuerySet develops generic typing.

Note that the return values of functions with annotations changed in this
commit are used elsewhere as QuerySets, so the Sequence[T] approach used
for some functions in models.py is not applicable.
2018-03-15 12:54:43 -07:00
neiljp (Neil Pilgrim) ce4ac0d2cf Mypy: Ensure consistency of QuerySet return types in models.py.
Other functions took the form of returning Sequence[T] when the QuerySet
functionality is unused beyond the function, with T being the objects
filtered for in the function body; this commit follows that practice for the
one remaining python2 comment-annotated function, completing the transition
of models.py to py3.5 function annotations.

A note is also added to another function regarding a need to return a
QuerySet, and ideally a QuerySet[T] in line with the other functions, as and
when QuerySet becomes annotated as a generic.
2018-03-15 12:54:43 -07:00
Tarun Kumar c8d5cf7d6c user-groups: Achieve 100% node-test coverage of user_groups.js.
Fixes #8704.
2018-03-15 10:09:28 -07:00
Harshit Bansal 40958e0824 emoji: Switch to 64px 256 color indexed sprite sheets.
This commit switches our emoji infrastructure to use 256 color indexed
64px spritesheets. Earlier we were using non-indexed 32px spritesheets
which were blurry on high dpi displays. These indexed spritesheets not
only provide a crispier display but are also smaller in size.

This commit also removes the `emoji-datasource` package as a dependency
as all the data is now sourced from individual datasource packages.

Fixes: #7862.
2018-03-14 10:28:45 -07:00
neiljp (Neil Pilgrim) 43ff4ea1f7 tools: Ensure mypy is run on .pyi files. 2018-03-13 13:49:39 -07:00
Tim Abbott 8866c23b5e test-backend: Enforce 100% test coverage on models.py.
Following the exclusions and code deletions we just did, we can now
lock down 100% test coverage on zerver/models.py.
2018-03-12 11:52:39 -07:00
neiljp (Neil Pilgrim) ee2b8bfbfb linter: Require Py3 function annotations, with some exceptions.
Main exceptions are scripts/tools/puppet & tests.

Other current exclusions:
- api_test_helpers.py (avoid changing test code);
- cache.py, due to comments in file;
- models.py, due to failure on QuerySet[Message];
- stream_subscription.py, due to failure on QuerySet[Subscription];
- tornado/descriptors.py;
- views/streams.py, due to remaining FuncKwargPair issue;
- zthumbor/, since thumbor is in python2.

Tweaked by tabbott to partially document the stauts in comments.
2018-03-12 11:23:30 -07:00
Umair Khan 728cd04c4d check-capitalization: Check for banned words.
Fixes #8619
2018-03-11 15:34:47 -07:00
Steve Howell fef2840531 Add FetchStatus class.
We will use this in subsequent commits to replace the
`load_more_enabled` flag.
2018-03-09 14:20:58 -08:00
sinwar 6bed6ccdcf team page: Remove duplicate contribution counts.
It removes duplicate contribution count from zulip-server.

Fixes #7836.
2018-03-07 17:25:41 -08:00
Steve Howell 3e76d523c7 node tests: Add tests for user_pill.js.
This gets coverage to 100%.  (We already had a lot of indirect
coverage, but these tests should have more obvious failure
symptoms.)
2018-03-07 15:53:11 -08:00
Greg Price f22712e8e8 install: Go straight through by default.
This flips the experimental `--express` option to be the default.

We retain the old behavior, where the script exits before
`initialize-database`, as an option `--no-init-db`; it might be useful
in e.g. a migration scenario (from a Zulip install elsewhere, or
another chat system) where the admin wants to set up the database
separately.

The install instructions are adjusted to match, getting shorter by two
steps and a bunch of words.  I think this opens up opportunities to
refactor the text to simplify things further, too, but leaving that
for another commit.

Also tweak the "production" test suite to match.
2018-03-06 19:43:02 -08:00
Greg Price 8111848ac4 test-install: Do a dist-upgrade in prepare-base.
This keeps the base tree up to date, saving the time we'd spend
doing the same upgrades in each test install.
2018-03-06 19:43:02 -08:00
Tim Abbott f253f96211 lint: Add "Gravatar" to capitalized terms in linter. 2018-03-05 10:29:18 -08:00
Tim Abbott 4d8e18e58f settings: Move display_checkmark to settings_ui.js. 2018-03-04 18:26:27 -08:00
Tim Abbott 6a06e453c7 display_settings: Extend de-duplication of unnecessary logic.
We add a new option to the change_display_setting function to support
cases with a custom on-success or on-failure message.
2018-03-04 18:09:46 -08:00
Balaji2198 5aa7098c81 display_settings: Change success/failure feedback interface.
This replaces the cumbersome system we had for giving users feedback
on settings state changes in the display settings UI.

We expect this new system to be what we will attempt to migrate other
settings widgets to match over the coming weeks and months.  It also
provides the opportunity to significant refactor away a lot of the
code duplication in settings_display.js.

Thanks to Brock Whittaker for redoing the styling and improving the
code simplicity.

Fixes #7622.
2018-03-04 17:47:05 -08:00
Tim Abbott a3ca7ee1c7 lint: Add a few strings to capitalization linter.
My hope is that we'll be able to eliminate these strings from the UI
altogether soon, but this is needed to keep CI passing for now.
2018-03-04 16:34:00 -08:00
Robert Hönig 81ba7a1e40 Mark DigestWorker and PushNotificationsWorker as nocoverage.
These two classes are tricky to test, and nocoverage-ing them
allows us to mark queue_processors.py as fully covered. We
still want to cover these two workers at some point, but for
now, it's nice to enforce full coverage for any future changes
to queue_processors.py.

Fixes (sort of) #6542.
2018-03-04 13:31:33 -08:00
Tim Abbott 530e098916 node: Remove 100% coverage requirement for user groups.
This is a temporary change to get CI passing again; we lost some
coverage in aed8cc9062, which fixed some
major bugs in the feature.
2018-03-04 10:11:11 -08:00
YJDave f5a24efdd8 static/js: Add `settings_ui.js` file for common settings UI functions. 2018-03-03 08:38:54 -08:00
Eeshan Garg 59581f6a32 tools/lint: Require a space after every `:` in JSON files. 2018-03-02 14:10:53 -08:00
Tim Abbott 335c383bbb lint: Add some excludes to capitalization linter. 2018-03-02 11:55:46 -08:00
Tim Abbott 895b5642f1 commit-message-list: Don't print anything if no commits.
This fixes some output spam when running the linter at
upstream/master.
2018-03-02 11:51:08 -08:00
Tim Abbott 14764d015a settings: Tag cookie bot placeholder short name for translation. 2018-03-02 09:33:16 -08:00
Tim Abbott e18537d842 emoji: Add a translation tag for new_emoji placeholder. 2018-03-02 09:32:09 -08:00
Tim Abbott 9c4b0cc100 lint: Clean up translation tag lint for filter settings. 2018-03-02 09:30:42 -08:00
Tim Abbott f9210bace8 lint: Add a translation tag for marketing. 2018-03-02 09:28:57 -08:00
Joshua Pan 5777582013 lint: Tighten placeholder lint rule to exclude links.
Fixes #8420.
2018-03-02 09:24:09 -08:00
Tim Abbott 5ccd3aa71b i18n: Run process-mobile-i18n when syncing translations. 2018-03-02 08:59:00 -08:00
Tim Abbott 3be0b3dd91 mypy: Fix errors in linter.
We should figure out a better way to manage this issue.
2018-03-01 20:33:30 -08:00
Tim Abbott 8d00266fd6 lint: Add a linter rule banning use of sudo in most scripts.
This should help prevent against bugs where we accidentally introduce
use of sudo somewhere in the production installer or upgrade code path
(these used to happen all the time), which doesn't work on production
systems that don't have sudo setup.
2018-03-01 16:30:31 -08:00
Greg Price 72497d61c1 test-migrations: Clean up a bit. 2018-03-01 15:03:55 -08:00
Aditya Bansal bd0a5d5d69 css-minifier: Lint for '+' operator in calc().
Fixes: #8403.
2018-03-01 13:26:16 -08:00
Tim Abbott 6b31752f82 user_groups: Change placeholder group to "Marketing team".
This is much more something that would apply to likely customer
organizations.
2018-03-01 11:28:24 -08:00
Vishnu Ks 36f6ad76b4 admin ui: Set usergroup name input maxlength to 100.
Same as that of the maxlength defined in backend.
2018-03-01 11:27:26 -08:00
Greg Price 4475950ddf queue: Restore prematurely-cut upgrade path.
Revert c8f034e9a "queue: Remove missedmessage_email_senders code."
As the comment in the code says, it ensures a smooth upgrade path
from 1.7.x; we can delete it in master after 1.8.0 is released.
The removal commit was merged early due to a communication failure.
2018-02-28 11:15:53 -08:00
Umair Khan c8f034e9a0 queue: Remove missedmessage_email_senders code.
After 68513952fb, all emails are sent through email_senders queue.
This commit removes code related to the legacy queue.
2018-02-21 16:43:56 -08:00
Eeshan Garg be0a04f33e api docs: Test sample fixtures for unauthorized_errors_fatal.
This commit adds tests for the sample fixtures for when
unauthorized_errors_fatal is passed to client.add_subscriptions.
2018-02-20 14:54:12 -08:00
Eeshan Garg 06bf47d2f5 api docs: Test sample fixture for user not authorized error.
This commit adds tests for the fixture for when a user is not
authorized (perhaps because the query requires the use of admin
privileges) for a particular query.
2018-02-20 14:52:09 -08:00
Eeshan Garg 124c672a7c api docs: Test sample fixture for message edit permission error.
In templates/zerver/api/update-message.md, we have a sample fixture
for when a zulip.Client does not have the permission to update/edit
a particular message. This commit adds a test for that fixture.
Also, tools/test-api now also uses a non-admin client for this test,
which might come in handy in the future.
2018-02-20 14:52:09 -08:00
Steve Howell 1f6ddf0110 refactor: Extract transmit.js from compose.js.
We now isolate the code to transmit messages into transmit.js.
It is stable code that most folks doing UI work in compose.js don't
care about the details of, so it's just clutter there.  Also, we may
soon have other widgets than the compose box that send messages.

This change mostly preserves test coverage, although in some cases
we stub at a higher level for the compose path (this is a good thing).
Extracting out transmit.js allows us to lock down 100% coverage on that
file.
2018-02-20 09:29:26 -08:00
rht 70a41cc2eb tools/tests: Use Python 3 syntax for typing. 2018-02-19 10:17:03 -08:00
ihsavru 4103996a22 org-settings: Fix styling of Allowed Domains modal.
Fixes: #7628.

Cleaned up by Brock Whittaker and Node tests fixed by Steve Howell.
2018-02-16 16:00:21 -08:00
Aditya Bansal 6fce1d7834 template_parser: Add parsing support for self closing tags as per HTML5.
In this commit we add support for some tags which are also called
void-elements according to
http://w3c.github.io/html/syntax.html#void-elements to be parsed by
our template parser and get tagged as singleton_html_tags.

Fixes: #8387.
2018-02-15 17:47:43 -05:00
Greg Price b0b0777cd0 translation: Make new tagmessages script a bit easier to run.
Now executable! Just run `tools/tagmessages`.

Also, get the username and password from a `.transifexrc` file.

And hardcode the project slug to `zulip-test` rather than to `zulip`;
the Transifex API is bad at namespacing, so this makes it possible to
run this script on a test project (the only way we're currently using
it) even for people like me who can also upload to the real Zulip
project on Transifex.
2018-02-15 13:38:09 -08:00
Robert Hönig ebf79427ca translation: Add script to tag strings in Transifex.
For now, tags are only used for backend translations with django.
2018-02-15 13:38:09 -08:00
Tim Abbott be3d43bb46 lint: Add linter rule banning $.get and friends.
These are not allowed in our style guide.
2018-02-13 16:47:59 -08:00
Shubham Padia e1f943913a lint: Allow revert commit messages in gitlint.
Fixes #8145.
2018-02-13 09:21:01 -08:00
Aditya Bansal 35969edd66 deps: Replace libz-dev with zlib1g-dev since the former was renamed. 2018-02-12 14:40:26 -08:00
Aditya Bansal d5736ef65a provision.py: Remove duplicate install candidated from APT_DEPENDENCIES.
This is necessary, since we now have some dependencies in common
between the Thumbor and main Zulip dependency lists.
2018-02-12 14:40:26 -08:00
Rhea Parekh 3ebd30120c slack importer: Remove depreciated test.
This test was added in '/tests/test_slack_importer' in
6addf79edb.
2018-02-12 00:43:02 +05:30
Greg Price b374ea7357 mypy: Pass --show-traceback.
This saves a bit of debugging in the event that we hit a crash
inside mypy.
2018-02-09 19:42:49 -08:00
Eeshan Garg 929724e5e7 api docs: Add page for common error payloads.
We now have a separate page for common error payloads, for example,
the payload for when the client's API key is invalid. All error
payloads that are presented on this page will be tested similarly
to our other non-error sample fixtures.
2018-02-08 17:58:41 -08:00