Commit Graph

3270 Commits

Author SHA1 Message Date
Anders Kaseorg 03cfe10da7 coveragerc: Omit blub.
The jedi package exec()s some code in the context of the fake module
blub, causing errors when generating the coverage report.  See
https://github.com/davidhalter/jedi/issues/1122.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 15:04:47 -07:00
Tim Abbott 095ff28277 test-backend: Avoid dropping/rebuiling zulip_test database.
The test-backend parallel test runner system doesn't actually use the
zulip_test database; instead, it creates its own databases off the
zulip_test_template database.

We were accidentally running `tools/generate_fixtures` even when there
are no changes, because this function is shared with the
tools/lib/test_server.py codebase, which needs us to do the work of
creating a test database for it off the zulip_test_template database.

Fixing this saves about 1.5s / 4s of the runtime of a single test.
2019-05-28 16:38:45 -07:00
Tim Abbott e0afdb675f test_fixtures: Rename and document update_test_databases_if_required.
The previous name was confusing because `generate_fixtures` isn't
actually a good name in the first place.
2019-05-28 16:38:45 -07:00
Tim Abbott 7b6f219663 test-backend: Flush stdout when starting testing.
This makes the output more properly ordered.
2019-05-28 16:38:45 -07:00
Tim Abbott 4442509ec3 node: Disable a test suite that's missing a line of coverage. 2019-05-28 16:36:30 -07:00
Anders Kaseorg f21d3be5fb dev-vagrant-docker: Copy the host user’s uid for the vagrant user.
This allows /srv/zulip to be writable from the guest even if the host
user’s uid is not 1000.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 19:21:51 -07:00
Anders Kaseorg 5231c8e575 dev-vagrant-docker: Unminimize the minimized Ubuntu cloud image.
This restores man pages and other documentation that have been
stripped from the default Ubuntu cloud image and installs
ubuntu-minimal and ubuntu-standard.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 12:53:11 -07:00
Rohitt Vashishtha eb44b7d69d linter: Do not lint scripts parsed with tail (like dev-motd). 2019-05-27 12:40:02 -07:00
Anders Kaseorg b594708915 Vagrantfile: Be nicer to /etc/update-motd.d.
Some of its information is helpful.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:33:06 -07:00
Anders Kaseorg 407564086d provision: Move apt-get update from retry handler to setup-apt-repo.
This avoids unnecessarily alarming error messages if the apt cache is
missing.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:07:38 -07:00
Anders Kaseorg ae524b677d ci: Move backend and production tests to Ubuntu 16.04 (xenial).
This is preparation for dropping support for Trusty in CI.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:07:15 -07:00
Anders Kaseorg 684ebc2a5e Vagrantfile: Support docker provider.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-24 15:43:23 -07:00
Hemanth V. Alluri c47521401f devtools: Move the integrations devtool JS to the portico folder. 2019-05-22 21:28:11 -07:00
Mateusz Mandera 58da8fd5f1 test-backend: Remove email_mirror from not_yet_fully_covered.
The preceding commits get the email mirror to 100% coverage, so we can
now remove zerver/lib/email_mirror.py from the list of not-fully-covered
files.
2019-05-20 19:35:32 -07:00
Tim Abbott 136a0da30b provision: Update unsupported architecture logging messages.
We don't use zulip-devel@ anymore, and also add a comment that it's
probably not a huge project to support e.g. arm.
2019-05-20 15:23:40 -07:00
Rishi Gupta fbc2239f73 keyboard shortcuts: Update description for S. 2019-05-20 15:10:11 -07:00
Kanishk Kakar 77ab9a0eb0 docs: Update desktop repo URL. 2019-05-20 11:01:11 -07:00
Anders Kaseorg 141088586b Completely replace perfect-scrollbar with SimpleBar.
perfect-scrollbar replaces both the appearance and the behavior of the
scrollbar, and its emulated behavior will never feel native on most
platforms.  SimpleBar customizes the appearance while preserving the
native behavior.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-17 12:06:51 -07:00
Vishnu Ks e7507a6eb1 tools: Avoid upgrading stripe automatically.
Upgrades to the stripe library can sometimes break semantics for our
billing system, and so we should make sure to use our documented 
testing process before doing them.
2019-05-17 11:36:35 -07:00
Vishnu Ks 56259c7acd lint: Catch exit(1) in management commands.
Using sys.exit(1) in a management command makes it impossible
to unit test the code in question.  The correct approach to
do the same thing in Django management commands is to raise
CommandError.
2019-05-17 11:34:00 -07:00
Tim Abbott 68255fe1df lint: Fix exclusion for development directory.
I forgot about running `test-tools` when making linter changes.
2019-05-15 13:40:33 -07:00
Tim Abbott 7f2fcea794 lint: Ban i18n usage under zerver/views/development. 2019-05-15 13:11:04 -07:00
Hemanth V. Alluri bae8295c52 devtools: Add integrations dev panel.
This commit adds a new developer tool: The "integrations dev panel"
which will serve as a replacement for the send_webhook_fixture_message
management command as a way to test integrations with much greater ease.
2019-05-15 13:07:44 -07:00
Vishnu Ks 06983298ba export: Add support for exporting realm with member consent.
This lets us handle directly in our tooling the user experience that
we document for exporting a realm with member consent (before, it
required unpleasant manual work).
2019-05-15 12:35:32 -07:00
Tim Abbott 5ec8f6e812 test_migrations: Disable migrations test.
Our migrations test suite only really works when no future migrations
conflict with it.
2019-05-12 22:06:17 -07:00
Tim Abbott 7c94d350d4 ci: Run test-backend with limited parallelism.
See the comment for details; this should fix timeouts we've been
seeing in CI.
2019-05-08 18:26:26 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
Wyatt Hoodes ececf0a209 provision.py: Add a conditional to remove '.eslintcache'.
It was discovered that the '.eslintcache' file was causing eslint to
throw a TypeError after a recent update/addition to the dependencies.
It makes sense to remove this file as part of the provisioning process
to avoid such exceptions.
2019-05-08 17:06:17 -07:00
Vishnu Ks 6c58603eaf support: Add support for scrubbing realm. 2019-05-06 20:12:54 -07:00
Mayank Madan 99414e2d96 provision: Add support for Debian 10 Buster. 2019-05-05 17:57:19 -07:00
Raymond Akornor e01d3be1ba tools: Remove check-urls linter.
This commit removes `tools/check-urls`. It was added as
a useful tool in preparation for the Django 1.10 migration.
Since we completed that migration, it is no longer needed.

Fixes #12180.
2019-05-05 17:47:41 -07:00
Wyatt Hoodes 21c67ea1c2 test-backend: Change --parallel default to use `cpu_count`.
The number of processes to run the backend tests is currently a
hardcoded value, this commit transistions the default to be based on the
number of logical CPUs available.
2019-05-02 10:48:20 -07:00
Yashashvi Dave 3649a9f15c subs: Add `stream_ui_updates.js` module for managing ui elements.
This commit adds `stream_ui_updates.js` module. This module
will includes functions which will update different ui elements
(i.e. subscription button, subscriber count).
2019-04-30 14:36:52 -07:00
Eeshan Garg a73e8109b7 webhooks: Remove the legacy GitHub integration.
The github-services model for how GitHub would send requests to this
legacy integration is no longer available since earlier in 2019.
Removing this integration also allows us to finally remove
authenticated_api_view, the legacy authentication model from 2013 that
had been used for this integration (and other features long since
upgraded).

A few functions that were used by the Beanstalk webhook are moved into
that webhook's implementation directly.
2019-04-27 15:13:44 -07:00
Vishnu Ks 16394e53a3 tools: Make dependency upgrader upgrade indirect dependencies separately. 2019-04-26 16:22:00 -07:00
Anders Kaseorg 122ad5be58 lint: Lint against calls to _() on computed strings.
These don't actually work from an i18n perspective.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:18 -07:00
Anders Kaseorg 44dd7c2d95 lint: Strengthen lint checks for string % non-tuple.
This is really a job for an AST parser rather than a pile of regexes;
among other issues, these will still miss violations that span
multiple lines.  But, you know, I tried.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:22:51 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Rafid Aslam d983611919 template_parser: Add support for jinja2 whitespace markers.
Fixes #11516.
2019-04-23 11:52:13 -07:00
Anders Kaseorg 5290519a62 scripts: Always use ON_ERROR_STOP=1 when running psql.
Also use psql -e (--echo-queries) in scripts that use ‘set -x’, so
errors can be traced to a specific query from the output.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:19 -07:00
Anders Kaseorg efd22fd2ae postgres-init-dev-db: Catch exception when recreating extant users.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:18 -07:00
Anders Kaseorg 25798c8605 linter_lib: Remove sudo exemption for terminate-psql-sessions.
terminate-psql-sessions no longer uses sudo as of #11771.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:17 -07:00
Anders Kaseorg 12e2e0d658 tools/webpack: Run webpack in the foreground.
`tools/run-dev.py` already backgrounds `tools/webpack` (and deals with
cleaning it up on exit), so there’s no need for `tools/webpack` to
also background the actual `webpack` process.  But when running
`tools/webpack` by itself, it’s annoying to clean up the backgrounded
process manually.

Run `webpack` in the foreground, using `os.execvp` so we don’t waste
memory on an intermediate wrapper process.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 13:59:11 -07:00
Anders Kaseorg dd3ccbc897 webpack: Fix node_modules path resolution.
Webpack applies special logic to relative paths provided in
`resolve.modules`, and this logic is expected to be used for
`node_modules`.  One case where this is important is when
`node_modules/foo` wants to import a different version of package
`bar` than the one at `node_modules/bar`, and so yarn gives it its own
copy at `node_modules/foo/node_modules/bar`.

It would probably be better to avoid screwing with `resolve.modules`
at all, but this at least brings us one step closer to the default of
just `["node_modules"]`.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 13:49:23 -07:00
Anders Kaseorg 9fc0aa7ffa scripts: Fix exec invocation for in-process virtualenv activation.
activate_this.py has always documented that it should be exec()ed with
locals = globals, and in virtualenv 16.0.0 it raises a NameError
otherwise.

As a simplified demonstration of the weird things that can go wrong
when locals ≠ globals:

    >>> exec('a = 1; print([a])', {}, {})
    [1]
    >>> exec('a = 1; print([a for b in [1]])', {}, {})
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<string>", line 1, in <module>
      File "<string>", line 1, in <listcomp>
    NameError: name 'a' is not defined
    >>> exec('a = 1; print([a for b in [1]])', {})
    [1]

Top-level assignments go into locals, but from inside a new scope like
a list comprehension, they’re read out of globals, which doesn’t work.

Fixes #12030.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-16 17:42:44 -07:00
Tim Abbott 31f6ace084 lint: Fix typo in eslint command line. 2019-04-13 22:45:07 -07:00
Thomas Ip 90478fc4b2 typescript: Fix violations of new typescript rules. 2019-04-13 11:42:48 -07:00
Thomas Ip 02cb85a8a9 typescript: Use ESLint instead of TSLint. 2019-04-13 11:42:47 -07:00
Thomas Ip ea9a74fe24 typescript: Fix violations of existing eslint rules. 2019-04-13 11:38:19 -07:00
Anders Kaseorg 53879c4673 scripts: Rename DEFAULT_USER to POSTGRES_USER.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-12 17:27:23 -07:00