Commit Graph

92 Commits

Author SHA1 Message Date
K.Kanakhin 0d8c18a6dd nagios-plugins: Add websocket checking to nagios message sending test.
- Add websocket client to create connection with SockJS websocket server.
  It contains callback method to launch after connection setup.
- Add '--websocket' parameter to 'check_send_receive_time' script to
  check websocket connection.
- Add testing  websocket connection to production installation checking.
- Add cronjob to launch websocket connection nagios test.

This makes it possible for Zulip Nagios monitoring to check for
problems impacting the websockets sending code path, which is what all
web users use.
2016-12-30 15:36:37 -08:00
Tim Abbott cd7d8bfbea travis: Add test-help-documentation to CI for real.
75c4e3191d actually only added it to
`test-all`.
2016-12-26 16:17:42 -08:00
Umair Khan 6c1d805495 travis: Fix production suite flakiness.
Previously, we were doing this request to the production server before
waiting for all the supervisord processes to start; it's possible this
could cause failures where we hit the server before the Django uwsgi
processes are up.

Hopefully fixes #2723.
2016-12-15 22:04:57 -08:00
Umair Khan efccefc386 Upgrade to the latest Django release, 1.10.4.
Most of the changes to support this were merged some time ago; what
remains are these changes:

* Update requirements.txt
* Django 1.10: Upgrade success-http-headers.txt file.

- We no longer get the absolute urls, see
  https://docs.djangoproject.com/en/1.10/releases/1.9/#http-redirects-no-longer-forced-to-absolute-uris
- The headers are capitalized, previously, they were in upper case.

* Bump PROVISON_VERSION to 3.0 since this is a disruptive change.

Fixes #3.
2016-12-14 22:43:20 -08:00
Tim Abbott 441c92e42f travis: Remove postgres-9.6 from production suite as well. 2016-12-12 10:12:08 -08:00
Tim Abbott d2d404afcc travis: Remove npm cache in prod installation.
With this change in place, we would have been able to catch the bug
fixed in the last commit in CI.
2016-12-08 10:34:09 -08:00
Tim Abbott 15bfedec99 travis: Improve debuggability of server wget failures.
The main improvement here is causing `wget` errors to be ignored so
that we see the server logs in the event of a `wget` failure.
2016-12-03 20:48:57 -08:00
Tim Abbott 1a8a329b44 production-helper: Expand the apt-mark hold list. 2016-12-01 12:29:31 -08:00
Tim Abbott 87879c3916 travis: Workaround broken Travis CI java configuration.
Travis CI seems to have broken /usr/bin/java in a recent
infrastructure change.
2016-12-01 11:46:21 -08:00
Tim Abbott 0ac78ca454 setup-production: Fix path to update-prod-static.log.
This regressed in 55f2c49bbf.
2016-12-01 11:15:20 -08:00
Anders Kaseorg 6549dfd1d0 build-release-tarball: Add --py3 option for python3 shebang lines.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-29 17:45:18 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Pweaver (Paul Weaver) 059cd1e7aa tests: enable the pep8 linter in travis CI. 2016-11-17 00:11:00 -08:00
K.Kanakhin b248a2c33e docs-test: Check only http 404 error for external links.
- Raise exception only for http 404 error rensponse, for other
  http error codes used logging to notify devs about wrong links.
2016-11-01 21:45:41 -07:00
Tim Abbott 4fe5fc849e travis: Disable test-documentation temporarily. 2016-10-28 11:27:22 -07:00
K.Kanakhin 455301ca13 docs-test: Add documentation test scripts to backend test case.
- Add script to compile documentation build and start crawler
  to check documentation.
- Add documentation test script to backend travis test case.
- Add log level argument to test-documentation script.

Fixes #1492
2016-10-25 11:36:43 -07:00
Tim Abbott 9e503f2dcf run-mypy: Check scripts by default.
Previously, we checked scripts in a separate run to work around mypy
not supporting multiple scripts with the same name.  Since we have
fixed that issue, we can restore the original behavior.

We leave the --scripts-only option available, though I'm not sure it's
particularly useful and we'll probably eventually remove it.
2016-10-15 22:53:29 -07:00
Tim Abbott a17a6a21c0 Upgrade Django to 1.8.15 with Zulip patches. 2016-09-27 20:29:08 -07:00
Tim Abbott 5c88fbdc6f mypy: Require new Python functions to have type annotations.
We're now at the point where 100% of functions checked by mypy is
fully annotated; to avoid regressions, we're enforcing the requirement
that it stay this way.  We still have a moderate amount of code that
is neither checked by mypy nor annotated, but it seems reasonable to
annotate that code at the same time as we get a chance to fix the mypy
issues in it.

This is implemented by using the --disallow-untyped-defs option in
mypy by default.
2016-09-12 14:56:08 -07:00
Vishnu Ks a7ead9e99d settings: Eliminate ADMIN_DOMAIN for creating initial realm.
We now use `./manage.py generate_realm_creation_link` as the flow flow
for creating one's first realm.
2016-08-25 09:37:33 -07:00
Tim Abbott 88a123d5e0 Fix excessive CPU usage by rabbitmq-numconsumers Nagios checks.
The previous model for these Nagios checks was kinda crazy -- every
minute, we'd run a full `rabbitmctl list_consumers` for each of the
dozen+ consumers that we have, and then do the exact same parsing
logic for each to determine whether the target queue has a running
consumer to write out a state file.

Because `rabbitmctl list_consumers` takes a small amount of resources,
on systems where CPU is very limited (e.g. t2 style AWS instances),
this minor CPU wastage could be problematic.

Now we just do that `rabbitmqctl list_consumers` once per minute, and
output all the state files from a single command.

Further TODO items on this front include removing the hardcoded list
of queues.
2016-08-12 14:09:36 -07:00
Tim Abbott 6496fe2a53 travis: Remove rabbitmq nodename dependency on hostname.
Because rabbitmq doesn't support changing the nodename of a running
rabbitmq node, Zulip installations suffered a plague of issues where
e.g. a Zulip server would reboot, the hostname would change, and
suddenly the local rabbitmq instance being used by Zulip would stop
working.

We address this problem by using, by default, a fixed rabbitmq
nodename, but providing server administrators the option to set the
rabbitmq nodename used by Zulip however they choose.

To upgrade an existing server to use this new configuration, one will
need to add something like the following to /etc/zulip/zulip.conf:

[rabbitmq]
nodename = zulip@localhost

However, I don't believe we have the puppet code in place to make this
work correctly at initial installation without rabbitmq-server being
already installed (but off), as we can easily setup in Travis CI but I
haven't been willing to do for the installer.  So for now, this just
fixes our Travis CI problems.

Fixes: #1579.
2016-08-12 09:38:23 -07:00
Tim Abbott c7059c9751 travis: Update success-http-headers to match current certs.
Travis CI seems to have changed the way the snakeoil SSL certs are
generated in their infrastructure, so we need to update our expected
"success" HTTP headers accordingly.
2016-08-12 09:35:41 -07:00
Tim Abbott a648513580 production-helper: Remove /root/zulip during setup process.
This fixes a problem that caused production-helper to not be
idempotent.
2016-08-11 22:21:13 -07:00
Tim Abbott 7011c94465 production-helper: Use ln -nsf to install snakeoil symlinks.
This fixes a problem where production-helper was not idempotent.
2016-08-11 22:20:24 -07:00
Tim Abbott 28a7d991f6 setup-production: Remove more postgres packages.
Travis CI seems to have added Postgres 9.5 to the default-installed
Postgres package list, and we also missed a few side packages
previously as well.
2016-08-11 22:19:57 -07:00
Tim Abbott cb21584ffe check_send_receive_time: Support being run in a cron job. 2016-08-11 14:48:21 -07:00
Tim Abbott dca585fa75 travis: re-disable test-queue-worker-reload.
It seems to still fail sometimes in Travis CI.
2016-08-10 16:43:00 -07:00
Umair Khan 740b5634ac Fix and re-enable test-queue-worker-reload.
It seems that we no longer get the message, 'zerver/lib/actions.py
modified; restarting server', but the server reloads successfully
nonetheless.

Fixes: #1341.
2016-08-10 16:17:23 -07:00
Tim Abbott b3a768f4b2 settings: Improve ALLOWED_HOSTS defaults logic and docs.
This removes the requirement for the user to put localhost/127.0.0.1
in their ALLOWED_HOSTS list, since it is now added automatically.

Fixes: #1358.
2016-08-05 21:25:29 -07:00
Umair Khan 1a6e8282c8 Run 'check_send_receive_time' as 'zulip' user.
Run '/puppet/zulip/files/nagios_plugins/zulip_app_frontend/check_send_receive_time'
script as 'zulip' user so that the connection to the database can be
made correctly.
2016-07-28 13:39:29 -07:00
Tim Abbott 226a57d2c8 travis: Disable test-queue-worker-reload temporarily.
It's failing nondetermistically pretty often, and I'd like to cut the
impact to other users' work.
2016-07-22 11:36:52 -07:00
Eklavya Sharma fb59a07a89 Run mypy on extensionless python scripts in Travis. 2016-07-22 11:16:29 -07:00
Eklavya Sharma d745f20b1b Run frontend and backend tests in python3 on Travis. 2016-07-19 14:15:35 -07:00
Eklavya Sharma e86539649c Use tools/travis/activate-venv in test suites. 2016-07-19 14:15:35 -07:00
Eklavya Sharma bc901ac6d8 tools/travis/py3-backend: Add tools/test-management. 2016-07-18 09:57:28 -07:00
Tim Abbott 4972154df9 setup-production: Improve debugging of build-release-tarball errors. 2016-07-12 19:09:28 -07:00
Eklavya Sharma 934a0f7c6c Add tools/test-backend to py3-backend test suite. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 158d67e702 Run tools/test-migrations in Travis. 2016-07-11 21:30:32 -07:00
Eklavya Sharma 57ce3f4af1 tools/travis/lint-all: Factor out error message. 2016-07-11 21:30:32 -07:00
Eklavya Sharma 08a4555e0f Merge mypy and py3k test suites in Travis. 2016-07-11 21:28:01 -07:00
Eklavya Sharma aa68fd1679 Run tools/lint-all on Travis in python 3. 2016-07-08 10:43:48 -07:00
Eklavya Sharma 5e8dba4c75 Add mypy in python 3 mode to travis.
This helps catch Python 3 compatibility issues.
2016-07-07 09:44:00 -07:00
Tim Abbott 7ed0ab8c4a Ensure Zulip virtualenvs are world-readable.
This is important for both ensuring the Nagios checks work correctly
in production, as well as making sure the `zulip` user can access the
virtualenv (owned by the `travis` user) in Travis CI.
2016-06-28 12:03:01 -07:00
Tim Abbott beb8b50623 Move provision.py under tools/. 2016-06-27 19:04:32 -07:00
rahuldeve 3ca53df152 Add test for worker queue autoreloading. 2016-06-26 20:12:37 -07:00
Tim Abbott 039c175d68 production-helper: Hold tons of packages.
This saves almost a minute doing apt upgrades in the production test
suite.
2016-06-22 10:41:09 -07:00
Tim Abbott 9b436c7190 setup-production: Remove more irrelevant postgres packages. 2016-06-22 10:41:09 -07:00
Tim Abbott 5a109cf816 travis: Skip development database setup in production tests.
This saves about 20s on the runtime of the production build.
2016-06-22 10:41:09 -07:00
Tim Abbott 0f2729f5fb production-helper: use dist-upgrade to match install script.
Previously, we were wasting time every time we installed packages,
because `apt-get upgrade` would only install most of the packages
`apt-get dist-upgrade` would.
2016-06-22 10:38:27 -07:00