Commit Graph

308 Commits

Author SHA1 Message Date
vaibhav 8881b5eb9f Outgoing Webhook System: Check for @-mentioned outgoing webhook bots.
Also puts them into a processing queue, though the queue processor
does nothing.

Rewritten by tabbott to avoid unnecessary database queries in
do_send_messages.
2017-05-02 09:22:04 -07:00
Alexander Trost 889547ff5e configure-rabbitmq: Add support for RABBITMQ_NODE flag.
This can potentially be used by things like a Docker configuration
that runs RabbitMQ on another server.
2017-04-29 15:03:05 -07:00
K.Kanakhin e3e52e7284 email-mirror: Move postfix email mirror integration to separate script.
This fixes a performance problem where we were previously starting up
a full Django process (~0.7s even on a fast machine) every time a new
email came in, potentially allowing users to accidentally DoS a Zulip
server.  Now, we just post over HTTPS, allowing the existing thread
pool support to do its job.

- Add script wrapper to communicate postfix pipe with django web server
  over HTTP(S). It uses shared_secret authentication mode.
- Add django view to process messages from email mirror server.
- Clean management command `email-mirror`. Left just functional
  for cron email processing.
- Add routes for new tornado view.
- Change pipe script in master process postfix config template
  based on updated script.
- Add tests.

Tweaked by tabbott to adjust the directory and set better defaults.

Fixes #2421.
2017-04-24 21:24:23 -07:00
Feorlen 6ff78ca0e8 Set umask 022 before starting prod upgrade.
Follow-on from #2373/ PR https://github.com/zulip/zulip/pull/4316, to set an
appropriate umask also when upgrading so files have appropriate permissions.

I've tested this by starting from a clean install, deleting /srv/* so new
files are downloaded, and then doing an upgrade. It worked starting with both
a current version from master and an older release installed with a less
restrictive umask and then the umask changed.

Fixes #2373.
2017-04-19 10:28:06 -07:00
Tim Abbott 55a9101573 settings: Add support for ZULIP_ORG secrets.
These can be used to authenticate the current Zulip server to
zulip.org.
2017-04-18 23:00:10 -07:00
Feorlen 10ccfcdc8e Set umask 022 before starting prod install.
Fixes #2372.
2017-03-25 23:59:44 -07:00
Tim Abbott 5bf01fb7d4 create-production-venv: Fix issues with api/ relative path.
Fixes #4313.
2017-03-25 19:49:26 -07:00
K.Kanakhin 6a801db1c2 missed-emails-sending: Move email sending to separate queue worker.
- Add new 'missedmessage_email_senders' queue for sending missed messages emails.
- Add the new worker to process 'missedmessage_email_senders' queue.
- Split aggregation missed messages and sending missed messages email
  to separate queue workers.
- Adapt tests for sending missed emails to the new logic.

Fixes #2607
2017-03-07 20:08:40 -08:00
Tim Abbott 0afe832fc7 check-rabbitmq-consumers: Fix typing import issue. 2017-03-04 15:35:26 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Rishi Gupta 2bbfdeeb7b Fix more errors caught by mypy 0.501.
Another set of relatively easy to review changes.
2017-03-03 14:15:38 -08:00
Rishi Gupta 28d3af0965 Fix several new errors caught by mypy 0.501.
Clear out a bunch of easy to review errors, so we can focus on the more
complicated ones.
2017-03-03 14:12:52 -08:00
Tim Abbott f6f2c62caf zulip-puppet-apply: Fix running it as ./zulip-puppet-apply. 2017-02-27 15:25:14 -08:00
Tim Abbott fe0c4cad85 check-rabbitmq-consumers: Go back to hardcoding for now.
This should fix the production test suite in Travis CI, so that we can
debug what's broken here offline.
2017-02-22 22:58:59 -08:00
Tim Abbott b81add60fe check-rabbitmq-consumers: Fix queue_workers call. 2017-02-22 00:48:43 -08:00
Tim Abbott aa6567ee34 queue_workers: Fix confusing --queue_type argument name. 2017-02-22 00:23:26 -08:00
Tim Abbott 19896460f0 nagios: Fix RabbitMQ Nagios checks running Django as root.
This can cause problems by making the /var/log/zulip files owned by
root (not zulip) and thus not writable by the Zulip user.
2017-02-22 00:20:57 -08:00
Tim Abbott 51d3ab1cb7 initialize-database: Clean up final instructions.
Fixes #3678.
2017-02-21 20:19:16 -08:00
Tim Abbott 620f1e444e travis: Fix various bugs in new queue worker test.
* Now queue_workers.py sorts queue names and prints them on their own
  line.  Previously it's output was nondeterministic.
* Simplified grep strategy for removing the "test" worker.
2017-02-19 21:17:42 -08:00
Tim Abbott 333062f08e nagios: Automate queue list in check-rabbitmq-consumers. 2017-02-19 16:19:55 -08:00
Tim Abbott d6bbcd2737 travis: Automate updates to production-helper Nagios test.
This list was likely to end up out of date quickly, since it wasn't
documented that you need to update it when adding a queue.  The best
solution is to just not require it to be updated.
2017-02-19 16:19:53 -08:00
Tim Abbott 34046c1f55 check-rabbitmq-consumers: Add missing embed_links consumer. 2017-02-19 13:12:00 -08:00
Tim Abbott 213af24e47 check-rabbitmq-consumers: Reformat worker_queues list. 2017-02-19 13:12:00 -08:00
Tim Abbott 473c0ee1fe mypy: Remove now-unused type: ignores. 2017-02-18 17:01:01 -08:00
Tim Abbott 609082c475 puppet: Remove hardcoding of /root/zulip from puppet path.
This is an important prerequisite to being able to remove dependence
on the /root/zulip symlink altogether.
2017-02-08 11:13:19 -08:00
Tim Abbott 31d4f99573 upgrade: Stop trying to copy node_modules out of tarballs.
Now that we no longer use node_modules at all in production (it's only
used to generate static assets), we don't include `node_modules` in
the production tarballs, and thus we shouldn't attempt to copy
`node_modules` out of the production tarballs when installing.

Fixes a regression introduced in
d71f2e7b9b.
2017-02-07 10:39:31 -08:00
Tim Abbott 126b1c4edc upgrade: Don't require authors updates deploying from Git.
Fixes #3392.
2017-01-23 23:00:09 -08:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00
Tim Abbott de99f48ce7 lint: Clean up E401 PEP-8 rule. 2017-01-23 21:36:39 -08:00
Tim Abbott 22d1aa396b lint: Clean up W503 PEP-8 warning. 2017-01-23 20:50:04 -08:00
Ayush Goyal a85b539c4a zulip_tools: Improve color and copy for run() errors.
Tweaks to the text are edited by tabbott.
2017-01-17 14:37:15 -08:00
Tim Abbott 747f66bfe1 docs: Add README.md files for scripts/ and tools/. 2017-01-17 11:01:27 -08:00
Tim Abbott 1d5edff927 setup_venv: Give up if virtualenv-clone isn't working.
virtualenv-clone can sometimes fail if the old virtualenv is
broken; in that case, we can just make a new one.
2017-01-09 11:21:42 -08:00
Tim Abbott 54acbc41ed prod: Ensure the Zulip version of node is installed. 2017-01-06 16:18:29 -08:00
Tim Abbott e4b065e03a Move tools/setup/install-node to scripts/setup/. 2017-01-06 16:18:29 -08:00
Tim Abbott 374e900432 upgrade-zulip: Fix bug in migrations detection. 2017-01-06 16:18:29 -08:00
Tim Abbott e5fbea1007 upgrade-zulip: Move static asset compilation before shutdown.
This saves about a minute of downtime when using
upgrade-zulip-from-git in the default configuration.

It should also save several seconds of downtime when upgrading to a
production release tarball as well.
2017-01-06 14:04:54 -08:00
Tim Abbott 90ee8d1207 upgrade-zulip: Only pause for migrations if not current.
This saves about 1s of downtime when doing a no-op upgrade with the
default settings.  Small, but worthwhile.
2017-01-06 14:03:55 -08:00
Saumya Rawat d3a7d0202f dev: Auto-create log directory in management command.
Check 'zulip/var/log' folder exists and create if not as FileHandler only checks for existence of log file.
2016-12-17 17:18:49 -08:00
Umair Khan 336a041ac0 Django 1.10: Use uWSGI.
Fixes: #1121

With some tweaks by tabbott to make the number of processes configurable.
2016-12-13 21:40:43 -08:00
Umair Khan fda0387695 Create Python version agnostic venv link for prod.
This link points to either 'zulip-venv' or 'zulip-py3-venv' and makes
uWSGI configration very easy.
2016-12-13 21:40:43 -08:00
Tim Abbott 95a49f6ea2 node_cache: Fix buggy production deployment code.
The previous version of the production deployment code for release
tarballs did not correctly install the node_modules directory.
2016-12-08 10:34:04 -08:00
anirudhjain75 beaa62cafa mypy: Convert several directories to use typing.Text.
Specifically, these directories are converted: [analytics/, scripts/,
tools/, zerver/management/, zilencer/, zproject/]
2016-12-07 20:51:05 -08:00
AZtheAsian c23f9e0df7 pep8: Fix E111 violations 2016-12-01 14:18:11 -08:00
Tim Abbott fd7cb10964 install: Check whether the system has at least 2GB RAM.
This should eliminate a common class of user error installing Zulip.

Fixes #2290, fixes #2320.
2016-11-30 16:07:57 -08:00
Tommy Ip e4091c6413 pep8: Fix E222 violations. 2016-11-30 21:49:02 +00:00
Tommy Ip 46b7d54b3e pep8: Fix E701 violations. 2016-11-30 20:45:09 +00:00
Igor Tokarev e6ae53cbff install: Add clear error message if upstart is installed on Xenial.
Fixes #2199.
2016-11-29 19:16:26 -08:00
Rafid Aslam 7a2282986a pep8: Fix E225 pep8 violations. 2016-11-28 15:21:15 -08:00
Anders Kaseorg 78d6c3d7e9 install: Fix RabbitMQ node name if RabbitMQ is not installed.
This indirectly causes the RabbitMQ node name for new Zulip
installations to default to zulip@localhost, which would eliminate the
persistent problems we have had

Fixes #194, #465, #1375, #1751.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 18:54:29 -08:00