0fb6779899
docker: Make user and dbname configurable in process_fts_updates.
2017-03-23 14:28:21 -07:00
17c8527856
nagios: Clean up check_send_receive_time arguments.
2017-03-15 12:52:27 -07:00
0ff1f3d663
nagios: Error on heartbeat events in check_send_receive_time.
...
It was probably going to fail anyway if those show up, but this
produces a clearer failure mode.
2017-03-15 12:51:19 -07:00
6a5e98b77e
puppet: Increase MaxStartups SSH configuration.
2017-03-08 22:28:16 -08:00
1e441b8d7c
uwsgi: Add master mode to the main uwsgi process.
...
- Enable `master` parameter for `uswgi` configuration.
It allows cleaning leaked processes if the parent
process is closed unexpectedly or with SIGKILL command.
Child processes follow to the master and kill themselves
after the main process.
Fixes #3855
2017-03-07 21:35:51 -08:00
6a801db1c2
missed-emails-sending: Move email sending to separate queue worker.
...
- Add new 'missedmessage_email_senders' queue for sending missed messages emails.
- Add the new worker to process 'missedmessage_email_senders' queue.
- Split aggregation missed messages and sending missed messages email
to separate queue workers.
- Adapt tests for sending missed emails to the new logic.
Fixes #2607
2017-03-07 20:08:40 -08:00
67219cf660
puppet: Use restart-server for weekly server restarts.
...
Using `supervisorctl restart all` carried longer downtime (since it
just restarts everything at the same moment) and was less under our
control; I'm not sure it had any advantages.
2017-03-05 11:36:10 -08:00
ab2ce9d5e9
lint: Fix whitespace error with recent mypy annotations.
2017-03-04 15:41:11 -08:00
75e81253f2
mypy: Work around several new mypy bugs in 0.501.
2017-03-04 15:33:39 -08:00
a3a03bd6a5
mypy: Added Dict, List and Set imports.
...
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
3d07ac0c49
Change timezone-naive datetimes to use timezone.now() where safe to do so.
...
Change timezone-naive datetimes to use timezone.now() in cases where there
is no change in behavior.
2017-03-01 22:54:28 -08:00
0218422e96
Use time.time() instead of datetime.now() to measure elapsed time.
...
Both because it is more idiomatic and because we will soon start enforcing
that all datetimes in Zulip are timezone aware.
2017-03-01 22:54:28 -08:00
aed3632cbb
puppet: Convert remaining queue workers to _ style.
2017-02-19 16:18:37 -08:00
4b54307d94
puppet: Generate most of main supervisor config with template.
2017-02-19 16:18:37 -08:00
ae09d55e46
puppet: Move zulip.conf to be a template file.
2017-02-19 16:18:37 -08:00
a2d935a2ee
puppet: Fix PostgreSQL user to create PGroonga extension
...
"root" user isn't a PostgreSQL administrator. "postgres" is a PostgreSQL
administrator.
2017-02-08 12:57:56 -08:00
fa8045a484
puppet: Add websockets Nagios test to configuration.
...
Since browser clients send messages via websockets and not the API,
this is an important element in making sure mission-critical Zulip
functionality is working.
2017-02-08 11:13:19 -08:00
8db13d0bb9
check_send_receive_time: Use a different state file for websockets.
...
Otherwise, the two Nagios checks will fight over the same state file
if both are in use.
2017-02-08 11:13:19 -08:00
ba5f454be5
puppet: Extract zulip::analytics.
...
I'm not altogether happy with this (a better solution would be
database-level locking), but I think it solves the immediate problem
of folks with 2 servers being very likely to run analytics on both of
them.
2017-02-07 12:29:15 -08:00
70388b17d2
puppet: Add missing dependency on ssl-cert.
2017-02-06 15:51:38 -08:00
76f3d02590
analytics: Add cron job to run analytics jobs.
...
This adds a cron job to update the Zulip analytics counts, complete
with locking etc.
Substantially tweaked by tabbott.
2017-02-01 17:02:46 -08:00
2fb51ff876
puppet: Use SIGINT to restart uwsgi.
...
This results in a brief service interruption (not a graceful restart),
but fixes a bug where on a `supervisorctl restart zulip-django`, we'd
end up leaking a bunch of uwsgi processes.
The mechanism was that sending SIGHUP to uwsgi was a command for it to
gracefully restart, so it'd start doing that (whereas supervisor
expected it to be dying)... and then supervisor would start up the new
uwsgi process group, resulting in 2 uwsgi process groups running.
This, in turn, led to a memory leak that could eventually result in
OOM kills.
2017-01-28 22:26:12 -08:00
36d54cf5ff
Replace references to zulip.com/dist with zulip.org/dist.
...
Now that zulip.org has all the files to distribute, there's no reason
to still point to the soon-to-be-decommissioned zulip.com/dist.
2017-01-28 17:56:25 -08:00
0ce29d7ad6
Remove some some duplicate words in copy.
2017-01-23 23:15:04 -08:00
4e171ce787
lint: Clean up E126 PEP-8 rule.
2017-01-23 22:06:13 -08:00
d6e38e2a5c
lint: Clean up E123 PEP-8 rule.
2017-01-23 21:34:26 -08:00
bde2da7dfd
lint: clean up PEP-8 W391 rule.
2017-01-23 20:39:02 -08:00
bbd853e208
puppet: Add redirect to https to zulip.org.
2017-01-22 21:52:50 -08:00
44776c43a1
puppet: Add configuration for zulip.org website.
...
This puppet configuration, plus cloning the zulip.github.io repo and
letsencrypt key setup, is all we need to run a zulip.org server.
2017-01-22 21:48:48 -08:00
9de75f5167
Fixed typos with separate
2017-01-12 04:52:05 -08:00
ff8639f9db
Fixed typos with threshold.
2017-01-12 04:50:20 -08:00
5008f45112
Fixed typo in munin.conf.erb
2017-01-12 04:49:19 -08:00
3e32102016
nagios: Fix various critical issues not tagged as pageable.
2017-01-06 21:49:20 -08:00
edebf7619b
puppet: Add PAM common_session disabling systemd-login.
...
This fixes a weird problem with systemd where logging into a server
via ssh frequently has a 15s+ lag.
2017-01-06 21:49:15 -08:00
93c2c19775
nagios: Increase process count limits.
2017-01-06 21:49:15 -08:00
2c6cb37385
munin: Add default munin configuration template.
2017-01-06 21:44:57 -08:00
9ab8e7ba34
nagios: Disable swap checks for servers with no swap.
2017-01-06 21:39:07 -08:00
3e01ed1f73
nagios: Increase NTP max_check_attempts.
...
NTP often suffers from brief interruptions of service that lead to
spurious Nagios alerts; it makes sense to suppress these.
2017-01-06 21:32:43 -08:00
e4420b08d2
zulip_ops: Disable unattended upgrades of security packages.
...
Since Zulip does not handle e.g. postgres server restarts gracefully,
it's best for a system administrator to manually trigger security
updates.
2017-01-06 21:30:56 -08:00
6f9c73d0e5
zmirror: Update Debathena release in configuration.
...
The zulip_ops configuration is now for xenial, not obsolete wheezy.
2017-01-06 21:30:41 -08:00
bd9176d1d9
nagios: Remove some default files.
...
Nagios ships with a bunch of default configuration files that one
needs to delete in order to configure it.
2017-01-06 21:25:12 -08:00
7083899e77
zulip_ops: Add postgres config for enabling Nagios.
...
The old zulip_ops Nagios configuration depended on Nagios having the
ability to login as the zulip user (with essentially full write
access); this configuration is helpful for limiting nagios to special
"nagios" user with more limited credentials.
2017-01-06 21:24:24 -08:00
204edb0f85
zulip_ops: Cleanup pg_hba.conf configuration.
2017-01-06 21:23:51 -08:00
30c57eb2ae
zulip_ops: Add basic .emacs for production.
2017-01-06 21:20:21 -08:00
eb87d04168
puppet: Remove xxxxx password hardcoding in recovery.conf.
2017-01-06 21:20:21 -08:00
6404a1a5ff
zulip_ops: Add nagios-plugins-contrib.
...
This has a number of useful nagios plugins.
2017-01-06 21:19:59 -08:00
f7b77008ef
zulip_ops: Add aptitude dependency.
...
This is useful for `aptitude why`.
2017-01-06 21:19:50 -08:00
2510a51a8a
zulip_ops: Add letsencrypt dependency.
2017-01-06 21:19:31 -08:00
65774e1c4f
zulip_ops: use check_postgres package from apt.
2017-01-06 21:18:55 -08:00
165b4d3126
nagios: Fix check_send_receive_time threshholds.
...
Previously, the CRITICAL state would never fire (because x > 6 =>
x > 3). Additionally, 6s is not so unusually high as to deserve being
immediately pageable.
2017-01-06 21:16:37 -08:00
0d8c18a6dd
nagios-plugins: Add websocket checking to nagios message sending test.
...
- Add websocket client to create connection with SockJS websocket server.
It contains callback method to launch after connection setup.
- Add '--websocket' parameter to 'check_send_receive_time' script to
check websocket connection.
- Add testing websocket connection to production installation checking.
- Add cronjob to launch websocket connection nagios test.
This makes it possible for Zulip Nagios monitoring to check for
problems impacting the websockets sending code path, which is what all
web users use.
2016-12-30 15:36:37 -08:00
336a041ac0
Django 1.10: Use uWSGI.
...
Fixes : #1121
With some tweaks by tabbott to make the number of processes configurable.
2016-12-13 21:40:43 -08:00
c93f1d4eda
Add oembed/Open Graph/Meta tags data retrieval from inline links.
...
This change adds support for displaying inline open graph previews for
links posted into Zulip.
It is designed to interact correctly with message editing.
This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.
By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.
Eventually, we may want to make this manageable via a (set of?)
per-realm settings. E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
2016-12-07 17:40:18 -08:00
144d82305d
mypy: Annotate puppet/zulip_ops.
2016-12-03 11:00:25 -08:00
adebc75740
pep8: Fix E502 violations
2016-12-03 10:56:36 -08:00
8c0c12c1d9
pep8: Fix E303 violations.
2016-12-02 15:34:11 -08:00
c5316b4002
lint: Fix E127 pep8 violations.
...
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
6b0df43463
pep8: Fix E125.
2016-11-30 20:03:29 -08:00
46b7d54b3e
pep8: Fix E701 violations.
2016-11-30 20:45:09 +00:00
7a2282986a
pep8: Fix E225 pep8 violations.
2016-11-28 15:21:15 -08:00
092fe4fecb
puppet: Write rabbitmq-env.conf before installing rabbitmq-server
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 18:54:12 -08:00
207cf6302b
Always start python via shebang lines.
...
This is preparation for supporting using Python 3 in production.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
1ea8abe493
Replace python -u with PYTHONUNBUFFERED=1
...
(Why is -u needed at all? I’m not sure, but test-run-dev spins forever
“Polling run-dev...” without it.)
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 13:20:22 -08:00
d1dc2cf30e
Mark scripts executable
...
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 13:20:22 -08:00
18e49bec3a
puppet: Add another missing dependency on postgresql-common
...
The postgres group must exist before we give files to it.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 11:23:32 -08:00
f78eb9d82d
puppet: Add missing dependency on postgresql-common.
...
postgresql-$(version) depends on postgresql-common depends on ssl-cert,
which creates the ssl-cert group.
2016-11-21 07:38:45 -08:00
2e65dc1206
puppet: make check_send_receive_time target host configurable.
2016-11-02 23:40:53 -07:00
9578908601
check_send_receive_time: Stop hardcoding EXTERNAL_API_URI.
2016-11-02 23:35:08 -07:00
eceaf36001
setup_disks: Fix postgres RAID setup to work correctly on Xenial.
...
Nobody's going to run this on Wheezy again.
2016-10-28 11:04:08 -07:00
9b7a3f040c
Remove now-unused /json/get_events endpoint.
2016-10-27 21:34:58 -07:00
4fbe201187
puppet: Automate autossh process monitoring maintenance.
...
Previously, the Zulip Nagios configuration effectively hardcoded the
count for how many system should have autossh connections.
2016-10-26 00:49:03 -07:00
6bdb10b71b
puppet: Update emacs dependency to emacs-nox metapackage.
...
This way, one doesn't need to keep updating the dependency every time
a new major emacs release comes out.
2016-10-26 00:42:22 -07:00
11b5d203f7
sshd_config: Increase MaxStartups.
...
This fixes connection problems when using the full Zulip recommended
Nagios configuration against a given server.
2016-10-26 00:41:03 -07:00
73f54dd0cb
sshd_config: Add updates from Xenial upstream.
...
It seems worth updating this to match the Linux distro this
configuration targets.
2016-10-26 00:40:44 -07:00
0a5a2c4eda
nagios: Automate authorized users list maintenance.
2016-10-26 00:37:29 -07:00
fa4998db59
puppet: Add zulip_zephyr_mirror plugins.
2016-10-26 00:35:57 -07:00
ac4f28050c
zmirror: Remove unnecessary krb5-clients dependency.
...
I'm pretty sure krb5-clients isn't needed to run the Zephyr mirroring
service.
2016-10-26 00:35:11 -07:00
d490e83645
puppet: Upgrade nagios cgi.cfg with modern defaults.
2016-10-26 00:31:41 -07:00
1159ad4857
puppet: Upgrade nagios.cfg with modern defaults.
2016-10-26 00:31:41 -07:00
73178e5e5a
puppet: Run check_send_receive_time via a cron job.
...
This allows the actual nagios work involved with
check_send_receive_time nagios checks to be done by an unprivileged
"nagios" user rather than the "zulip" user.
2016-10-26 00:26:52 -07:00
96cf330649
puppet: ssh as the nagios user instead of zulip user.
...
This is a follow-up to 4f58fef54b
2016-10-26 00:23:47 -07:00
a350d43683
puppet: Add recovery.conf configuration to postgres_slave.pp.
...
This file is needed to run a valid postgres slave; it's not clear why
this wasn't installed in the original zulip.com configuration.
2016-10-26 00:22:57 -07:00
c3727c9886
nagios: Remove old zulip.com trac/git/replica servers.
...
These are unlikely to be relevant to anyone.
2016-10-26 00:21:53 -07:00
383f39b543
nagios: Enable allow_empty_hostgroup_assignment.
...
This fixes the configuration being broken when we remove some of the
old zulip.com hosts that are unlikely to be of interest to anyone.
2016-10-26 00:19:21 -07:00
4f58fef54b
zulip_ops: Use nagios user for all Nagios checks.
...
There's no reason these Nagios checks needs to run as the
semi-priviliged Zulip user.
2016-10-26 00:17:26 -07:00
32d244dbe5
puppet: Add Nagios checks for other consumers.
2016-10-26 00:11:08 -07:00
f1fa4397f3
puppet: Fix package deps for zulip-ec2-configure-interfaces.
2016-10-26 00:11:08 -07:00
3448ab4c7a
zulip-ec2-configure: Fix network IDs for Ubuntu Xenial.
2016-10-26 00:11:08 -07:00
91da4bd59b
puppet: Add check_cron_file generic helper.
2016-10-26 00:11:08 -07:00
080dd8c987
nagios: Ignore kthreads in check_procs tests.
...
Modern Linux can have a lot of kernel threads not doing anything.
Since this isn't interesting from a monitoring perpsective, we ignore
these.
2016-10-26 00:10:40 -07:00
4c9a283542
puppet: Remove configuration for old builder host.
...
I don't think this configuration was ever even used; it's just
clutter.
2016-10-26 00:01:52 -07:00
f9ad75f58e
puppet: Remove configuration for old zulip.com bots host.
...
This configuration didn't do anything anyway and just clutters the
repo.
2016-10-26 00:01:29 -07:00
9d4f3f1e1b
puppet: Replace zulip_ops postgres configs with postgres_appdb_tuned.
...
There's no longer a reason to have copies of forked postgres
configuration files in our repository, since some time ago we merged
the features of these configuration files into the main
postgres_appdb_tuned.pp.
2016-10-25 23:58:53 -07:00
105ea972f6
puppet: Remove now-unncessary kernel.shm sysctl values.
...
With modern Linux and postgres, these settings are not required.
2016-10-25 23:58:33 -07:00
2227e77cce
puppet: Remove Dropbox usernames from Nagios config.
2016-10-25 23:55:42 -07:00
8584c05d80
zulip_ops: Remove unnecessary loadbalancer stanzas.
2016-10-25 23:52:37 -07:00
624ee3989f
puppet: Remove old Dropbox certificates.
2016-10-25 23:52:30 -07:00
f0bb78ba2d
puppet: Fix iptables-persistent->netfilter-persistent rename.
2016-10-25 23:45:21 -07:00
c4ca7ee6e1
puppet: Move Apache sites files to correct paths.
...
Apache now actually requires its configuration files have names ending
with .conf.
2016-10-25 23:44:28 -07:00
2b8324b778
emoji: Fix caching permissions issues.
...
Previously, you needed to be root to update the emoji cache, which
caused problems with how Zulip is upgraded in production.
2016-10-25 17:52:19 -07:00
Alexander Trost
Tim Abbott
K.Kanakhin
Raghav Jajodia
Rishi Gupta
Kouhei Sutou
umkay
Eitan Adler
JefftheBest1
Tim Abbott
Umair Khan
Igor Tokarev
Jason Le
bulat22101
Sidhant Bhavnani
Rafid Aslam
Bickio
Tommy Ip
Anders Kaseorg