Commit Graph

622 Commits

Author SHA1 Message Date
Tim Abbott acb0b6ee43 process_fts_updates: Fix pgroonga search in development.
For some reason, we have the USING_PGROONGA setting on in development
right now.  I'm going to disable that in another commit to match what
we're doing in production, but we'll still want that setting to work
in development.

The problem here was that process_fts_updates only attempted to read
the USING_PGROONGA setting from a /etc/zulip/zulip.conf source, and
thus would just not be updating the index in development.
2017-10-30 11:44:04 -07:00
Tim Abbott 0d1194811f mypy: Remove ignores for a few typeshed bugs fixed upstream. 2017-10-27 17:09:00 -07:00
Tim Abbott 89b97e7480 python3: Fix REMOTE_USER Apache configuration for Python 3.
We were previously still installing the Python 2 version of mod_wsgi,
which of course doesn't work and can't use the Zulip virtualenv.
2017-10-24 11:48:14 -07:00
Tim Abbott 15f3d5f714 nginx: Fix some buggy gzip compression configuration.
We weren't compressing SVG, while at the same time were incorrectly
compressing octet-stream (Which meant downloading .tar.gz files in
Chrome would get double-compressed).
2017-10-20 11:01:28 -07:00
Tim Abbott 540cae19a8 puppet: Remove obsolete sparkle configuration.
Sparkle was the auto-update system used by the legacy desktop app.  We
haven't been capable of using it for auto-update in years, so there's
no reason to keep around the configuration.

The new Electron app uses a different system anyway.
2017-10-19 16:35:55 -07:00
rht b57289aacd py3: Remove all `from __future__ import print_function.
Except for these files:
- tools/linter_lib/*
- tools/lib
- tools/lister.py
2017-10-18 12:07:19 -07:00
rht 2f3ae84e5a py3: Remove all `__future__ import division`. 2017-10-17 23:09:12 -07:00
Tim Abbott 6a5cb0e48c puppet: Make problems with Zephyr mirroring pageable.
Generally this indicates sending messages is completely broken.
2017-10-12 00:16:32 -07:00
rht de30400fc5 pg_backup_and_purge.py: Remove .py extension. 2017-10-08 15:32:43 -07:00
Tim Abbott 47c5aae5b2 log2zulip: Enforce using python 3 in cron job.
We aren't guaranteed to have the Zulip dependencies installed on
Python 2.
2017-10-06 16:37:17 -07:00
Tim Abbott 0f2e4a55c0 soft deactivation: Shorten management command name.
This command is really for soft deactivation; there's just an undo
feature.
2017-10-06 08:48:43 -07:00
Tim Abbott f2055397c1 nagios: Update apache configuration to be generated.
Since this is basically just stock Apache configuration for Nagios
with a hostname put in, we can just fetch the hostname from our
configuration.
2017-10-05 21:51:29 -07:00
Tim Abbott 3af01bed85 puppet: Simplify zulip_ops nginx configuration.
Whatever dist/ functionality this had in 2014 is now served by
zulip.org, and since this serves as a sample, it should be as simple
as possible.

Previously, this was more cluttered than it needed to be.
2017-10-05 21:17:57 -07:00
Tim Abbott e6e7bcf6e1 nagios: Move camo_check_url into configuration. 2017-10-05 21:09:24 -07:00
Tim Abbott 82cee4fde9 check_worker_memory: Increase limits for what leaking means.
The old limits were such that these would sometimes oscillated too
high and page erroneously.  The purpose of this check is to prevent
large memory leaks, and will still achieve that with a higher limit.
2017-10-05 20:54:03 -07:00
Tim Abbott 1c453fdf2a puppet: Add redis_password file for Nagios.
This allows the Nagios user to access redis without having full access
to the redis system.  Ideally, this would eventually use a password
that only has statistics read access, but I'm not sure redis supports
that.
2017-10-05 20:42:07 -07:00
Tim Abbott 13a36d9af3 puppet: Make old redis_tunnel configuration usable.
This old puppet configuration was never really used, and regardless
hardcoded an ancient zulip.net hostname.  We fix this to use the
zulipconf system to get the host domain (though not, at present, the
hostname).
2017-10-05 20:40:22 -07:00
Tim Abbott 96c3014da0 nagios: Automate configuration of outgoing email with msmtp.
Now we no longer need to check in a bunch of hostnames in order to
configure Nagios.
2017-10-05 20:29:47 -07:00
Tim Abbott 5b4c260c3f puppet: Add munin apache auth configuration.
This is completely stock configuration, and seems to be required for
munin to run properly.
2017-10-05 20:17:12 -07:00
Tim Abbott ba7be4102e puppet: Update munin tunnels configuration to use zulipconf.
This eliminates another old hardcoding of zulip.net.
2017-10-05 20:14:43 -07:00
Tim Abbott 162eaf8917 nagios: Modify check for swap to allow no swap.
If a machine is configured with no swap intentationally, that
shouldn't be a Nagios problem.  This alert is intended to flag
machines which are swapping.
2017-10-05 20:07:44 -07:00
Tim Abbott 80a16bf873 nagios: Fix path to source zulip_nagios.cfg.
Arguably, we should make this a symlink, but it's probably a good idea
to have every change in the production Nagios configuration go through
the zulip-puppet-apply diff experience.
2017-10-05 20:06:50 -07:00
Tim Abbott 886a8853ac nagios: Move server-specific config into hostgroups.
These new hostgroups exist so we can eliminate explicit references to
individual hosts in services.cfg.
2017-10-05 20:06:48 -07:00
Tim Abbott b6ce9583a9 nagios: Fetch list of hosts from zulip.conf.
This makes this much more configurable and much less hardcoded.
2017-10-05 20:06:30 -07:00
Tim Abbott 5193936bc3 nagios: Add Memcached and Redis monitoring.
These are standard Nagios plugins that might be sometimes helpful.
2017-10-05 20:06:16 -07:00
Tim Abbott f7d554d533 nagios: Rename zmirror2 to zmirrorp in configuration.
The "p" stands for "personals", aka zephyr private messages, which is
what this host manages.
2017-10-05 20:06:08 -07:00
Tim Abbott 062d280914 puppet: Clean up unnecessary pagerduty_nagios.cfg. 2017-10-05 19:23:33 -07:00
Tim Abbott 7e328ba865 nagios: Move email addresses for contacts into variables. 2017-10-05 19:23:33 -07:00
Tim Abbott 6017d3dec5 puppet: Move contacts.cfg to be a template. 2017-10-05 19:23:33 -07:00
Tim Abbott 09aec3e467 puppet: Move hosts.cfg to be managed by a template. 2017-10-05 19:23:33 -07:00
Tim Abbott 692f4b77d1 puppet: Remove messy Nagios crontab. 2017-10-05 19:23:33 -07:00
Tim Abbott 26982ff55f puppet: Remove pageduty_nagios.pl.
This hasn't been used in like 4 years, and clutters the repo.
2017-10-05 18:46:09 -07:00
Tim Abbott 5a80c029a2 nagios: Update path to sync_public_streams to match new config. 2017-10-05 13:34:27 -07:00
Tim Abbott fdd021fd6a zephyr-mirror: Update supervisor configuration for repository split.
This now points to the path of the integration in the new package.
2017-10-05 13:18:37 -07:00
Tim Abbott 1eff717146 zephyr-mirror: Update cron job to use python-zulip-api.
This is a deferred follow-up project to the repository split.
2017-10-05 13:07:45 -07:00
Alexander Trost b33126a3e5 Add Docker-specific puppet files.
Based on #450, with commits
restructured by Rein Zustand.
Additionally, tweaked by Rein Zustand for typo mechanics and syntax.
2017-09-30 09:55:48 -07:00
hollywoodno ee7d4808fe emails: Add log for outgoing emails.
Fixes #5900.
2017-09-25 15:37:09 -07:00
Tim Abbott 3708f5708a upgrade: Include zulip::static_asset_compiler by default.
Since we've found that it's fairly frequent that we want to recommend
to developers that they upgrade to a version of Zulip from Git, it
makes sense to include that by default.
2017-09-22 11:38:20 -07:00
Tim Abbott 64e950ae48 puppet: Add curl as a dependency.
It's needed by scripts/install-yarn.sh.  This hadn't been discovered
because most systems end up having curl installed even though it isn't
technically a required package.
2017-09-19 19:14:25 -07:00
Tim Abbott 6c4006625d nginx: Update zulip.org nginx configuration.
* Added zulipbot to configuration.
* Added redirect to zulipchat.com for homepage.
2017-09-16 02:09:53 -07:00
Tim Abbott 2d11a67975 soft_deactivation: Change cron job to run daily.
This seems like a more consistent default model than weekly.
2017-08-27 18:40:31 -07:00
Aditya Bansal b232563e12 soft-deactivation: Add cron job for weekly soft deactivating users. 2017-08-27 11:33:06 -07:00
Greg Price d02101a401 APNs: Rip out the existing, broken implementation.
This code empirically doesn't work.  It's not entirely clear why, even
having done quite a bit of debugging; partly because the code is quite
convoluted, and because it shows the symptoms of people making changes
over time without really understanding how it was supposed to work.

Moreover, this code targets an old version of the APNs provider API.
Apple deprecated that in 2015, in favor of a shiny new one which uses
HTTP/2 to meet the same needs for concurrency and scale that the old
one had to do a bunch of ad-hoc protocol design for.

So, rip this code out.  We'll build a pathway to the new API from
scratch; it's not that complicated.
2017-08-26 14:16:05 -07:00
Tim Abbott d449346bb5 puppet: Fix process leaks in small-server configuration.
Whenever you restarted supervisord services, we'd end up leaking one
process from the process_queue group, eventually resulting in running
out of memory.

Fixes #6184.
2017-08-25 12:36:29 -07:00
Tim Abbott f91a2a8ef2 supervisord: Improve memory footprint for small servers.
This cuts the number of uwsgi and message_sender works for smaller
servers with only 2GB of RAM.
2017-08-25 12:34:41 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Tim Abbott 989cfb1963 mypy: Fix process_fts_updates annotations. 2017-08-15 22:04:52 -07:00
Tim Abbott a38d846e8b process_fts_updates: Fix check for USING_PGROONGA to not use settings.
The Zulip server's settings are only available if process-fts-updates
is running is on the same server as a Zulip production deployment.  So
we instead check whether we have pgroonga configured in
/etc/zulip/zulip.conf.
2017-08-15 20:33:36 -07:00
Greg Price e4d1d22e9f py3: Explicitly keep our wal-e PostgreSQL replication on Python 2.
On `trusty` there is no package for `boto` or `gevent` on Python 3, both
of which are dependencies of `wal-e` (at the version we've pinned.)  This
is something used only on database servers and only in a replication
scenario, and it doesn't involve any of our code outside the wal-e repo,
so the Python version it uses is quite independent of the Zulip
application server itself and the rest of our code.  For now, keep it
explicitly on Python 2 while we move forward for most everything else.
2017-08-15 17:30:31 -07:00
Greg Price 2a4d851a7c py3: Explicitly keep one boto-using ops script on Python 2.
This script in `zulip_ops` is handy for managing EC2 instances.  It uses
`boto`, which isn't available in `trusty` for Python 3.  The use of
`boto` here isn't particularly deep, so we could replace it with some
more manual HTTP calls if it comes to that.  For now, just mark it to
stay on Python 2 while we move the app and all the rest of the ops code
(except this and another straggler or two) to Python 3.

Also make a comment on this package in the Puppet manifest clearer
about what it specifically refers to.
2017-08-15 17:30:31 -07:00