Commit Graph

146 Commits

Author SHA1 Message Date
Tim Abbott d98be2f19f puppet: Only run analytics Nagios checks on machine running cron.
Running this on additional machines would be redundant; additionally,
the FillState checker cron job runs only on cron systems, so this will
crash on other app frontends.
2018-03-06 13:38:27 -08:00
Tim Abbott 8e8faab006 puppet: Move clearsessions cron job to app_frontend_once.
While this is a different system than I'd written up in #8004, I think
this is a better solution to the general problem of cron jobs to run
on just one server.

Fixes #8004.
2018-03-06 13:35:51 -08:00
Tim Abbott 3ae645ed12 puppet: Rename analytics.pp to app_frontend_once.pp. 2018-03-06 13:35:51 -08:00
Tim Abbott 24b6106c9c puppet: Dsiable checking for evictions in memcached nagios.
Zulip's caching model for message history is such that it is normal
and healthy for there to eventually be a nontrivial volume of
evictions.
2018-03-06 13:34:02 -08:00
Greg Price 4475950ddf queue: Restore prematurely-cut upgrade path.
Revert c8f034e9a "queue: Remove missedmessage_email_senders code."
As the comment in the code says, it ensures a smooth upgrade path
from 1.7.x; we can delete it in master after 1.8.0 is released.
The removal commit was merged early due to a communication failure.
2018-02-28 11:15:53 -08:00
Umair Khan c8f034e9a0 queue: Remove missedmessage_email_senders code.
After 68513952fb, all emails are sent through email_senders queue.
This commit removes code related to the legacy queue.
2018-02-21 16:43:56 -08:00
Tim Abbott 005b0fb566 puppet: Clean up ssh authorized_keys configuration rules. 2018-02-09 16:37:03 -08:00
Tim Abbott aca25b6f0a puppet: Move ssh configuration to use notify.
This handles more correctly the case where we're using the upstream
sshd_config file.
2018-02-09 16:37:03 -08:00
Tim Abbott 486de8abfc puppet: Edit some rules to support chat.zulip.org.
This should make it possible to use the zulip_ops base rules
successfully on chat.zulip.org.  Many of the changes in this commit
are hacks and probably can be cleaned up later, but given that we plan
to drop trusty support soon, it's likely that most of them will simply
be deleted then.
2018-02-09 16:37:03 -08:00
Rishi Gupta 1d581a9c6e nagios: Add nagios check for analytics state.
This should help us detect issues where the analytics cron jobs aren't
running properly.

The cron/nagios part of the implementation done by tabbott.
2018-02-09 16:36:05 -08:00
Tim Abbott 9ed2a94b8c nagios: Add configuration designed for full-stack servers.
This doesn't yet pass all Nagios checks correctly, and still has a few
flaws:
* The ideal setup code for the `nagios` user in the database isn't included.
* Some of the other details are a bit off; we need to split some host roles.

But it's better than nothing, and we can iterate from here.
2018-01-24 14:16:03 -08:00
Tim Abbott 2365b13b68 puppet: Move postgres Nagios plugin to main postgres-common.
This plugins package is required in order to use Nagios checks to
verify the Zulip postgres database, and thus belongs in the default
package set.
2018-01-23 10:31:48 -08:00
Umair Khan 68513952fb email-worker: Create EmailSendingWorker.
This commit just copies all the code from MissedMessageSendingWorker
class to a new EmailSendingWorker class. All the logic to send an email
through a queue was already there. This commit only makes the logic
generic. It does so by creating a special purpose queue called
'email_senders' to send any type of email. To make
MissedMessageSendingWorker still work we derive it from
EmailSendingWorker. All the tests that were testing
MissedMessageSendingWorker now run against EmailSendingWorker.
2017-12-20 19:36:27 -08:00
Vishnu Ks 766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Tim Abbott 94554c65da certbot: Modify nginx configuration to support automated renewal. 2017-11-08 12:32:26 -08:00
Tim Abbott 62bb465896 puppet: Modify lb0 nginx configuration. 2017-11-08 12:32:26 -08:00
rht 549a26860f refactor: Remove six.moves.range import. 2017-11-07 10:46:42 -08:00
Tim Abbott 0d1194811f mypy: Remove ignores for a few typeshed bugs fixed upstream. 2017-10-27 17:09:00 -07:00
Tim Abbott 540cae19a8 puppet: Remove obsolete sparkle configuration.
Sparkle was the auto-update system used by the legacy desktop app.  We
haven't been capable of using it for auto-update in years, so there's
no reason to keep around the configuration.

The new Electron app uses a different system anyway.
2017-10-19 16:35:55 -07:00
rht b57289aacd py3: Remove all `from __future__ import print_function.
Except for these files:
- tools/linter_lib/*
- tools/lib
- tools/lister.py
2017-10-18 12:07:19 -07:00
rht 2f3ae84e5a py3: Remove all `__future__ import division`. 2017-10-17 23:09:12 -07:00
Tim Abbott 6a5cb0e48c puppet: Make problems with Zephyr mirroring pageable.
Generally this indicates sending messages is completely broken.
2017-10-12 00:16:32 -07:00
rht de30400fc5 pg_backup_and_purge.py: Remove .py extension. 2017-10-08 15:32:43 -07:00
Tim Abbott 47c5aae5b2 log2zulip: Enforce using python 3 in cron job.
We aren't guaranteed to have the Zulip dependencies installed on
Python 2.
2017-10-06 16:37:17 -07:00
Tim Abbott f2055397c1 nagios: Update apache configuration to be generated.
Since this is basically just stock Apache configuration for Nagios
with a hostname put in, we can just fetch the hostname from our
configuration.
2017-10-05 21:51:29 -07:00
Tim Abbott 3af01bed85 puppet: Simplify zulip_ops nginx configuration.
Whatever dist/ functionality this had in 2014 is now served by
zulip.org, and since this serves as a sample, it should be as simple
as possible.

Previously, this was more cluttered than it needed to be.
2017-10-05 21:17:57 -07:00
Tim Abbott e6e7bcf6e1 nagios: Move camo_check_url into configuration. 2017-10-05 21:09:24 -07:00
Tim Abbott 1c453fdf2a puppet: Add redis_password file for Nagios.
This allows the Nagios user to access redis without having full access
to the redis system.  Ideally, this would eventually use a password
that only has statistics read access, but I'm not sure redis supports
that.
2017-10-05 20:42:07 -07:00
Tim Abbott 13a36d9af3 puppet: Make old redis_tunnel configuration usable.
This old puppet configuration was never really used, and regardless
hardcoded an ancient zulip.net hostname.  We fix this to use the
zulipconf system to get the host domain (though not, at present, the
hostname).
2017-10-05 20:40:22 -07:00
Tim Abbott 96c3014da0 nagios: Automate configuration of outgoing email with msmtp.
Now we no longer need to check in a bunch of hostnames in order to
configure Nagios.
2017-10-05 20:29:47 -07:00
Tim Abbott 5b4c260c3f puppet: Add munin apache auth configuration.
This is completely stock configuration, and seems to be required for
munin to run properly.
2017-10-05 20:17:12 -07:00
Tim Abbott ba7be4102e puppet: Update munin tunnels configuration to use zulipconf.
This eliminates another old hardcoding of zulip.net.
2017-10-05 20:14:43 -07:00
Tim Abbott 162eaf8917 nagios: Modify check for swap to allow no swap.
If a machine is configured with no swap intentationally, that
shouldn't be a Nagios problem.  This alert is intended to flag
machines which are swapping.
2017-10-05 20:07:44 -07:00
Tim Abbott 80a16bf873 nagios: Fix path to source zulip_nagios.cfg.
Arguably, we should make this a symlink, but it's probably a good idea
to have every change in the production Nagios configuration go through
the zulip-puppet-apply diff experience.
2017-10-05 20:06:50 -07:00
Tim Abbott 886a8853ac nagios: Move server-specific config into hostgroups.
These new hostgroups exist so we can eliminate explicit references to
individual hosts in services.cfg.
2017-10-05 20:06:48 -07:00
Tim Abbott b6ce9583a9 nagios: Fetch list of hosts from zulip.conf.
This makes this much more configurable and much less hardcoded.
2017-10-05 20:06:30 -07:00
Tim Abbott 5193936bc3 nagios: Add Memcached and Redis monitoring.
These are standard Nagios plugins that might be sometimes helpful.
2017-10-05 20:06:16 -07:00
Tim Abbott f7d554d533 nagios: Rename zmirror2 to zmirrorp in configuration.
The "p" stands for "personals", aka zephyr private messages, which is
what this host manages.
2017-10-05 20:06:08 -07:00
Tim Abbott 062d280914 puppet: Clean up unnecessary pagerduty_nagios.cfg. 2017-10-05 19:23:33 -07:00
Tim Abbott 7e328ba865 nagios: Move email addresses for contacts into variables. 2017-10-05 19:23:33 -07:00
Tim Abbott 6017d3dec5 puppet: Move contacts.cfg to be a template. 2017-10-05 19:23:33 -07:00
Tim Abbott 09aec3e467 puppet: Move hosts.cfg to be managed by a template. 2017-10-05 19:23:33 -07:00
Tim Abbott 692f4b77d1 puppet: Remove messy Nagios crontab. 2017-10-05 19:23:33 -07:00
Tim Abbott 26982ff55f puppet: Remove pageduty_nagios.pl.
This hasn't been used in like 4 years, and clutters the repo.
2017-10-05 18:46:09 -07:00
Tim Abbott 5a80c029a2 nagios: Update path to sync_public_streams to match new config. 2017-10-05 13:34:27 -07:00
Tim Abbott fdd021fd6a zephyr-mirror: Update supervisor configuration for repository split.
This now points to the path of the integration in the new package.
2017-10-05 13:18:37 -07:00
Tim Abbott 1eff717146 zephyr-mirror: Update cron job to use python-zulip-api.
This is a deferred follow-up project to the repository split.
2017-10-05 13:07:45 -07:00
Greg Price d02101a401 APNs: Rip out the existing, broken implementation.
This code empirically doesn't work.  It's not entirely clear why, even
having done quite a bit of debugging; partly because the code is quite
convoluted, and because it shows the symptoms of people making changes
over time without really understanding how it was supposed to work.

Moreover, this code targets an old version of the APNs provider API.
Apple deprecated that in 2015, in favor of a shiny new one which uses
HTTP/2 to meet the same needs for concurrency and scale that the old
one had to do a bunch of ad-hoc protocol design for.

So, rip this code out.  We'll build a pathway to the new API from
scratch; it's not that complicated.
2017-08-26 14:16:05 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Greg Price e4d1d22e9f py3: Explicitly keep our wal-e PostgreSQL replication on Python 2.
On `trusty` there is no package for `boto` or `gevent` on Python 3, both
of which are dependencies of `wal-e` (at the version we've pinned.)  This
is something used only on database servers and only in a replication
scenario, and it doesn't involve any of our code outside the wal-e repo,
so the Python version it uses is quite independent of the Zulip
application server itself and the rest of our code.  For now, keep it
explicitly on Python 2 while we move forward for most everything else.
2017-08-15 17:30:31 -07:00