Commit Graph

695 Commits

Author SHA1 Message Date
Tim Abbott dc0696af74 puppet: Ensure zulip user owns key /etc/zulip files.
The main purpose of this change is to make it guaranteed that
`manage.py register_server --rotate-key` can edit the
/etc/zulip/zulip-secrets.conf configuration via crudini.

But it also adds value by ensuring zulip-secrets.conf is not readable
by other users.
2018-05-06 21:54:02 -07:00
Tim Abbott 3ac7f01e4b puppet: Replace dockervoyager supervisord hack with a better one.
This hack is still pretty bad, but at least it still works on puppet 4.
2018-05-06 21:28:03 -07:00
Tim Abbott eab1d1d9e7 puppet: Fix puppet-apt bug with modern apt-key.
With modern apt-key, the fingerprints are displayed in the more fully
written-out format with spaces, and so `apt-key add` was being run
every time.

This fixes some unnecessary work being done on each puppet run on
Debian stretch.

I would have preferred to not need to do this by upgrading to
upstream, but see #7423 for notes on why that isn't going to work
(basically they broke support for puppet older than 4).
2018-05-06 21:10:24 -07:00
Tim Abbott 7fe19ef8e7 puppet: Remove </% example characters from sample postgres config.
Apparently, these confused the puppet template parser, since they are
somewhat similar to its syntax, resulting in errors trying to use
these templates.  It's easy enough to just remove the example
content from the base postgres config file.
2018-05-06 19:50:38 -07:00
Tim Abbott 63fe39e381 zulip_ops: Disable Ubuntu's built-in update-motd.d files.
We can't really do this in the zulip manifests (since it's sorta a
sysadmin policy decision), but these scripts can cause significant
load when Nagios logs into a server (because many of them take 50ms or
more of work to run).  So we just get rid of them.
2018-05-06 18:47:40 -07:00
Tim Abbott 427b404b9b puppet: Fix detection of Debian/Ubuntu version.
The previous configuration, missing a ^, had the 8.x for Debian Jessie
match Ubuntu 18.04 by accident.
2018-05-05 11:52:45 -07:00
Tim Abbott cf90b9cec0 puppet: Make memory computations work with Puppet 4.
The actual approach for achieving this goal is to take our manual
parsing and move it to the central base.pp.
2018-05-05 11:25:48 -07:00
Tim Abbott 0c1a0a35ec static_asset_compiler: Assume non-trusty is newer.
It seems unlikely we're going to add support for additional older
Debian-based distributions, so it makes sense to just use an else
statement.  This should save a bit of busywork every time we add a new
distro.
2018-05-05 10:50:17 -07:00
Tim Abbott 8ea8bfe285 puppet: Add basic configuration for Ubuntu bionic. 2018-05-05 10:49:09 -07:00
Tim Abbott a03e4784c7 puppet: Add Zulip specific postgres configuration for 10.
Mostly, this involves adding the big block at the bottom and making
10 a variable so that it's easier to compare different versions of
these.

I did an audit of the configuration changes between 9.6 and 10, so
this should be fine, but it hasn't been tested yet.
2018-05-05 10:48:46 -07:00
Tim Abbott 964a1ac8a7 puppet: Commit an upstream version of postgres 10 config. 2018-05-05 10:48:37 -07:00
Tim Abbott 4ee762a52c apt: Add packagecloud repository for Debian. 2018-05-05 10:03:03 -07:00
Tim Abbott 06cfc591fe setup-apt-repo: Require apt-transport-https be installed.
Doing our apt operations over HTTPS has better security properties.
2018-05-05 10:02:50 -07:00
Tim Abbott 4e8487c886 nagios: Bump maximum processes limits.
These seemed to be flapping for no good reason.
2018-05-02 11:12:47 -07:00
Tim Abbott 2f937d81e2 puppet: Add Zulip specific postgres configuration for 9.6.
Mostly, this involves adding the big block at the bottom and making
9.6 a variable so that it's easier to compare different versions of
these.
2018-04-23 18:33:49 -07:00
Tim Abbott 9930e3de09 puppet: Add a stock Postgres 9.6 configuration file from Debian.
This will make it easier to see what we customize.
2018-04-23 18:29:02 -07:00
Tim Abbott 718492638b puppet: Fix name for dhcpcd5 package.
Apparently the name dhcpcd isn't installable.
2018-04-23 11:32:07 -07:00
Tim Abbott 56b0479656 puppet: Clean up indentation in various manifests.
These are inspired by puppet-lint, though we didn't take all of their
changes, since some seem to be bugs in the tool.
2018-04-23 00:15:54 -07:00
Tim Abbott b493748ddb puppet: Use single quotes where valid.
This brings our puppet codebase more in line with the standard puppet
style guide.  Changes done via `puppet-lint --fix`.
2018-04-23 00:15:54 -07:00
Tim Abbott 871078db30 puppet: Fix alignment of arrow operators.
This was done using puppet-lint --fix.
2018-04-23 00:15:54 -07:00
Tim Abbott 19cee30bf8 puppet: Fix use of under-scoped variables. 2018-04-22 23:53:34 -07:00
Tim Abbott 6e55aa2ce6 puppet: Fix mispelled variable name.
Apparently, we weren't uninstalling the old WSGI module properly.
2018-04-22 23:53:34 -07:00
Tim Abbott 6988f13201 puppet: Move safepackage definitions out of class definitions.
Also, deduplicate it while we're at it.

This fixes a puppet-lint issue that becomes an error with puppet 4.
2018-04-22 23:53:22 -07:00
Tim Abbott a6aa7042a2 puppet: Fix some unnecessarily quoted strings.
Flagged by puppet-lint.
2018-04-22 23:42:35 -07:00
Tim Abbott 35aa4f0377 puppet: Sort ensure attributes to be always first.
This inconsistency was flagged by puppet-lint.
2018-04-22 23:41:49 -07:00
Tim Abbott a56968ce68 puppet: Fix variables not clearly enclosed.
This improves readability and robustness.  Found and fixed via puppet-lint.
2018-04-22 23:35:33 -07:00
Tim Abbott 169ee5d8a1 puppet: Fix use of tab-based whitespace. 2018-04-22 23:34:30 -07:00
Tim Abbott e103c2ff2d puppet: Switch to modern quoted, octal file modes.
This is one of the prerequisite tasks for Puppet 4 support.

Constructed using puppet-lint.
2018-04-22 23:30:48 -07:00
Tim Abbott a06c7bc247 puppet: Allow manual configuration of postfix_mailname.
This allows users to configure a mailname for postfix in
/etc/zulip/zulip.conf
2018-04-19 14:41:05 -07:00
Tim Abbott 62b12e0c34 zulip_ops: Add missing dependency on dhcpcd. 2018-04-19 14:27:48 -07:00
Aditya Bansal 4898fe7ebc uploads: Change Content-Security-Policy to fix issue with pdf's.
Our recent addition of Content-Security-Policy to the file uploads
backend broke in-browser previews of PDFs.

The content-types change in the last commit fixed loading PDFs for
most users; but the result was ugly, because e.g. Chrome would put the
PDF previewer into a frame (so there were 2 left scrollbars).

There were two changes needed to fix this:
* Loading the style to use the plugin.  We corrected this by adding
  `style-src 'self' 'unsafe-inline';`
* Loading the plugin.  Our CSP blocked loading the PDf viewer plugin.
  To correct this, we add object-src 'self', and then limit the
  plugin-type to just the one for application/pdf.

We verified this new CSP using https://csp-evaluator.withgoogle.com/
in addition to manual testing.
2018-04-17 12:23:24 -07:00
Tim Abbott 568a12e254 nginx: Add PDF files to the content-types list.
Previously, user-uploaded PDF files were not properly rendered by
browsers with the local uploads backend, because we weren't setting
the correct content-type.
2018-04-17 11:50:10 -07:00
Tim Abbott a463743107 puppet: Add Content-Security-Policy for user avatars.
This adds a basic Content-Security-Policy for user-uploaded avatars
served by the LOCAL_UPLOADS backend.

I think this is for now an unnecessary follow-up to
d608a9d315, but is worth doing because
we may later change what can be uploaded in the avatars directory.
2018-04-10 14:43:08 -07:00
Aditya Bansal d608a9d315 uploads: Add Content-Security-Policy for user uploads.
This adds a basic Content-Security-Policy for user-uploaded files with local uploads.

While over time, we plan to add CSP for the main site as well, this CSP is particularly
important for the local-uploads backend, which often shares a domain with the main site.
2018-04-09 14:43:02 -07:00
Tim Abbott 0d35bbc464 install: Install the wget package.
We depend on it for installing node, and it's a standard package, not
a required one, so we do need to explicitly declare the dependency.
2018-03-29 16:03:44 -07:00
Ghislain Antony Vaillant 00dd86967b puppet: Add Debian 9 package names to definitions.
This doesn't add support for Debian 9, but it will save time for folks
working on Stretch support in the future.
2018-03-28 12:33:45 -07:00
neiljp (Neil Pilgrim) 090b47ed19 mypy: Add explicit Optional for default=None parameters in various files. 2018-03-28 12:31:51 -07:00
neiljp (Neil Pilgrim) f32f3cbf72 mypy: Amend zulip-ec2-configure-interfaces to avoid None. 2018-03-23 11:39:54 -07:00
Tim Abbott d98be2f19f puppet: Only run analytics Nagios checks on machine running cron.
Running this on additional machines would be redundant; additionally,
the FillState checker cron job runs only on cron systems, so this will
crash on other app frontends.
2018-03-06 13:38:27 -08:00
Tim Abbott 8e8faab006 puppet: Move clearsessions cron job to app_frontend_once.
While this is a different system than I'd written up in #8004, I think
this is a better solution to the general problem of cron jobs to run
on just one server.

Fixes #8004.
2018-03-06 13:35:51 -08:00
Tim Abbott 9a74ef5056 puppet: Move some cron jobs to app_frontend_once.pp.
Several cron jobs had incorrectly ended up in the app_frontend.pp
template, and thus would only run on voyager instances.
2018-03-06 13:35:51 -08:00
Tim Abbott 3ae645ed12 puppet: Rename analytics.pp to app_frontend_once.pp. 2018-03-06 13:35:51 -08:00
Tim Abbott ad7f38ab3b puppet: Move analytics cron job to analytics.pp.
This better groups it with the related code.
2018-03-06 13:35:51 -08:00
Tim Abbott 24b6106c9c puppet: Dsiable checking for evictions in memcached nagios.
Zulip's caching model for message history is such that it is normal
and healthy for there to eventually be a nontrivial volume of
evictions.
2018-03-06 13:34:02 -08:00
Greg Price 4475950ddf queue: Restore prematurely-cut upgrade path.
Revert c8f034e9a "queue: Remove missedmessage_email_senders code."
As the comment in the code says, it ensures a smooth upgrade path
from 1.7.x; we can delete it in master after 1.8.0 is released.
The removal commit was merged early due to a communication failure.
2018-02-28 11:15:53 -08:00
Tim Abbott 65767e5226 localhost_sso: Fix missing enabling of mod_wsgi.
This is apparently required on Ubuntu Xenial, at least.
2018-02-22 10:09:29 -08:00
Umair Khan c8f034e9a0 queue: Remove missedmessage_email_senders code.
After 68513952fb, all emails are sent through email_senders queue.
This commit removes code related to the legacy queue.
2018-02-21 16:43:56 -08:00
Aditya Bansal efe8545303 local-uploads: Start running authentication checks on file requests.
From here on we start to authenticate uploaded file request before
serving this files in production. This involves allowing NGINX to
pass on these file requests to Django for authentication and then
serve these files by making use on internal redirect requests having
x-accel-redirect field. The redirection on requests and loading
of x-accel-redirect param is handled by django-sendfile.

NOTE: This commit starts to authenticate these requests for Zulip
servers running platforms either Ubuntu Xenial (16.04) or above.

Fixes: #320 and #291 partially.
2018-02-16 05:06:37 +05:30
Greg Price 20c734c90a puppet: Fix type error in new Nagios check for analytics state. 2018-02-09 17:46:46 -08:00
Tim Abbott 005b0fb566 puppet: Clean up ssh authorized_keys configuration rules. 2018-02-09 16:37:03 -08:00