Commit Graph

711 Commits

Author SHA1 Message Date
Tim Abbott afdfdf775c nginx: Set X-Frame-Options header to DENY.
While there are legitimate use cases for embedded Zulip in an iFrame,
they're rare, and it's more important to prevent this category of
attack by default.

Sysadmins can switch this to a whitelist when they want to use frames.
2018-05-30 09:24:17 -07:00
Sampriti Panda 250015a5d5 pgroonga: Fix issues with HTML escaping in queries. 2018-05-28 16:53:30 -07:00
Vishnu Ks 54a002c2e2 requirements: Upgrade pyflakes to 2.0.0.
We fix a few errors that only the new version finds.
2018-05-24 11:31:36 -07:00
Tim Abbott 42da4522a9 puppet-apt: Fix buggy access to caller_module_name.
New versions of Puppet on Ubuntu bionic don't like this.
2018-05-24 09:52:16 -07:00
Tim Abbott b83ba85100 puppet: Switch memcached to using common total_memory_mb value.
This just cuts a bit of unnecessary code duplication.
2018-05-24 09:49:43 -07:00
Tim Abbott 9b4b15cd0a static_asset_compiler: Remove dependency on node packages.
We no longer need or use these, since Zulip installs a pinned version
of node directly with the scripts/setup/install-node tool.

Noticed because in the effort of adding Ubuntu bionic support, we
noticed the package names changed again.
2018-05-24 09:43:45 -07:00
Tim Abbott c843276196 nginx: Fix accidental load-balancing between IPv4 and IPv6.
Apparently, our nginx configuration's use of "localhost", combined
with the default in modern Linux of having localhost resolve to both
the IPv4 and IPv6 addresses on a given machine, resulted in `nginx`
load-balancing requests to a given Zulip server between the IPv4 and
IPv6 addresses.  This, in turn, resulted in irrelevant 502 errors
problems every few minutes on the /events endpoints for some clients.

Disabling IPv6 on the server resolved the problem, as does simply
spelling localhost as 127.0.0.1 for the `nginx` upstreams that we
declare for proxying to non-Django services on localhost.
2018-05-22 11:56:59 -07:00
Tim Abbott 12dcabcdbd docker: Remove need for static_asset_compiler.
Now that the way we're installing from Git involving building a
release tarball with a 2-stage build, we no longer need to do this.
2018-05-20 13:15:21 -07:00
Tim Abbott 61d6965634 puppet: Add option for controlling file upload nginx config.
Now, one can just set `no_serve_uploads` in `zulip.conf` to prevent
`nginx` from serving locally uploaded files.

This should help simplify the S3 integration setup process.
2018-05-17 07:02:30 -07:00
Tim Abbott 6d74ba8271 puppet: Add zulip.conf option for HTTP only.
This option is intended to support situations like a quick Docker
setup where doing HTTPS adds more setup overhead than it's worth.
It's not intended to be used in actual production environments.
2018-05-17 06:58:35 -07:00
Jason Michalski 3d8e424d84 puppet: Add cron package dependency
The Zulip puppet installs various cron jobs and will fail if cron is not
installed. This was found when installing Zulip in a minimal docker
image.
2018-05-16 15:04:31 -07:00
Tim Abbott f2efa122a6 puppet: Include static_asset_compiler in dockervoyager.
This is required to build static assets from Git.
2018-05-15 18:27:01 -07:00
Tim Abbott 9498260516 puppet: Include process_fts_updates in dockervoyager manifest.
This is preferred, since we don't currently have a way to run Django
logic on the postgres hosts with the Docker implementation.

This is a necessary part of removing the need for the docker-zulip
package to patch this file to make Zulip work with Docker.
2018-05-15 15:37:12 -07:00
Tim Abbott ee3cd95bd1 puppet: Remove python 2 psycopg2 package.
We no longer need this, since we're a Python 3 project now.
2018-05-15 15:37:12 -07:00
Tim Abbott bd5e2ddc74 puppet: Extract zulip::process_fts_updates.
In theory, one might want to run this either on the postgres server or
on an application server.
2018-05-15 15:37:12 -07:00
Tim Abbott b656388ee2 check_send_receive_time: Fix mypy Optional issues.
These were not correctly interacting with the Zulip API, and improved
annotations in the API bindings are now actually being type-checked.
2018-05-15 09:06:56 -07:00
Tim Abbott dc0696af74 puppet: Ensure zulip user owns key /etc/zulip files.
The main purpose of this change is to make it guaranteed that
`manage.py register_server --rotate-key` can edit the
/etc/zulip/zulip-secrets.conf configuration via crudini.

But it also adds value by ensuring zulip-secrets.conf is not readable
by other users.
2018-05-06 21:54:02 -07:00
Tim Abbott 3ac7f01e4b puppet: Replace dockervoyager supervisord hack with a better one.
This hack is still pretty bad, but at least it still works on puppet 4.
2018-05-06 21:28:03 -07:00
Tim Abbott eab1d1d9e7 puppet: Fix puppet-apt bug with modern apt-key.
With modern apt-key, the fingerprints are displayed in the more fully
written-out format with spaces, and so `apt-key add` was being run
every time.

This fixes some unnecessary work being done on each puppet run on
Debian stretch.

I would have preferred to not need to do this by upgrading to
upstream, but see #7423 for notes on why that isn't going to work
(basically they broke support for puppet older than 4).
2018-05-06 21:10:24 -07:00
Tim Abbott 7fe19ef8e7 puppet: Remove </% example characters from sample postgres config.
Apparently, these confused the puppet template parser, since they are
somewhat similar to its syntax, resulting in errors trying to use
these templates.  It's easy enough to just remove the example
content from the base postgres config file.
2018-05-06 19:50:38 -07:00
Tim Abbott 63fe39e381 zulip_ops: Disable Ubuntu's built-in update-motd.d files.
We can't really do this in the zulip manifests (since it's sorta a
sysadmin policy decision), but these scripts can cause significant
load when Nagios logs into a server (because many of them take 50ms or
more of work to run).  So we just get rid of them.
2018-05-06 18:47:40 -07:00
Tim Abbott 427b404b9b puppet: Fix detection of Debian/Ubuntu version.
The previous configuration, missing a ^, had the 8.x for Debian Jessie
match Ubuntu 18.04 by accident.
2018-05-05 11:52:45 -07:00
Tim Abbott cf90b9cec0 puppet: Make memory computations work with Puppet 4.
The actual approach for achieving this goal is to take our manual
parsing and move it to the central base.pp.
2018-05-05 11:25:48 -07:00
Tim Abbott 0c1a0a35ec static_asset_compiler: Assume non-trusty is newer.
It seems unlikely we're going to add support for additional older
Debian-based distributions, so it makes sense to just use an else
statement.  This should save a bit of busywork every time we add a new
distro.
2018-05-05 10:50:17 -07:00
Tim Abbott 8ea8bfe285 puppet: Add basic configuration for Ubuntu bionic. 2018-05-05 10:49:09 -07:00
Tim Abbott a03e4784c7 puppet: Add Zulip specific postgres configuration for 10.
Mostly, this involves adding the big block at the bottom and making
10 a variable so that it's easier to compare different versions of
these.

I did an audit of the configuration changes between 9.6 and 10, so
this should be fine, but it hasn't been tested yet.
2018-05-05 10:48:46 -07:00
Tim Abbott 964a1ac8a7 puppet: Commit an upstream version of postgres 10 config. 2018-05-05 10:48:37 -07:00
Tim Abbott 4ee762a52c apt: Add packagecloud repository for Debian. 2018-05-05 10:03:03 -07:00
Tim Abbott 06cfc591fe setup-apt-repo: Require apt-transport-https be installed.
Doing our apt operations over HTTPS has better security properties.
2018-05-05 10:02:50 -07:00
Tim Abbott 4e8487c886 nagios: Bump maximum processes limits.
These seemed to be flapping for no good reason.
2018-05-02 11:12:47 -07:00
Tim Abbott 2f937d81e2 puppet: Add Zulip specific postgres configuration for 9.6.
Mostly, this involves adding the big block at the bottom and making
9.6 a variable so that it's easier to compare different versions of
these.
2018-04-23 18:33:49 -07:00
Tim Abbott 9930e3de09 puppet: Add a stock Postgres 9.6 configuration file from Debian.
This will make it easier to see what we customize.
2018-04-23 18:29:02 -07:00
Tim Abbott 718492638b puppet: Fix name for dhcpcd5 package.
Apparently the name dhcpcd isn't installable.
2018-04-23 11:32:07 -07:00
Tim Abbott 56b0479656 puppet: Clean up indentation in various manifests.
These are inspired by puppet-lint, though we didn't take all of their
changes, since some seem to be bugs in the tool.
2018-04-23 00:15:54 -07:00
Tim Abbott b493748ddb puppet: Use single quotes where valid.
This brings our puppet codebase more in line with the standard puppet
style guide.  Changes done via `puppet-lint --fix`.
2018-04-23 00:15:54 -07:00
Tim Abbott 871078db30 puppet: Fix alignment of arrow operators.
This was done using puppet-lint --fix.
2018-04-23 00:15:54 -07:00
Tim Abbott 19cee30bf8 puppet: Fix use of under-scoped variables. 2018-04-22 23:53:34 -07:00
Tim Abbott 6e55aa2ce6 puppet: Fix mispelled variable name.
Apparently, we weren't uninstalling the old WSGI module properly.
2018-04-22 23:53:34 -07:00
Tim Abbott 6988f13201 puppet: Move safepackage definitions out of class definitions.
Also, deduplicate it while we're at it.

This fixes a puppet-lint issue that becomes an error with puppet 4.
2018-04-22 23:53:22 -07:00
Tim Abbott a6aa7042a2 puppet: Fix some unnecessarily quoted strings.
Flagged by puppet-lint.
2018-04-22 23:42:35 -07:00
Tim Abbott 35aa4f0377 puppet: Sort ensure attributes to be always first.
This inconsistency was flagged by puppet-lint.
2018-04-22 23:41:49 -07:00
Tim Abbott a56968ce68 puppet: Fix variables not clearly enclosed.
This improves readability and robustness.  Found and fixed via puppet-lint.
2018-04-22 23:35:33 -07:00
Tim Abbott 169ee5d8a1 puppet: Fix use of tab-based whitespace. 2018-04-22 23:34:30 -07:00
Tim Abbott e103c2ff2d puppet: Switch to modern quoted, octal file modes.
This is one of the prerequisite tasks for Puppet 4 support.

Constructed using puppet-lint.
2018-04-22 23:30:48 -07:00
Tim Abbott a06c7bc247 puppet: Allow manual configuration of postfix_mailname.
This allows users to configure a mailname for postfix in
/etc/zulip/zulip.conf
2018-04-19 14:41:05 -07:00
Tim Abbott 62b12e0c34 zulip_ops: Add missing dependency on dhcpcd. 2018-04-19 14:27:48 -07:00
Aditya Bansal 4898fe7ebc uploads: Change Content-Security-Policy to fix issue with pdf's.
Our recent addition of Content-Security-Policy to the file uploads
backend broke in-browser previews of PDFs.

The content-types change in the last commit fixed loading PDFs for
most users; but the result was ugly, because e.g. Chrome would put the
PDF previewer into a frame (so there were 2 left scrollbars).

There were two changes needed to fix this:
* Loading the style to use the plugin.  We corrected this by adding
  `style-src 'self' 'unsafe-inline';`
* Loading the plugin.  Our CSP blocked loading the PDf viewer plugin.
  To correct this, we add object-src 'self', and then limit the
  plugin-type to just the one for application/pdf.

We verified this new CSP using https://csp-evaluator.withgoogle.com/
in addition to manual testing.
2018-04-17 12:23:24 -07:00
Tim Abbott 568a12e254 nginx: Add PDF files to the content-types list.
Previously, user-uploaded PDF files were not properly rendered by
browsers with the local uploads backend, because we weren't setting
the correct content-type.
2018-04-17 11:50:10 -07:00
Tim Abbott a463743107 puppet: Add Content-Security-Policy for user avatars.
This adds a basic Content-Security-Policy for user-uploaded avatars
served by the LOCAL_UPLOADS backend.

I think this is for now an unnecessary follow-up to
d608a9d315, but is worth doing because
we may later change what can be uploaded in the avatars directory.
2018-04-10 14:43:08 -07:00
Aditya Bansal d608a9d315 uploads: Add Content-Security-Policy for user uploads.
This adds a basic Content-Security-Policy for user-uploaded files with local uploads.

While over time, we plan to add CSP for the main site as well, this CSP is particularly
important for the local-uploads backend, which often shares a domain with the main site.
2018-04-09 14:43:02 -07:00