Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
12,771 Commits 28 Branches 143 Tags 632 MiB
Commit Graph

434 Commits

Author SHA1 Message Date
Tim Abbott 922a4acdc5 puppet: Add zulip.conf setting for listen_addresses tuning. 2016-07-30 21:23:12 -07:00
Tim Abbott b2e32e6e3f puppet: Add postgres replication option for postgres_appdb_tuned.pp. 
This adds a setting that controls whether to include the Zulip default
streaming replication configuration in the postgres configuration file.
 2016-07-30 21:23:12 -07:00
Tim Abbott 3ab0295061 puppet: Add zulip.conf setting for effective_io_concurrency tuning. 2016-07-30 21:23:12 -07:00
Tim Abbott f3db368a3c puppet: Add zulip.conf setting for random_page_cost tuning. 2016-07-30 21:23:12 -07:00
Tim Abbott 63c757eac3 puppet: add zulipconf function for reading settings from zulip.conf. 
This makes it convenient for us to have optional user-defined settings
in the main Zulip puppet configuration.
 2016-07-30 21:23:12 -07:00
Eklavya Sharma 3b3b5c7c16 Rename management command email-mirror to email_mirror. 
All other zulip management command names have underscores, so
rename email-mirror to email_mirror.

This will also make it possible to import this module, which will
help in writing tests for it.
 2016-07-28 14:52:09 -07:00
Tim Abbott 69528790a5 check_worker_memory: Fix handling of no queue workers running. 2016-07-28 13:26:31 -07:00
Eklavya Sharma 408d070170 puppet/: Make subprocess calls unicode-aware. 2016-07-26 12:06:41 -07:00
Eklavya Sharma 3d87c376c2 Make zulip-ec2-configure-interfaces pass mypy check. 2016-07-24 12:52:58 +05:30
Eklavya Sharma 0a5aa2ccc7 Make check_send_receive_time pass mypy check. 2016-07-24 12:41:06 +05:30
Tim Abbott 2a209e46dd puppet: Fix remaining hardcoding of postgres 9.1 in manifests. 2016-07-20 21:11:22 -07:00
Tim Abbott d529bc12ef pg_backup_and_purge: Fix hardcoding of postgres version. 2016-07-19 19:19:42 -07:00
Tim Abbott bdb1ce04a2 puppet: Move pg_backup_and_purge to main config. 2016-07-19 19:19:42 -07:00
Tim Abbott 81136ff092 env-wal-e: Eliminate hardcoding of AWS keys. 
Pre-Zulip being open sourced, this file just had the AWS keys for
backups hardcoded.

Instead, these are simply read from zulip-secrets.conf.
 2016-07-19 19:19:42 -07:00
Tim Abbott 3023745ed7 puppet: Move env-wal-e to Zulip main distribution. 
This doesn't yet include the dependencies for env-wal-e.
 2016-07-19 19:19:42 -07:00
Tim Abbott d0dcc8bf26 puppet: Create static_asset_compiler manifest. 2016-07-19 16:38:09 -07:00
Tim Abbott 601ab24d2a zulip_internal::app_frontend: Remove now-unnecessary dependencies. 
These Python packages are no longer relevant, since we install all of
our Python dependencies via virtualenvs.
 2016-07-19 16:37:50 -07:00
Tim Abbott 77ccc0d87f Patch httpoxy security issue. 2016-07-18 10:24:03 -07:00
Eklavya Sharma da36947400 Change unbuffering strategy in runtornado.py. 
runtornado unbuffers its output using
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0).
This is not python 3 compatible since we can't specify
buffering on a text stream in python 3.  So use the '-u'
option of python when calling runtornado.py to make output
unbuffered.
 2016-07-17 10:31:15 -07:00
Tim Abbott ace8ae8301 puppet: Fix supervisor restart rule catching itself in pgrep. 2016-07-12 10:39:33 -07:00
Tim Abbott dfc9e75342 nginx: Enable shared SSL session cache. 2016-07-10 08:30:10 -07:00
Alex Gaynor ee47da8790 nginx: Disable RC4 and prefer server cipher order. 2016-07-10 08:30:05 -07:00
Tim Abbott ea089518ee Add Zulip customizations to Postgres 9.5 config file. 
This includes reasonable tuning of memory usage parameters based on
the values that pgtune would use, roughly.
 2016-07-08 16:16:12 -07:00
Tim Abbott e818dff0b0 postgres: Remove comment matching Puppet ERB syntax. 2016-07-08 16:16:12 -07:00
Tim Abbott 6ba659aeec Add a stock Ubuntu Xenial Postgres 9.5 configuration file. 2016-07-08 16:16:12 -07:00
Tim Abbott 0f5e62e994 puppet: Don't use pgtune on Linux versions where it doesn't exist. 2016-07-08 16:16:12 -07:00
Tim Abbott 903f728587 Scope postgresql.conf templates by postgres version. 2016-07-08 16:16:12 -07:00
Tim Abbott ffe79e0d50 Fix EPMD restart being attempted on every puppet apply. 2016-07-08 16:16:11 -07:00
Taranjeet a137bf15ed Wrap some lines with length greater than 120. 
With some tweaks by tabbott.
 2016-07-06 14:35:16 -07:00
Tim Abbott 809a45394f puppet: Start supervisord if it isn't running on restart. 2016-06-28 22:09:36 -07:00
Eklavya Sharma 7a98d7bd24 manifests/base.pp: Remove apt python package dependencies. 2016-06-27 19:55:35 -07:00
Eklavya Sharma ebad1415c0 app_frontend_base.pp: Remove apt python package dependencies. 2016-06-27 19:55:35 -07:00
Eklavya Sharma a9835c0ab2 Activate virtualenv in production Python code. 
The manage.py change effectively switches the Zulip production server
to use the virtualenv, since all of our supervisord commands for the
various Python services go through manage.py.

Additionally, this migrates the production scripts and Nagios plugins
to use the virtualenv as well.
 2016-06-27 19:55:35 -07:00
Tim Abbott 46c2416cc8 puppet: Add postgres version map for xenial. 
This is an early step towards Ubuntu Xenial support in production.
 2016-06-17 16:41:42 -07:00
Eklavya Sharma 94e4b39112 Replace python2.7 by python everywhere. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Eklavya Sharma b6bd5445bc End puppet/.../process_fts_updates gracefully. 
Catch KeyboardInterrupt in puppet/zulip/files/postgresql/process_fts_updates,
print a message and exit.
 2016-05-24 13:28:05 -07:00
Eklavya Sharma 3441f0848c Annotate pg_backup_and_purge.py. 2016-05-18 17:10:17 +05:30
Umair Khan 5359e6b0d4 Convert Zulip to use Jinja2 templates. 
This results in a substantial performance improvement for all of
Zulip's backend templates.

Changes in templates:
- Change `block.super` to `super()`.
- Remove `load` tag because Jinja2 doesn't support it.
- Use `minified_js()|safe` instead of `{% minified_js %}`.
- Use `compressed_css()|safe` instead of `{% compressed_css %}`.
- `forloop.first` -> `loop.first`.
- Use `{{ csrf_input }}` instead of `{% csrf_token %}`.
- Use `{# ... #}` instead of `{% comment %}`.
- Use `url()` instead of `{% url %}`.
- Use `_()` instead of `{% trans %}` because in Jinja `trans` is a block tag.
- Use `{% trans %}` instead of `{% blocktrans %}`.
- Use `{% raw %}` instead of `{% verbatim %}`.

Changes in tools:
- Check for `trans` block in `check-templates` instead of `blocktrans`

Changes in backend:
- Create custom `render_to_response` function which takes `request` objects
  instead of `RequestContext` object. There are two reasons to do this:
    1. `RequestContext` is not compatible with Jinja2
    2. `RequestContext` in `render_to_response` is deprecated.
- Add Jinja2 related support files in zproject/jinja2 directory. It
  includes a custom backend and a template renderer, compressors for js
  and css and Jinja2 environment handler.
- Enable `slugify` and `pluralize` filters in Jinja2 environment.

Fixes #620.
 2016-05-09 09:55:18 -07:00
Tim Abbott 00ccf147cd check_nagios_send_time: Add --insecure option for use in tests. 2016-05-08 17:35:50 -07:00
Tim Abbott c0d38f42f1 supervisor: Use 127.0.0.1 rather than localhost for tornado. 
In theory these should be the same, but in misconfigured environments
(such at Travis CI) where /etc/hosts has multiple entries for
"localhost", 127.0.0.1 is safer than "localhost".
 2016-05-08 17:35:50 -07:00
Tim Abbott cb81a59e38 Move write-rabbitmq-consumers-state-file to scripts/nagios/. 2016-05-07 19:37:06 -07:00
Tim Abbott 2761c012e5 Move rabbitmq consumer checks from bots/ to scripts/nagios/. 2016-05-07 19:37:06 -07:00
Tim Abbott be6566dc5c nagios: Move cron_file_helper from bots/ to scripts/lib. 
This ensures the tool is available in Zulip production deployments.
 2016-05-07 19:37:06 -07:00
Tim Abbott 52c1e8ac7d Run a local camo server in voyager production environments. 
Camo is a caching image proxy, used in Zulip to avoid mixed-content
warnings by proxying HTTP image content over HTTPS.  We've been using
it in zulip.com production for years; this change makes it available
in standalone Zulip deployments.
 2016-05-02 17:21:31 -07:00
Tim Abbott 5195d1ecb7 Fix missing newlines at ends of JSON files. 2016-04-14 10:48:52 -07:00
Tim Abbott caba24b2af Fix existing trailing whitespace in bash scripts. 2016-04-08 11:52:11 -07:00
Tim Abbott a7f83c9e05 Fix check_postgres_replication_lag nagios command. 2016-04-06 15:30:51 -07:00
Tim Abbott 991341867c nagios: Remove unnecessary dependency on netcat. 2016-04-06 15:30:40 -07:00
Tim Abbott c92221dcd3 Remove old humbughq apache configuration. 2016-04-06 15:30:36 -07:00
Tim Abbott 4855296771 puppet: Migrate check_postgres plugins to postgres_common.pp. 2016-04-06 15:20:36 -07:00
Tim Abbott 69a8925076 check_user_zephyr_mirror_liveness: Fix importing settings. 2016-04-05 13:27:04 -07:00
Tim Abbott 55172e2e0c Remove old zulip_internal nagios_plugins installation. 2016-04-05 13:27:04 -07:00
Tim Abbott 934e8641ee Migrate Zephyr mirror Nagios plugins to subdirectory. 2016-04-05 13:27:04 -07:00
Tim Abbott 7b753e5882 Migrate check_debian_packages to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott 2da9fc56d6 Migrate check_pg_replication_lag to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott c2e210ca0d Migrate check_website_response.sh to new zulip::nagios.pp. 2016-04-05 13:27:04 -07:00
Tim Abbott eb72cecd9e Migrate check_fts_update_log to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott 92d696d007 Migrate check_postgres plugins to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott e155ecdc49 Migrate check_rabbitmq plugins to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott 3ed7d658f8 Migrate check_send_receive_time to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott ca45ec3f3f Migrate check_email_deliverer plugins to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott 4e10424512 Migrate check_worker_memory to zulip/. 2016-04-05 13:27:04 -07:00
Tim Abbott 59b46278be Move check_queue_worker_errors into subdirectory. 
This fixes an issue where this worker wasn't even being installed
properly in a way that sets us up for doing further reorganization of
the Zulip Nagios plugins.
 2016-04-05 13:27:04 -07:00
Tim Abbott 6f20c43097 Move dependency on nagios_plugins into base.pp. 2016-04-05 13:27:04 -07:00
Ryan Moore 2fe0700f55 Update memcache -> remote cache in inline documentation. 2016-03-31 12:56:42 -07:00
Tim Abbott 8c0b110e9a Add python-typing dependency. 2016-03-30 21:50:25 -07:00
Tim Abbott c16749d783 Add missing dependency on netcat in both dev and prod. 
Fixes #474.
 2016-03-29 21:30:48 -07:00
Tim Abbott 2adf6d822f puppet: Fix process_queue command lines to use the new argument style. 
cd2348e9ae broke installing Zulip in
production since it didn't correctly update the puppet configuration
to call the process_queue script using the new argument format.

This commit isn't ideal in that I'd prefer to not require updating
puppet in sync with the actual running code, but we don't have a great
mechanism for doing that.

Fixes #586.
 2016-03-27 23:17:16 -07:00
Zev Benjamin 965f923ac3 Remove postgres2 configuration 2016-03-23 20:41:42 -07:00
Zev Benjamin ae2560a027 Add postgres3 configuration 2016-03-23 20:41:25 -07:00
Tim Abbott 6137ae9902 Fix incorrect shell quoting in check_worker_memory. 2016-03-23 20:40:06 -07:00
Tim Abbott 210c2897e7 Fix check_worker_memory regular expression. 2016-03-23 20:40:02 -07:00
Josh Mandel b06739df11 Move email digest triggering to default zulip config. 
Previously, even though the Zulip digest emails were documented in the
settings, the cron job to run the script that actually sends the daily
digest emails wasn't included in the non-zulip.com part of the Zulip
production distribution.  The overall consequence is that digest
emails didn't work for non-zulip.com users.  This fixes that issue by
moving that cron job into the zulip manifests.

[commit message details expanded by tabbott]
 2016-03-19 10:34:41 -07:00
Eklavya Sharma 37f9520666 Make the remaining ambiguous divisions python 3 compatible. 
Refer to #256
 2016-03-12 10:53:51 -08:00
Eklavya Sharma b9e792c4e6 Apply Python 3 futurize transform libmodernize.fixes.fix_xrange_six 
Refer to #256
 2016-03-10 22:03:58 -08:00
Eklavya Sharma d3b63f9a2d Apply Python 3 futurize transform libmodernize.fixes.fix_file 
Refer to #256
 2016-03-10 22:02:34 -08:00
Eklavya Sharma c59185e119 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import 
Refer #256
 2016-03-10 22:02:17 -08:00
Eklavya Sharma e2d5ec1868 Apply Python 3 futurize transform lib2to3.fixes.fix_ws_comma 2016-03-10 22:02:04 -08:00
Eklavya Sharma 4fb549abe8 Apply Python 3 futurize transform lib2to3.fixes.fix_idioms 
Refer to #256
 2016-03-10 22:02:01 -08:00
Eklavya Sharma f3d387e727 Apply Python 3 futurize transform lib2to3.fixes.fix_except 2016-03-10 22:01:50 -08:00
Zev Benjamin 4887a79d21 Make the nginx log directory owned by zulip 
This is required to make log2zulip not error out when reading the nginx
error.log.
 2016-02-02 23:05:44 -08:00
Tim Abbott 757e89260e Migrate use of StringIO to Python 2+3 compatible six.moves.cStringIO. 
And add a check for additional usage of the original StringIO module.
 2016-01-26 21:09:43 -08:00
Tim Abbott 1c73c992dd Fix missing puppet dependencies on postgres package. 2016-01-26 20:32:33 -08:00
Tim Abbott 2e16b44b24 puppet: Use $postgres_version in postgres template. 
This eliminates hardcoding of the postgres version from the Zulip
puppet configuration.
 2016-01-26 20:32:33 -08:00
Tim Abbott 806aa986b7 puppet: Use a variable to configure the postgres version. 2016-01-26 20:32:33 -08:00
Tim Abbott a3ac56efe2 puppet: Make apt repository conditional on the Ubuntu version. 
We still will need to address this in the install script as well.
 2016-01-26 20:32:33 -08:00
Tim Abbott f6c59feb05 Document the puppet configuration somewhat in zulip::voyager. 2016-01-26 20:32:33 -08:00
Tim Abbott 345b5254d7 puppet: Move default nginx configuration out of voyager.pp. 2016-01-26 20:32:33 -08:00
Tim Abbott dd61e3f97d puppet: Move memcached and rabbitmq include out of app_frontend_base.pp. 2016-01-26 20:32:33 -08:00
Tim Abbott c3153274c1 puppet: Move memcached into its own puppet module. 2016-01-26 20:32:33 -08:00
Tim Abbott 8a0e07fe1a puppet: Rename app_frontend.pp to app_frontend_base.pp. 
This will enable us to move the remaining app-frontend related content
out of voyager.pp.
 2016-01-26 20:32:33 -08:00
Tim Abbott 91286d00aa puppet: Move Zulip apt repository to its own manifest. 2016-01-26 20:32:33 -08:00
Tim Abbott 69dd17dfb6 puppet: Move prod-static creation from voyager.pp to app_frontend.pp. 
Every app frontend will need this directory and this should help
enable more modular puppet rules.
 2016-01-26 20:32:33 -08:00
Tim Abbott 702f501638 puppet: Move tuned postgres configuration out of voyager.pp. 
This should make it easier for someone to run just the tuned Zulip
database on one server and the Zulip frontend on another.
 2016-01-26 20:32:28 -08:00
Tim Abbott d5f04bd20b Rename zulip::postgres_appdb to zulip::postgres_appdb_base. 
The purpose of this rename is to allow us to move the postgres-related
configuration out of voyager.pp.
 2016-01-26 20:30:12 -08:00
Tim Abbott 3f27573cb2 puppet: Move several debugging tool dependencies out of base.pp. 2016-01-26 20:30:11 -08:00
Tim Abbott 2b0394d807 Add documentation explaining what process_fts_updates does. 2016-01-11 08:56:03 -08:00
Tim Abbott 0162dc4bc0 process_fts_updates: Cleanup and document new settings import logic. 2016-01-11 08:56:03 -08:00
Javier Ros a6a47aacde Add easy support for using a remote postgres database. 2016-01-11 08:56:03 -08:00