Commit Graph

123 Commits

Author SHA1 Message Date
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott b8e7369dee mypy: Remove type: ignores not needed in Python 3. 2017-08-25 11:04:20 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Greg Price 0debc3f7ea install: Update a comment for our post-/root/zulip world.
We may not necessarily be running out of /root/zulip or any particular path,
but the point this comment was really trying to make in the first place stands.
Make it more clearly and still-accurately.
2017-08-15 17:41:07 -07:00
Pweaver (Paul Weaver) 1afaa67c7c deps: Change npm to yarn for reliablity, security, and speed. 2017-08-05 12:29:06 -07:00
Tim Abbott 53e4d8562b lint: Add a lint check for bare `type: ignore`s. 2017-07-27 16:31:55 -07:00
rht 614e3bb1ef scripts: Add script to autogenerate a self-signed SSL cert.
This will simplify step 1 of prod-install instruction to reduce
suffering in testing/experimenting production environments.

Attribution: the scripts/setup/configure-certs is based on @galexrt's
5c0daf6211

Further tweaked by tabbott to rename the script and edit the messages.
2017-07-16 17:22:32 -07:00
Tim Abbott c1d2654f3a setup-apt-repo: Add pgroonga code for Debian stretch.
The Groonga apt repository for Debian has a slightly different
structure.
2017-07-14 17:22:20 -07:00
EdOverflow 86cfed3e81 Update generate_secrets.py 2017-07-11 12:43:46 -07:00
sinwar 9cab965601 install-node: hardcode the path for npm.
This replaces nvm in npm-wrapper by harcoding the path the way we do
with node.  The main benefit is that this saves a few hundred
milliseconds every time we invoke npm.
2017-07-06 17:44:28 -07:00
Tim Abbott b70986469c generate_secrets: Fix handling of missing trailing newlines.
When we added support for automatically adding new secrets in
generate_secrets.py, we failed to account for the possibility that a
human editor might have let the secrets file without a trailing
newline.

We address this by adding a leading newline before our new secret.

Fixes #5209.
2017-06-03 23:17:04 -07:00
Tim Abbott 11adbf5783 generate_secrets: Fix placement of mypy type: ignore. 2017-05-17 00:05:57 -07:00
Tim Abbott b01ba5f389 generate_secrets: Fix mypy errors.
I'm pretty sure these errors reflect a problem with Typeshed, but
don't have time to investigate.
2017-05-16 23:28:44 -07:00
Tim Abbott 2c6a91e24a scripts: Make generate_secrets.py idempotent.
Now, generate_secrets.py will never overwrite existing secrets.  In
addition to being a safer model in generate, this fixes 2 significant
issues:

(1) It makes it much easier to preserve secrets like Oauth tokens in a
development environment (previously, provision would destroy them).
(2) It makes it possible to automatically add new secrets as part of
the upgrade process.  In particular, this is useful for the
zulip_org_id settings.

Fixes #4797.
2017-05-16 22:15:25 -07:00
Tim Abbott 03b5200d8b generate_secrets: Reformat list of autogenerated secrets. 2017-05-16 22:15:25 -07:00
rht 00e057bf44 install-node: bypass nvm wrapper for faster node startup.
This fixes a significant performance issue with LaTeX rendering (and
other things that invoked node) where starting up node took a few
hundred milliseconds due to nvm initialization.

Tweaked by tabbott to avoid copying the node binary itself, instead
using a tiny wrapper script.

This is important primarily because it's possible a future version of
node will expect to find libraries/dependencies/etc. installed via NVM
at some path related to the path of the node binary itself, and that's
more guaranteed with this new model.

Fixes #4618.
2017-05-09 09:17:54 -07:00
Alexander Trost 889547ff5e configure-rabbitmq: Add support for RABBITMQ_NODE flag.
This can potentially be used by things like a Docker configuration
that runs RabbitMQ on another server.
2017-04-29 15:03:05 -07:00
Tim Abbott 55a9101573 settings: Add support for ZULIP_ORG secrets.
These can be used to authenticate the current Zulip server to
zulip.org.
2017-04-18 23:00:10 -07:00
Feorlen 10ccfcdc8e Set umask 022 before starting prod install.
Fixes #2372.
2017-03-25 23:59:44 -07:00
Rishi Gupta 2bbfdeeb7b Fix more errors caught by mypy 0.501.
Another set of relatively easy to review changes.
2017-03-03 14:15:38 -08:00
Tim Abbott 51d3ab1cb7 initialize-database: Clean up final instructions.
Fixes #3678.
2017-02-21 20:19:16 -08:00
Tim Abbott de99f48ce7 lint: Clean up E401 PEP-8 rule. 2017-01-23 21:36:39 -08:00
anirudhjain75 beaa62cafa mypy: Convert several directories to use typing.Text.
Specifically, these directories are converted: [analytics/, scripts/,
tools/, zerver/management/, zilencer/, zproject/]
2016-12-07 20:51:05 -08:00
Tommy Ip 46b7d54b3e pep8: Fix E701 violations. 2016-11-30 20:45:09 +00:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Tim Abbott 8821b269bf flush-memcached: Replace 'set -x' with more clear output.
Similar to `terminate-psql-sessions`, this was just unnecessary spam
output.
2016-10-14 17:10:06 -07:00
Tim Abbott 382c8853f3 terminate-psql-sessions: Remove set -x.
Now that we're no longer actively debugging this tool, there's no need
to have it print everything it's doing.

This will make `test-backend` a lot nicer to use.
2016-10-14 17:08:05 -07:00
Tim Abbott 14f6e4c740 scripts: Stop using apt-add-repository.
Unfortunately, apt-add-repository is highly unreliable and was causing
problems both in Travis CI and with developers provisioning their
environment.
2016-10-11 22:10:36 -07:00
Diptanshu8 d7253b144c generate-secrets: Refactor to make development/production explicit.
generate-secrets.py now requires --development for development environment
setup or --production for production environment setup (and one of these
options is mandatory).

This solves the problem that it was somewhat easy to accidentally run
generate-secrets.py without the `-d` option while doing manual development
environment setup.

Fixes: #1911.
2016-10-06 17:12:49 -07:00
Tim Abbott c819a66e09 postgres-init-db: Use default bash to call flush-memcached. 2016-09-28 20:46:34 -07:00
umkay 798e6faa9e provision: Use NVM to install node and npm.
NVM takes a specific node version and installs the node package and
a corresponding compatible npm package.

We use it in a somewhat hackish way to install node/npm globally with
a pinned version, since that's how we actually want to consume node in
our development environment.

Other details:
- Travis CI now is configured to use the version of node installed by
provision; the easiest way to do this was to sabotage the existing node
installation.
- jsdom is upgraded to a current version, which both requires recent
node and also is required for the tests to pass with recent node.
This fixes running the node tests on Xenial.

Fixes #1498.

[tweaked by tabbott]
2016-09-23 14:34:44 -07:00
Vishnu Ks a7ead9e99d settings: Eliminate ADMIN_DOMAIN for creating initial realm.
We now use `./manage.py generate_realm_creation_link` as the flow flow
for creating one's first realm.
2016-08-25 09:37:33 -07:00
Alexander Trost 8d4896809a Deduplication of configuration: Database Initialization
Comment tweaked slightly by tabbott.
2016-08-05 13:33:22 -07:00
Tim Abbott 3839a25c74 Annotate generate_secrets.py. 2016-08-04 15:53:23 -07:00
Tim Abbott 3b277c3b1f install: Give nice error message on installation failure. 2016-07-31 19:24:45 -07:00
Tim Abbott 8b285ec0ff puppet: Read camo key from zulip-secrets.conf. 2016-07-31 00:23:24 -07:00
Tim Abbott 35339f5117 Rename local_settings_template to prod_settings_template. 2016-07-19 20:59:59 -07:00
Sumana Harihareswara 07afc9d34b Move production health check doc to separate page. 2016-07-12 15:46:10 -07:00
Eklavya Sharma a9835c0ab2 Activate virtualenv in production Python code.
The manage.py change effectively switches the Zulip production server
to use the virtualenv, since all of our supervisord commands for the
various Python services go through manage.py.

Additionally, this migrates the production scripts and Nagios plugins
to use the virtualenv as well.
2016-06-27 19:55:35 -07:00
Tim Abbott f7ce5fc179 generate_secrets: Silence mypy error with configparser.
See https://github.com/python/typeshed/issues/307.
2016-06-20 09:03:09 -07:00
Umair Khan a9a6687b7d Make generate-secrets script use existing values.
Fixes #1035
2016-06-20 17:00:27 +05:00
Michael Cordover b401ec0af7 Warn on postgres-init-db if >200 messages exist.
Closes #548.
2016-06-13 15:57:51 -07:00
acrefoot cf15b0b4e6 Fix hanging nc and 'invalid wait time' error on Trusty.
Apparently, 0 isn't a supported wait time value in some versions of nc.
2016-06-02 18:02:13 -07:00
Eklavya Sharma 94e4b39112 Replace python2.7 by python everywhere. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Tim Abbott 6e1872987d Move bin/get-django-setting to scripts/. 2016-05-07 19:37:06 -07:00
Tim Abbott 6e1e4aaef6 postgres-init-db: Add POSTGRES_USER argument. 2016-04-26 15:27:35 -07:00
Tim Abbott dc772518e7 Don't chown supervisor socket if it doesn't exist. 2016-04-26 15:27:35 -07:00
Luke Faraone 9d9bfb27ef Correct shell quoting around $DEFAULT_USER in terminate-psql-sessions
Previously, we used shell quoting that would result in the shell variable not
being substituted. Instead, we use `"`s that will allow for variable
substitution.
2016-02-19 02:09:50 +00:00
Vladislav Manchev dfbea01c8f Add support for running OpenBSD in development environment. 2016-01-21 22:33:55 -08:00