Commit Graph

10143 Commits

Author SHA1 Message Date
Tim Abbott 700055c194 Apply modernize transform libmodernize.fixes.fix_file.
This replaces use of file() with open() which is python 3 compatible,
and also adds it to our python 3 support test suite.
2016-01-26 21:09:42 -08:00
Tim Abbott 83dd51dcd6 Remove now-obsolete get_message_by_id_dbwarn transition code. 2016-01-26 21:05:11 -08:00
Tim Abbott eecd1513b3 Don't access/store full Client objects in Tornado queue servers.
At present, we only do a few simple checks on the client type inside
the event system, and this saves database/memcached queries.

Note that this preserves the structure of the marshalled name in
to_dict/from_dict as client_type to avoid an unnecessary migration.
2016-01-26 21:04:32 -08:00
Tim Abbott e3b6bfa3ca Remove code for pickle event queue store.
Since we've had the JSON store in all environments for some time, we
no longer need this legacy code.
2016-01-26 20:59:34 -08:00
Tim Abbott f6073d1708 Move HTTPResponse construction out of event_queue.py. 2016-01-26 20:59:19 -08:00
Tim Abbott a9bf4b4cc7 Store the client name for the current handler in ClientDescriptor. 2016-01-26 20:58:01 -08:00
Tim Abbott c7e3c3ce38 Look up client descriptors by handler_id.
Previously, client descriptors were referenced directly from the
handler object.  Once we split the Tornado process into separate queue
and connection servers, these will no longer be in the same process,
so we need to reference them by ID instead.
2016-01-26 20:57:25 -08:00
Tim Abbott ea6211c041 Allocate handler ids in AsyncDjangoHandler setup process. 2016-01-26 20:56:53 -08:00
Tim Abbott ae760a351e Move most of ClientDescriptor.finish_current_handler into library. 2016-01-26 20:56:08 -08:00
Tim Abbott 7df61fccbd Add user_profile_email field to ClientDescriptor. 2016-01-26 20:55:59 -08:00
Tim Abbott 2ea0daab19 Track Tornado handlers by uniquely assigned IDs rather than objects.
This is early preparation for splitting apart Tornado into a queue
server and a frontend server.
2016-01-26 20:55:41 -08:00
Tim Abbott 8b42fdd0d7 Move get_events logic into a backend function in event_queue.py.
This commit is somewhat ugly, but its purpose is to be early
preparation for splitting Tornado into a queue server and a frontend
server, and this code belongs, by and large, in the queue server
component.
2016-01-26 20:51:54 -08:00
Tim Abbott 5a6154c8ba test_helpers: Fetch streams from cache in subscribe_to_stream. 2016-01-26 20:41:01 -08:00
Tim Abbott a5d4d0aae0 test_backend: Add option to profile the backend test suite. 2016-01-26 20:41:01 -08:00
Tim Abbott f9791558e9 test_runner: Support continuing running tests after a failure. 2016-01-26 20:41:01 -08:00
Tim Abbott b43aadad8b test-backend: Rewrite in python to support computing test coverage.
The code for doing test coverage is just a lot cleaner this way over
adding it to the shell script version.

Based on the basic test runner code here:
https://docs.djangoproject.com/en/1.9/topics/testing/advanced/
2016-01-26 20:41:01 -08:00
Tim Abbott 24fd3bbf55 travis: Test whether migrations are consistent with models.
This should automatically catch mistakes where someone updates the
database models but forgets to generate migrations afterwards.
2016-01-26 20:38:46 -08:00
Tim Abbott 5ef57a07e1 Add missing migrations present in models.py.
89a2765553 didn't include the database
migration corresponding to the change, which means it didn't take full
effect when it was merged.

I noticed this because `manage.py makemigrations` would generate these
migrations; that suggests a good idea for a test to add.
2016-01-26 20:38:46 -08:00
Tim Abbott 1c73c992dd Fix missing puppet dependencies on postgres package. 2016-01-26 20:32:33 -08:00
Tim Abbott 2e16b44b24 puppet: Use $postgres_version in postgres template.
This eliminates hardcoding of the postgres version from the Zulip
puppet configuration.
2016-01-26 20:32:33 -08:00
Tim Abbott 806aa986b7 puppet: Use a variable to configure the postgres version. 2016-01-26 20:32:33 -08:00
Tim Abbott a3ac56efe2 puppet: Make apt repository conditional on the Ubuntu version.
We still will need to address this in the install script as well.
2016-01-26 20:32:33 -08:00
Tim Abbott f6c59feb05 Document the puppet configuration somewhat in zulip::voyager. 2016-01-26 20:32:33 -08:00
Tim Abbott 345b5254d7 puppet: Move default nginx configuration out of voyager.pp. 2016-01-26 20:32:33 -08:00
Tim Abbott dd61e3f97d puppet: Move memcached and rabbitmq include out of app_frontend_base.pp. 2016-01-26 20:32:33 -08:00
Tim Abbott c3153274c1 puppet: Move memcached into its own puppet module. 2016-01-26 20:32:33 -08:00
Tim Abbott 8a0e07fe1a puppet: Rename app_frontend.pp to app_frontend_base.pp.
This will enable us to move the remaining app-frontend related content
out of voyager.pp.
2016-01-26 20:32:33 -08:00
Tim Abbott 91286d00aa puppet: Move Zulip apt repository to its own manifest. 2016-01-26 20:32:33 -08:00
Tim Abbott 69dd17dfb6 puppet: Move prod-static creation from voyager.pp to app_frontend.pp.
Every app frontend will need this directory and this should help
enable more modular puppet rules.
2016-01-26 20:32:33 -08:00
Tim Abbott 702f501638 puppet: Move tuned postgres configuration out of voyager.pp.
This should make it easier for someone to run just the tuned Zulip
database on one server and the Zulip frontend on another.
2016-01-26 20:32:28 -08:00
Tim Abbott d5f04bd20b Rename zulip::postgres_appdb to zulip::postgres_appdb_base.
The purpose of this rename is to allow us to move the postgres-related
configuration out of voyager.pp.
2016-01-26 20:30:12 -08:00
Tim Abbott 3f27573cb2 puppet: Move several debugging tool dependencies out of base.pp. 2016-01-26 20:30:11 -08:00
Tim Abbott fdc7f5b86a Vagrantfile: Default to using LXC when Virtualbox is also available.
This solves the problem reported in #331 with needing to specify
--provider=lxc to use the LXC provider in an Ubuntu Linux environment;
additionally, it adds the LXC option needed to run LXC on Ubuntu
15.10, but not on 14.04 where that option is unavailable and would
totally break LXC.
2016-01-23 11:45:47 -08:00
Tim Abbott 50bc32dc95 integrations: Document server-side setup for twitter integration. 2016-01-23 11:44:53 -08:00
Tim Abbott c6d06b0c4e Add zulip distribution tarballs to backups. 2016-01-23 11:44:15 -08:00
Tim Abbott 529d7a2877 Release API version 0.2.5. 2016-01-23 11:38:47 -08:00
Tim Abbott d3588cb7d0 api: Include get_subscribers endpoint in public release.
It's possible we should just eliminate this mechanism, but this fixes
a proximal problem where the multi-line get_subscribers endpoint
description was being handled wrong.
2016-01-23 11:38:42 -08:00
Tim Abbott fdf708039b Disable empty-stream notifications for email gateway bot. 2016-01-23 11:37:52 -08:00
Vladislav Manchev df4d1b3c14 Add linting code for detecting shebang bashisms.
This will prevent regressions in OpenBSD compatibility, since OpenBSD
doesn't support passing arguments in the #! line.
2016-01-21 22:33:55 -08:00
Vladislav Manchev dfbea01c8f Add support for running OpenBSD in development environment. 2016-01-21 22:33:55 -08:00
Alexander Trost 84f7a1f1ea Make rabbitmq, redis, and memcached configurable via user settings.py.
Previously these were hardcoded in zproject/settings.py to be accessed
on localhost.

[Modified by Tim Abbott to adjust comments and fix configure-rabbitmq]
2016-01-21 22:07:56 -08:00
Tim Abbott 6943a142ea Fix postgres errors in Travis CI again.
Travis CI's model of installing every version of postgres on the test
VM and then shutting all the versions other than the one requested
down seems to not work very well with doing apt upgrades.  It seems
the best way to resolve this is to just uninstall the versions we
don't need.
2016-01-21 22:07:10 -08:00
Tim Abbott e1e7ea01ca Add changelog for Zulip 1.3.10 release. 2016-01-21 20:24:53 -08:00
Vladislav Manchev 62d021f399 Change administration UI to utilize tabs. 2016-01-12 22:08:10 -08:00
Reid Barton ed412281d0 Fix typo in Google OAuth error message. 2016-01-12 09:32:09 -05:00
Tim Abbott 5d54f66047 get_deployment_lock: Flush output when reporting lock delays.
This fixes an issue where the output that the lock wasn't unavailable
showed up all at once after the 5 minute timeout.
2016-01-11 21:36:42 -08:00
Tim Abbott 26e9d55e16 deployments: Refactor locking libraries into zulip_tools.py.
The code in update-deployment and upgrade-zulip for managing the
deployment lock was nearly identical.
2016-01-11 21:36:42 -08:00
Tim Abbott f871090bb6 upgrade-zulip: Archive release tarballs at /home/zulip/archives.
A common issue when doing a Zulip upgrade is trying to pass
upgrade-zulip a tarball path under /root, which doesn't work because
the Zulip user doesn't have permission to read the tarball.  We
could fix this by just unpacking the tarballs as root, but it seemed
like a nicer approach would be to archive the release tarballs
somewhere readable by the Zulip user (/home/zulip/archives) and unpack
them from there.

Fixes #208.
2016-01-11 21:36:42 -08:00
Tim Abbott c101bf663d Run upgrade-zulip-stage-2 from an absolute path.
This should make it more obvious in tracebacks that we are running the
script from the version of Zulip we're upgrading to, not the old
version.
2016-01-11 21:36:42 -08:00
Tim Abbott 52d0423591 Document structure of upgrade-zulip-stage-2 more clearly. 2016-01-11 21:36:41 -08:00