Commit Graph

108 Commits

Author SHA1 Message Date
Pweaver (Paul Weaver) 8f05ea877d lint: Fix pep8 linter error regressions.
Since `lint-all --pep8` wasn't running in CI, a few errors have leaked
in.
2016-11-17 00:09:21 -08:00
Shashank 702d3cf103 Check provision version in run-dev.py.
Don't start run-dev.py if the provisioning version is
incorrect.
2016-11-09 16:06:16 -08:00
Tim Abbott a93530e413 Remove unused route for development tornado proxy server.
This was a duplicate of the other sockjs line.
2016-11-03 10:46:18 -07:00
K.Kanakhin 870de03ede run-dev: Add websockets support to tornado dev proxy server.
Fixes #1036.
2016-11-01 20:35:31 -07:00
K.Kanakhin 09e17fbe17 run-dev: Use tornado for dev proxy server for HTTP requests.
- Use tornado as proxy server for development environment,
  replacing twisted (this doesn't support websockets).
- Upgrade tornado version to 4.4.1 (needs to be coupled to the
  above since neither change works without the other)
2016-11-01 20:35:31 -07:00
Tim Abbott 6a8f8c2abf run-dev: Fix minor whitespace issues. 2016-11-01 20:27:44 -07:00
Tim Abbott 9b7a3f040c Remove now-unused /json/get_events endpoint. 2016-10-27 21:34:58 -07:00
K.Kanakhin 06e70e1fbd run-dev-proxy: Fix twisted py3 compatibility.
- All necessary strings was converted to bytestring
 - Added twisted as py3 dependency
 - Change type annotation for method getchild of class Resource
 - Remove activating python2 env section from run-dev.py script

Fixes #1256
2016-10-06 20:21:25 -07:00
Tim Abbott 82764d0215 run-dev: Display the port 9991 warning in colored text. 2016-10-05 22:07:46 -07:00
Feorlen e09c27f1ae run-dev.py: Log what service is being started on which port.
The new messages make it more obvious which services are started
from run-dev.py, and explicitly call out where to access the web
proxy to reach the Zulip web UI. This is a common confusion for
new administrators/developers. Messages are output before the
processes are launched, as run-dev.py does not currently have a
way to know if they started successfully.

Example output:

Starting Zulip services on ports:  web proxy: 9991, Django: 9992, Tornado: 9993, webpack: 9994
Note: only port 9991 is exposed to the host in a Vagrant environment.

Alternate behavior for automated testing:

If run-dev.py is invoked with --test, don't include the webpack
port as it isn't used.

Tested on Ubuntu 14.04, by running run-dev.py at a shell prompt and
via the test-all script.

Fixes #1861
2016-10-05 22:07:46 -07:00
Tim Abbott 279f805448 run-dev: Automate passing --interface='' for vagrant environment.
Fixes #877.
2016-10-04 10:48:26 -07:00
Tim Abbott 3c34264559 run-dev: Automatically fixup missing handlebars binary. 2016-09-07 19:35:41 -07:00
Tim Abbott 89267f926e Annotate tools/run-dev.py. 2016-08-04 15:57:02 -07:00
Tim Abbott c2bed83a84 run-dev: Hardcode 127.0.0.1 rather than localhost.
Using "localhost" is problematic in Travis CI, where they have
/etc/hosts configured to make `localhost` resolve to both 127.0.0.1
and ::1.
2016-07-27 11:39:12 -07:00
Eklavya Sharma 5054a8d6c9 tools/run-dev.py: Use twisted venv if on python 3. 2016-07-19 14:15:35 -07:00
Eklavya Sharma 38b1353f42 tools/run-dev.py: Clear memcached.
Clear memcached when tools/run-dev.py is run.  This prevents
errors on using a different python version because values are
pickled before being stored in memcached and different python
versions implement pickling differently.

Also provide a command-line option --no-clear-mc to prevent
memcached from being cleared.
2016-07-18 14:25:13 -07:00
Eklavya Sharma da36947400 Change unbuffering strategy in runtornado.py.
runtornado unbuffers its output using
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0).
This is not python 3 compatible since we can't specify
buffering on a text stream in python 3.  So use the '-u'
option of python when calling runtornado.py to make output
unbuffered.
2016-07-17 10:31:15 -07:00
Tim Abbott a78c1b9750 Fix casper tests hanging while development server is running.
This works around a nasty problem with Webpack that you can't run two
copies of the Webpack development server on the same project at the
same time (even if on different ports).  The second copy doesn't fail,
it just hangs waiting for some lock, which is confusing; but even if
that were to be solved, we don't actually need the webpack development
server running to run the Casper tests; we just need bundle.js built.
So the easy solution is to just run webpack manually and be sure to
include bundle.js in the JS_SPECS entry.

As a follow-up to this change, we should clean up how test_settings.py
is implemented to not require duplicating code from settings.py.

Fixes #878.
2016-06-30 14:18:50 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Tim Abbott 98d5f64f36 webpack: Use the correct port in Casper tests.
Previously we used 9994 unconditionally, whereas we should be using
9984 or 9994 depending whether it's being run manually or via the
Casper tests.
2016-04-13 13:18:38 -07:00
Tim Abbott df4ab3c788 run_dev: Fix return for twisted finish function.
This was flagged by mypy; it's not clear this should be needed.
2016-02-03 19:29:07 -08:00
Tim Abbott eee36618fe run-dev: Fix overwritten manage_args variable with the wrong type.
manage_args is set to a list of arguments a few lines later in the
function, making this initialization as the empty string useless and
confusing.

Discovered using mypy.
2016-02-03 19:25:18 -08:00
Tim Abbott dfaf45b2b6 Wrap Django runserver to prevent spammy logging.
Django's `manage.py runserver` prints a relatively low-information log
line for every request of the form:

[14/Dec/2015 00:43:06]"GET /static/js/message_list.js HTTP/1.0" 200 21969

This is pretty spammy, especially given that we already have our own
middleware printing a more detailed version of the same log lines:

2015-12-14 00:43:06,935 INFO     127.0.0.1       GET     200   0ms /static/js/message_list.js (unauth via ?)

Since runserver doesn't have support controlling whether these log
lines are printed, we wrap it with a small bit of code that silences
the log lines for 200/304 requests (aka the uninteresting ones).
2015-12-25 16:23:57 -08:00
Allie Jones 85809e6140 Add webpack build process. 2015-11-06 09:13:25 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott 758baca01a run-dev.py: Report a nice error if you run it as root.
Fixes #172.
2015-10-15 12:45:38 -04:00
Reid Barton 3cfebbd9c0 Fix peer authentication error for process_fts_updates in run-dev.py
(imported from commit 1c73877a1c437ec51d909034460c3c1b54178183)
2015-08-21 14:10:52 -07:00
Anders Kaseorg dd8eda4edc run-dev: Run compile-handlebars-templates inside the pgrp
(imported from commit bb13f8804b7e9e930ecd2542ea8772c76370d32e)
2015-08-20 15:54:53 -07:00
Anders Kaseorg bd66b2139b run-dev: Eschew useless shells
(imported from commit 34dab15941f5c63f381f5b2b7b78482dbc2b02ea)
2015-08-20 15:54:53 -07:00
Luke Faraone 04d3eb33b1 run-dev.py: Reword --interface help text.
The old language was confusing because "the interface" could refer to something
like eth0, but in actuality refers to the IP/hostname to listen on.

(imported from commit 4f77d72a4dfcdbe7e7747c6228975aa68dfbe6ac)
2015-08-19 21:07:22 -07:00
Jason Michalski 2530009123 Use optparse for command line argument parsing
optparse was already being the interface change was conflicting with it.

(imported from commit abdb153484a98a46b21fa77cc6deb00afd615720)
2014-02-12 14:03:58 -05:00
Jason Michalski 97d4f2b009 Allow passing a bind interface to run-dev.py
This makes it simpler to test between two VMs by allowing you to bind to
non localhost interfaces.

(imported from commit f70755533b52ff8c49fd916941d2210fb8c33b47)
2014-02-12 11:59:45 -05:00
Jason Michalski f31b61ea5b Pass the client's Host header to django as X-Forwarded-Host
This allows Django to build correct redirects to hostnames other than
localhost.

(imported from commit 886b345f29fc03518cedd93ecbcea0be251f3230)
2014-02-12 11:59:45 -05:00
Zev Benjamin 395bf9abf6 Start up process_fts_updates in run-dev.py
(imported from commit 46912d89160f7a9b992f7c21a069030ba3823a43)
2014-02-03 17:46:57 -05:00
Zev Benjamin 4c2800ef9d Start up queue processor workers in parallel with the rest of run-dev.py
Importing zerver.worker.queue_processors (which is needed to get the list of
workers to start) is slow because it, in turn, imports a bunch of stuff.  So we
move the process of starting up queue processing workers into another script
that gets started in parallel with everything else.

(imported from commit 839bada6dc7b93825c69b0d8fd9fbe2de75eabee)
2014-02-03 17:46:57 -05:00
Steve Howell 1ad82b1f70 Make run-dev.py watch for template changes.
Have run-dev.py watch for template changes by calling
`./tools/compile-handlebars-templates forever`.  This doesn't
have much effect until the subsequent commit, but it does
alert users to broken templates.

(imported from commit 3fa5f403cabe0057f6f43180f1d09db669d98682)
2014-01-10 21:39:04 -05:00
Steve Howell 751f400afb Add a --minimal option to run-dev.py.
This replaces the --noworkers option; the new --minimal option
starts a couple "essential" workers.

(imported from commit 4ca08709052c47257bc0448e51760edb4969d92e)
2013-12-16 14:36:06 -05:00
Steve Howell 504d882090 Add --noworkers option to run_dev.
(imported from commit d8253f50af79fcf290e75d31d5c5d9b79118a463)
2013-12-13 10:17:27 -05:00
Tim Abbott ca8225cf47 [manual] Add endpoint to cleanup a finished events queue.
This requires a puppet apply on each of staging and prod0 to update
the nginx configuration to support the new URL when it is deployed.

(imported from commit a35a71a563fd1daca0d3ea4ec6874c5719a8564f)
2013-11-20 18:34:15 -05:00
Tim Abbott e06722657a [manual] Remove /messages/latest API and related legacy code.
This requires doing a puppet apply on our servers to take effect
properly.

(imported from commit 19dc56f071f07a5d2571eef49dd835121b2e82b6)
2013-11-05 14:19:40 -05:00
Tim Abbott 901a5c342c Use os.path directly rather than sometimes importing it separately.
(imported from commit 48486c4ea64d02a15faeebb0f490d31e9b113d62)
2013-10-28 15:44:13 -04:00
Zev Benjamin e5470b2b66 Run queue processor workers from run-dev.py
(imported from commit d8977fbb899bf4a1b7e3a644a9259279ac744efd)
2013-10-28 14:30:53 -04:00
Zev Benjamin 5979af3a45 [manual] Add asynchronous message sender via sockjs-tornado
New dependency: sockjs-tornado

One known limitation is that we don't clean up sessions for
non-websockets transports.  This is a bug in Tornado so I'm going to
look at upgrading us to the latest version:
https://github.com/mrjoes/sockjs-tornado/issues/47

(imported from commit 31cdb7596dd5ee094ab006c31757db17dca8899b)
2013-10-22 18:45:11 -04:00
Tim Abbott b8b0eb4508 Some zulip => humbug updates in comments and print statements.
(imported from commit 9253569a1df7f96fda81ab162d710cdda03f30ca)
2013-10-08 08:57:29 -04:00
Zev Benjamin 2f1551e745 Replace comment references to trac.humbughq.com with trac.zulip.net
(imported from commit 231d0884345be525d58ce51f48a76bba6960f09d)
2013-09-05 15:43:08 -04:00
Tim Abbott 8879b84293 run-dev: Ignore certain request.finish() exceptions.
(imported from commit e797c0a6f5cd753bd84069bff202b421f0ff3be9)
2013-08-21 12:01:55 -04:00
Tim Abbott 7b9305b06f Rename Django project to zproject.
This includes a hack to preserve humbug/backends.py as a symlink, so
that we don't need to regenerate all our old sessions.

(imported from commit b7918988b31c71ec01bbdc270db7017d4069221d)
2013-08-07 11:04:03 -04:00
Zev Benjamin 401fa6063e [manual] Add get_events URL routing
The new nginx configuration file needs to be copied to
/etc/nginx/humbug-include and nginx needs to be restarted when this
commit is deployed.

(imported from commit 6c43f3c2c7a6acee6a852c672c96a38bda01dd0d)
2013-03-28 16:48:52 -04:00
Luke Faraone 3738d8215d Update run-dev to prefix match messages/latest.
This means all GET parameters were chopped off previously in local testing.

Nginx does pattern matching so this is not a problem on staging.

(imported from commit 25a28155b70d168228ca793fc0122b2ebea408e9)
2013-03-28 12:53:42 -07:00
Luke Faraone 0d51e59fd5 Implement URLs for API redesign.
(imported from commit 2020491a737ec4c1e99a63f84eb6cfc594a2dd56)
2013-03-28 07:57:38 -07:00
Keegan McAllister ae45217671 run-dev: Clean repo first
Makes sense because test-all has the same behavior.

(imported from commit 4c4e5ed539bf36fbb5f298abc40f4718dd9a93f1)
2013-03-26 13:50:03 -04:00
Keegan McAllister 638b8d29bc Enable testing minified files in dev
(imported from commit 257b8547849a85c447319d3d211f2c989616ce64)
2013-01-31 15:41:01 -05:00
Keegan McAllister 88a76518ec run-dev: Print traceback on exception
We can't rely on the builtin exception printing, because killpg will kill the
process before that happens.

(imported from commit e7db6a39e0549e21653b4a705d2278eb3dc7b2d4)
2012-11-16 15:33:37 -05:00
Keegan McAllister 5d8b66bf33 run-dev: Don't leak servers when we fail to bind to the proxy port
(imported from commit da7cd1f62f3d073bdae6fe72e71f4127931f4d57)
2012-11-16 15:33:37 -05:00
Keegan McAllister eee951440d run-dev: Use a process group to kill all spawned processes
(imported from commit 9dfddd4d9cb45c70139a167b7949bd5c4fc2791a)
2012-11-13 10:59:02 -05:00
Keegan McAllister ed490c672f Allow running the dev server with the test database, on different ports
For use by frontend tests.

(imported from commit c8f81b862963f00e5b5517ba05b2d1adcab6d78a)
2012-11-13 10:59:02 -05:00
Keegan McAllister 6de3321f02 run-dev: Update for new get_updates URL
(imported from commit 2617ddd1c2ec377d6f17f1aafd0da251226d39b8)
2012-10-17 18:24:15 -04:00
Keegan McAllister 21ed4d2506 Add a script for running a local dev instance of the app
(imported from commit 29813060ef17360d3a7dcc6fe1873992404457c2)
2012-10-17 18:23:01 -04:00