Commit Graph

701 Commits

Author SHA1 Message Date
Tim Abbott 26463bb34d Fix nondeterministic subscriptions for default test users.
Previously, the UserProfile objects were created in the order
generated by a Set, which meant tests would randomly start failing if
the code that runs before this part of populate_db changed (and thus
caused the Set object used to pass users into bulk_create_users to
have a different order when enumerated).

This fixes the issue in two ways -- one by sorting the users inside
bulk_create_users, and second by attaching subscriptions to users
based on a deterministic ordering.
2016-04-13 13:19:02 -07:00
Tim Abbott ffccb572f0 Don't autoreload Tornado when running inside test suite.
The restarted Tornado processes seemed to escape the process group and
thus continue running after run-dev.py finished.

While we're at it, we don't need to dump/reload event queues in the
test suite either.
2016-04-13 13:19:01 -07:00
Ashish cb8da46bbf Replace /json/get_old_messages with REST style route. 2016-04-11 21:38:23 -07:00
Ashish 41993ef2f5 Replace /json/update_message_flags with REST style route. 2016-04-11 21:38:22 -07:00
Ashish 9cfa7d5765 Annotation of zerver/lib/handlers.py. 2016-04-08 11:18:36 -07:00
Ashish 86a8d3d0f5 Annotation of zerver/lib/rate_limiter.py 2016-04-08 11:18:36 -07:00
Ashish 2f8c717e52 List to tuple conversion for consistency in user rules zerver/lib/rate_limiter.py. 2016-04-08 11:18:35 -07:00
Ashish 8abca4f319 Annotation of zerver/lib/migrate.py 2016-04-08 11:18:35 -07:00
Ashish 038af80889 Annotation of zerver/lib/statistics.py 2016-04-08 11:11:58 -07:00
Ashish 2cf8731444 Annotation of zerver/lib/alert_words.py 2016-04-08 11:11:58 -07:00
Varshit 4e1060076d Purge 'from typing import *' from zerver/.
This is a partial implementation of #636.
2016-04-07 14:07:07 -07:00
Tim Abbott 75d134a9b2 Import tornado.autoreload explicitly.
Fixes #623.
2016-04-03 16:52:23 -07:00
Varshit e0ef1a991e Rewrite sanitize_name to better preserve filenames.
The previous version of sanitize_name dropped all unicode characters
and mangled filenames with multiple `.`s in the extension, leading to
confusing URLs for files uploaded to Zulip.

Fixes #321.

[tweaked significantly by tabbott]
2016-04-03 16:26:12 -07:00
Tim Abbott 4a50336476 Add type: Any for tornado.ioloop.add_timeout return value. 2016-04-03 15:40:24 -07:00
Tim Abbott b6dd6413d0 Add type: ignore for tornado_ioloop_logging Tornado versioning code. 2016-04-03 15:40:24 -07:00
Tim Abbott 53ab18eea0 Add type: ignore for chain.from_iterable. 2016-04-03 15:40:24 -07:00
Tim Abbott e3ec3e2526 Add type: ignore for test_helpers monkey-patching. 2016-04-03 15:40:24 -07:00
Tim Abbott 2059f650ab Add PEP-484 type annotations to zerver/lib/. 2016-04-03 15:40:23 -07:00
Tim Abbott b99313545e Add PEP-484 type annotations to global dictionaties. 2016-04-03 15:40:06 -07:00
Tim Abbott 77be524dc4 Add PEP-484 type annotations to generator functions. 2016-04-03 15:40:05 -07:00
Tim Abbott 96eb81e5d5 Add documentation of Zulip's markdown implementation. 2016-04-01 09:56:59 -07:00
Ryan Moore 2fe0700f55 Update memcache -> remote cache in inline documentation. 2016-03-31 12:56:42 -07:00
Ryan Moore 1489f0992c s/fill_memcached_cache/fill_remote_cache/g 2016-03-31 12:54:29 -07:00
Ryan Moore 18139fd86f s/memcached_prefix/remote_cache_prefix/g 2016-03-31 12:54:29 -07:00
Ryan Moore 1f120aa3a8 s/memcached_total/remote_cache_total/g 2016-03-31 12:54:29 -07:00
Ryan Moore 3c180b43df s/memcached_stats/remote_cache_stats/g 2016-03-31 12:54:28 -07:00
Ryan Moore 1a2117292f s/memcached_requests/remote_cache_requests/g 2016-03-31 12:54:28 -07:00
Ryan Moore 16c936f638 s/memcached_time/remote_cache_time/g 2016-03-31 12:54:28 -07:00
Ryan Moore 9f29b80f8a s/items_for_memcached/items_for_remote_cache/g 2016-03-31 12:54:28 -07:00
Tim Abbott 723d8c288a Add tornado autoreload hook to dump event queues on reload.
It's always been the case that in production, Tornado dumps all the
event queues when shut down so that they can be reloaded by the
replacement Tornado process.  This never worked in development because
the codepath for auto-reload didn't go through either a signal or
sys.exit (it re-execs the process instead).

This meant that we didn't have a mechanism for testing the event queue
dump/load functionality in the development environment.  We fix this
by adding such dumping/loading.  However, this breaks the automatic
reloading of open browser windows on a server restart, so we add that
back in by adjusting the special `restart` events to pass a special
`immediate` flag when used in development.

This also has the benefit of removing the "Bad event queue" errors one
would get on every file save induced restart on the Python console.
2016-03-30 23:09:16 -07:00
Tim Abbott 59e2be2f5f Notify clients when an event queue is garbage collected. 2016-03-30 23:09:14 -07:00
Tim Abbott bf43db0dad Send responses from fetch_events in dict format. 2016-03-30 22:04:09 -07:00
Tim Abbott 485e46f136 Wrap most of fetch_events in a try/except JsonableError.
This is a no-op right now, but we'll want the new structure for the
next commit, and splitting this out makes it a lot easier to read what
is actually changed in the next commit.
2016-03-30 22:02:32 -07:00
Tim Abbott ad1494f8e0 Use dictionaries for passing data into event server subsystem.
This marshalls data in a portable format in preparation for splitting
the event queue server from the Tornado server.
2016-03-30 22:01:08 -07:00
Tim Abbott 6cd14af18f events: Migrate client_type => client_type_name in marshalling code. 2016-03-30 22:00:50 -07:00
Tim Abbott cd2348e9ae Run queue processers multithreaded in development.
This change drops the memory used for Python processes run by Zulip in
development from about 1GB to 300MB on my laptop.

On the front of safety, http://pika.readthedocs.org/en/latest/faq.html
explains "Pika does not have any notion of threading in the code. If
you want to use Pika with threading, make sure you have a Pika
connection per thread, created in that thread. It is not safe to share
one Pika connection across threads.".  Since this code only connects
to rabbitmq inside the individual threads, I believe this should be
safe.

Progress towards #32.
2016-03-20 18:04:24 -07:00
Tim Abbott 0419430000 Enhance Tornado logging with Handler stats.
This was useful data for debugging handler/memory leaks.
2016-03-20 16:53:13 -07:00
Tim Abbott e51811aa9e Add comment documenting logic flow of fetch_events. 2016-03-20 16:53:13 -07:00
Tim Abbott 7fabfe9cb9 Add __repr__s for ClientDescriptor and AsyncDjangoHandler. 2016-03-20 16:53:13 -07:00
Tim Abbott 0b96e5e43f Fix connection not being closed cleanly on event queue GC.
Apparently, our event queue garbage collection logic never actually
disconnected any existing handler objects.

We fix this by disconnecting the handler inside cleanup(), adding a
special check to avoid creating a pointless timeout object.
2016-03-20 16:53:13 -07:00
Tim Abbott 12a5a3a6e1 Actually fix main Tornado memory leak of handler objects.
This line appears to have been lost in rebasing from the original
implementation of 1396eb7022faec4c2d91553800a35781a96dd5bd; so the
previous fix actually only addressed the issue in a rare exception
case.
2016-03-20 16:53:12 -07:00
Eklavya Sharma 9e3c3e14f5 Partially apply Python 3 libmodernize.fixes.fix_dict_six.
Refer to #256
2016-03-19 15:52:58 -07:00
Eklavya Sharma 176c507b0a Removed calls to ifilterfalse.
Replaced calls to ifilterfalse by list comprehensions because
ifilterfalse is not part of python 3.  Also changed some lists to sets
for faster lookup.

Refer to #256.
2016-03-19 15:46:31 -07:00
Eklavya Sharma 186efc6a6d Handle unicode properly in camo.
In Bugdown's InlineHttpsProcessor.run, hex_encoded_url was not being
decoded to utf-8. This was causing problems in python 3.

Refer to #256.
2016-03-19 15:46:07 -07:00
Tim Abbott 1396eb7022 Fix Tornado memory leak of handler objects.
In 2ea0daab19, handlers were moved to
being tracked via the handlers_by_id dict, but nothing cleared this
dict, resulting in every handler object being leaked.  Since a Tornado
process uses a different handler object for every request, this
resulted in a significant memory leak.  We fix this by clearing the
handlers_by_id dict in the two code paths that would result in a
Tornado handler being de-allocated: the exception codepath and the
handler disconnect codepath.

Fixes #463.
2016-03-17 18:33:59 -07:00
Eklavya Sharma c59185e119 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import
Refer #256
2016-03-10 22:02:17 -08:00
Tim Abbott df0d2a726d python3: Add missing utf-8 encoding/decoding in various places. 2016-03-08 09:14:15 -08:00
Tim Abbott a46647a87a python3: Mark certain strings as unicode strings.
This is required in Python 3 since these strings are combined with
other unicode strings.
2016-03-08 09:14:11 -08:00
Reid Barton 9735025167 Refactor logic around restricted_to_domain.
Add a function email_allowed_for_realm that checks whether a user with
given email is allowed to join a given realm (either because the email
has the right domain, or because the realm is open), and use it
whenever deciding whether to allow adding a user to a realm.

This commit is not intended to change any behavior, except in one case
where the Zulip realm's domain was not being converted to lowercase.
2016-02-07 10:54:52 -05:00
Tim Abbott 3469fd4bb2 Fix last remaining use of file() -> open(). 2016-02-03 19:31:46 -08:00