Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/zerver/lib
History
Steve Howell a331b4f64d Optimize query_all_subs_by_stream(). 
Using lightweight objects will speed up adding new users
to realms.

We also sort the query results, which lets us itertools.groupby
to more efficiently build the data structure.

Profiling on a large data set shows about a 25x speedup for this
function, and before the optimization, this function accounts
for most of the time spend in bulk_add_subscriptions.

There's a lot less memory to allocate.  I didn't measure
the memory difference.

When we test-deployed this to chat.zulip.org, we got about a 6x
speedup.
 		2017-10-06 11:03:44 -07:00
..
bugdown emoji: Migrate bugdown emoji to use sprite sheets. 2017-09-29 11:14:34 -07:00
url_preview zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
webhooks mypy: strict optional fixes. 2017-05-24 18:50:59 -07:00
__init__.py
…
actions.py Optimize query_all_subs_by_stream(). 2017-10-06 11:03:44 -07:00
addressee.py refactor: Move default_sending_stream logic to Addressee. 2017-09-28 12:14:08 -07:00
alert_words.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
attachments.py settings: Show error or success message when deleting uploaded files. 2017-09-27 17:50:22 -07:00
avatar.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
avatar_hash.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
bot_lib.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
bulk_create.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
cache.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
cache_helpers.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
camo.py pep8: Add compliance with rule E261 to camo.py. 2017-05-07 23:21:50 -07:00
ccache.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
context_managers.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
create_user.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
db.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
debug.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
digest.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
domains.py Add stricter domain validation and improve error messages. 2017-01-26 17:24:25 -08:00
email_mirror.py email_mirror: Filter out null characters in email bodies. 2017-10-03 15:32:05 -07:00
emoji.py reactions: Switch to using `name_to_codepoint`. 2017-10-04 23:09:14 -07:00
error_notify.py error_notify: Add IP address to browser error reports. 2017-10-04 13:46:05 -07:00
events.py Simplify how we apply events for unread messages. 2017-10-05 09:42:20 -07:00
exceptions.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
export.py django: Bump version to 1.11.5. 2017-10-03 08:27:06 -07:00
feedback.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
fix_unreads.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
generate_test_data.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
hotspots.py hotspots: Use tutorial_status to manage whether we show hotspots. 2017-09-15 04:14:52 -07:00
html_diff.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
i18n.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
initial_password.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
integrations.py email_mirror: Clean up EMAIL_GATEWAY_EXAMPLE setting. 2017-09-29 21:32:26 -07:00
logging_util.py logging: Abbreviate log-level names to 4 characters. 2017-09-28 18:26:39 -07:00
management.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
mention.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
message.py Collect sender_ids (by topic) in `unread_msgs`. 2017-10-05 10:37:15 -07:00
migrate.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
mobile_auth_otp.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
name_restrictions.py name_restrictions: Update with new competitors and portico pages 2017-07-26 12:10:58 -04:00
narrow.py Update narrow.py, messages.py to support is:unread. 2017-06-19 06:51:13 -04:00
notifications.py emoji: Migrate bugdown emoji to use sprite sheets. 2017-09-29 11:14:34 -07:00
onboarding.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
outgoing_webhook.py outgoing webhook: Make exception handling more granular and verbose. 2017-09-30 10:14:28 -07:00
parallel.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
profile.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
push_notifications.py push_notifications: Set APNS badge as 0 for now. 2017-10-03 17:07:49 -07:00
queue.py mypy: Use Dict instead of Mapping in queues. 2017-10-03 10:35:07 -07:00
rate_limiter.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
realm_icon.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
redis_utils.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
request.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
request.pyi JsonableError: Move into a normally-typed file. 2017-07-24 16:41:22 -07:00
response.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
rest.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
retention.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
send_email.py backends: Create custom email backend EmailLogBackEnd. 2017-10-04 08:20:29 -07:00
sessions.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
soft_deactivation.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
sqlalchemy_utils.py mypy: Remove a bunch of now-unnecessary type: ignore annotations. 2016-10-17 11:48:34 -07:00
statistics.py zerver/lib: Remove from __future__ import *. 2017-09-28 10:14:03 -07:00
str_utils.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
stream_recipient.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
streams.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
test_classes.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
test_data.source.txt test_data.source.txt: Replace postmodernism generator text with a play. 2017-08-23 13:00:39 -07:00
test_fixtures.py Create complex sample messages for dev VM. 2017-08-01 20:54:39 -07:00
test_helpers.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
test_runner.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
tex.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
timeout.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00
timestamp.py analytics: Enforce various datetime arguments are in UTC. 2017-10-05 11:22:06 -07:00
timezone.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
topic_mutes.py Have topic_is_muted take a stream_id. 2017-10-05 09:32:16 -07:00
type_debug.py zerver/lib: Remove print_function. 2017-09-27 18:05:45 -07:00
unminify.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
upload.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
user_agent.py tests: Replace test_user_agent_parsing with a normal test. 2017-09-16 03:56:18 -07:00
users.py zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
utils.py subdomains: Hardcode REALMS_HAVE_SUBDOMAINS=True. 2017-10-02 16:42:43 -07:00
validator.py zerver/lib: remove `import six`. 2017-09-27 19:10:28 -07:00