Commit Graph

5768 Commits

Author SHA1 Message Date
Tim Abbott 660d8d4bc4 ScheduledJob: Add a __str__ function to make them more easily printed. 2017-09-21 06:05:18 -07:00
Steve Howell 9110ff935e tests: Restore 100% coverage to views/messages.py.
A recent change moved some untested code into messages.py, which
requires 100% coverage.  Now we test bogus stream names in
exclude_muting_conditions.
2017-09-20 13:02:22 -07:00
Steve Howell 428d3027c2 Only require ids for finding DefaultStream objects.
We don't need full Realm objects to find DefaultStream
objects for a realm.  So now a few functions related to
adding/removing default streams use realm_id for lookups.

Similarly, we don't need a full Stream object to find
out if a stream exists in DefaultStream, so we do id
lookups there as well.

This sets us up to use thinner objects in callers.
2017-09-20 10:31:33 -07:00
Steve Howell fd58d472a5 Use update_fields in do_deactivate_stream.
We are generally explicit about which fields we save.
2017-09-20 10:31:33 -07:00
Steve Howell f13cb94d84 Have exclude_topic_mutes() accept a stream id.
We want to convert stream names to stream ids as close
to the "edges" of our system as possible, so we let our
caller do the work of finding the stream id for a stream
narrow.
2017-09-20 10:31:33 -07:00
Steve Howell 7b2340decd Extract stream_name_in_use().
Checking to see if a stream exists is more idiomatic
if we just use exists() from Django.  We encapsulate it
for case insensitivity purposes.
2017-09-20 10:31:33 -07:00
Steve Howell 9046efb71a Use `prereg.users.set()` in do_invite_users.
This is a bit more idiomatic for many-to-many relationships.
2017-09-20 10:31:33 -07:00
Steve Howell 8ad7133351 Cache active_user_ids() more directly.
We now have a dedicated cache for active_user_ids() that only
stores a list of user_ids.

Before this commit, active_user_ids() used a cache of UserProfile
dictionaries, so it incurred unnecessary deserialization costs for
all the user fields that it sliced away in a list comprehension.

Because the cache is skinnier here, we also need to invalidate it
less frequently.  Basically, all we care about is new users, realm
deactivations, and user deactivations.

It's hard to measure how much this will improve performance, because
the speedup for any operation here is pretty minor, but we use this
function a lot, so hopefully it will make the overall system more
healthy.
2017-09-20 10:31:33 -07:00
Steve Howell cad3a35b6a Only require realm_id for active_user_ids().
This is mostly a preparatory commit for an upcoming optimization
related to stream data, but it probably does save us an
occasional DB hop to the realm table.
2017-09-20 10:31:33 -07:00
Steve Howell 26735eeeac Only require realm_id for get_active_user_dicts_in_realm().
This is a preparatory commit that will eventually allow us
to avoid fetching realm info that we don't need, in other
parts of the codebase.
2017-09-20 10:31:33 -07:00
Steve Howell 0966bf1a48 Simplify get_stream_cache_key().
Before this commit, we could pass in either a Realm object
or a realm_id to get_stream_cache_key().  Now we consistently
pass it a realm_id.
2017-09-20 10:31:33 -07:00
Steve Howell 0e24e6bdfa Add test_num_queries_with_streams() for home page. 2017-09-20 10:31:33 -07:00
Steve Howell 9773750375 tests: Count queries for home page. 2017-09-20 10:31:33 -07:00
Harshit Bansal 122167148a notifications: Fix the misalignment of emojis with the message text.
Fixes: #6579.
2017-09-20 10:30:35 -07:00
Greg Price 0c7dbd2e8a message send: Cut is_active from the values query in get_recipient_info.
This is unused since the query started filtering on is_active=True, in
51d4f16fe "Ignore inactive users in get_recipient_info()."
2017-09-19 20:08:39 -07:00
kunall17 1436d558a3 APNs: Add message body to notification payload.
(Edited by greg to leave the badge logic as is for now.)
2017-09-18 16:37:02 -07:00
Tim Abbott b4aa1336e7 logging: Avoid logging certain common 404 errors to error logs.
This should help keep /var/log/zulip/errors.log relatively clean on
production Zulip servers.
2017-09-16 08:07:06 -07:00
invinciblycool 5475c5cedb tests: Replace test_user_agent_parsing with a normal test.
Previously, this was its own separate test script; now it's a normal
part of the test suite.

Tweaked by tabbott to use a proper test method.

Fixes #6327.
2017-09-16 03:56:18 -07:00
Aditya Bansal ddf2075228 Make migration 0032 use an old version of user_avatar_path.
This fixes upgrading from very old Zulip servers (e.g. 1.4.3) all the
way to current.

Fixes: #6516.
2017-09-16 03:27:02 -07:00
Tim Abbott a33c98f23e push_notifications: Require the message.triggers fields to exist.
The code was already assuming it implicitly, and making it explicit
improves the readability.
2017-09-16 03:14:56 -07:00
Steve Howell c4b17f2f80 Optimize get_recipient_info() with get_ids_for().
The get_ids_for() function produces a 2x speedup for
1000 users.
2017-09-16 03:07:13 -07:00
Tim Abbott a1ddd934b4 bots: Fix calculation of bot domain with REALMS_HAVE_SUBDOMAINS.
Previously, the bot domain was calculated correctly in most
circumstances, but if you were using the root domain, it would be
e.g. ".chat.zulip.org", not "chat.zulip.org".  We fix this, with
perhaps more use of setting REALMS_HAVE_SUBDOMAINS than would be ideal
if we weren't about to set that True unconditionally.
2017-09-16 02:49:15 -07:00
Tim Abbott 0d5badb71a urls: Remove robots.txt blocking Zulip serves from search engines.
This was added back in 2012 and seems to have been accidentally not
updated since then.
2017-09-16 01:59:40 -07:00
Tim Abbott e1498988d9 test_choose_realm: Hardcode REALMS_HAVE_SUBDOMAINS.
This is the only case that'll be important in the future, and this
is a nice checkpoint on the path to making REALMS_HAVE_SUBDMAINS=True.
2017-09-15 13:11:12 -07:00
Tim Abbott 064c8e39d6 backends: convert GitHub auth tests to consistently use zulip subdomain. 2017-09-15 13:09:02 -07:00
Tim Abbott f1133ab53e test_notifications: Fix HTML tests to assume REALMS_HAVE_SUBDOMAINS.
This is a step towards making REALMS_HAVE_SUBDOMAINS the default.
2017-09-15 12:28:38 -07:00
Steve Howell 84041d3195 Use itertools.groupby in bulk_get_subscriber_user_ids().
This results in about a 20% speedup by making more O(N)
things happen in C vs. Python.
2017-09-15 10:44:32 -07:00
Steve Howell 24b9f72b22 Use raw SQL in bulk_get_subscriber_user_ids().
This leads to more than a 2x speedup when tested with
20k+ total subscribers.  (For large realms with lots of default
streams, this function deals with LOTS of data, so it is important
to optimize.)
2017-09-15 10:44:32 -07:00
Steve Howell 1553dc00e0 Introduce StreamRecipient class.
This class encapsulates the mapping of stream ids to
recipient ids, and it is optimized for bulk use and
repeated use (i.e. it remembers values it already fetched).

This particular commit barely improves the performance
of gather_subscriptions_helper, but it sets us up for
further optimizations.

Long term, we may try to denormalize stream_id on to the
Subscriber table or otherwise modify the database so we
don't have to jump through hoops to do this kind of mapping.
This commit will help enable those changes, because we
isolate the mapping to this one new class.
2017-09-15 10:44:32 -07:00
Steve Howell fc2e485ca7 Sort emails in gather_subscriptions().
This helps makes the tests more deterministic.
2017-09-15 10:44:32 -07:00
Tim Abbott 15781dbc83 server_settings: Add email auth related features to data sent to clients.
This should make it possible for the mobile app to correctly allow
non-email addresses as usernames exactly when it makes sense to do so.
2017-09-15 10:30:20 -07:00
Supermanu ac79bbfe08 login: Enable non-email username to login. 2017-09-15 10:30:19 -07:00
Supermanu 5f41f3c3cb backends.py: Expose backends that require email usernames 2017-09-15 10:29:02 -07:00
Supermanu 28beddfd76 backends.py: Enable auth with any ldap attributes as username.
This commit enables user to authenticate with any attribute set in
AUTH_LDAP_USER_SEARCH given that LDAP_EMAIL_ATTR is set to an email
attributes in the ldap server. Thus email and username can be
completely unrelated.

With some tweaks by tabbott to squash in the documentation and make it
work on older servers.
2017-09-15 10:28:41 -07:00
Rishi Gupta b9c8747bd0 hotspots: Use tutorial_status to manage whether we show hotspots. 2017-09-15 04:14:52 -07:00
Rishi Gupta ac48772732 hotspots.js: Fix timeout millisecond conversion. 2017-09-15 04:14:52 -07:00
Rishi Gupta 21f076ac5d hotspots.py: Clean up SEND_ALL codepath.
Moves SEND_ALL to inside get_next_hotspots, since it is not something other
files should call.
Also changes the delay to 0s, and gates the code behind an
`if settings.DEVELOPMENT`.
2017-09-15 04:14:52 -07:00
Rishi Gupta a8deedbbb6 hotspots: Replace stream_settings with intro_streams and intro_topics. 2017-09-15 04:14:52 -07:00
Rishi Gupta 0677bd2a6d hotspots: Rename and update new_topic_button. 2017-09-15 04:14:52 -07:00
Rishi Gupta 4a383544af hotspots: Rename and update click_to_reply. 2017-09-15 04:14:52 -07:00
Steve Howell 51d4f16fe0 Ignore inactive users in get_recipient_info().
We were mostly excluding inactive users before this fix, but
now we completely ignore them.

This potentially changes some of the data we return from
get_recipient_info(), but the extra user ids before this fix
were effectively ignored by the caller.
2017-09-15 03:08:52 -07:00
Steve Howell 1759137e4f Don't queue feedback unless the bot is active.
The prior code would queue up feedback messages even if the
feedback bot was deactivated, which was just due to oversight
most likely.  (People probably rarely disable the feedback bot,
but they should have that option.)
2017-09-15 03:08:52 -07:00
Vishnu Ks 2993ae5433 models: Include create_time in Attachment to_dict method. 2017-09-15 01:27:28 -07:00
Vishnu Ks e34931971b models: Include size in Attachment to_dict method. 2017-09-15 01:27:28 -07:00
Vishnu Ks 22bab1c7ec events: Add upload_quota to initial state data. 2017-09-15 01:27:27 -07:00
Vishnu Ks 0ce62cd3e1 events: Add total_uploads_size to initial state data. 2017-09-15 01:27:20 -07:00
Tim Abbott f1a0f23d23 bugdown: Document the mention fetches code block. 2017-09-15 01:21:04 -07:00
Steve Howell 7159f34bed Avoid getting realm emoji unless there is emoji syntax. 2017-09-15 01:09:08 -07:00
Steve Howell 8a941d03ae Avoid querying streams unless there are stream links. 2017-09-15 01:09:08 -07:00
Steve Howell c3032a7fe8 Avoid looking up emails when rendering messages.
We now fetch email -> id mappings for messages ONLY if
it potentially uses the !avatar(foo@example.com) syntax.
2017-09-15 01:09:08 -07:00