Commit Graph

5895 Commits

Author SHA1 Message Date
Tim Abbott 71fc8363b4 tests: Add a test suite for maybe_enqueue_notifications.
This ensures that as we expand the logic for under what circumstances
email and push notifications should be sent, we can be confident about
this code path always doing the right thing.
2017-09-27 15:55:29 -07:00
Tim Abbott 830d3e8d07 event_queue: Use maybe_enqueue_notifications in missedmessage_hook.
This fixes a problem introduced in the recent refactoring where
`triggers` would not be set correctly when a push or email
notification was triggered by missedmessage_hook.

Fixes #6612.
2017-09-27 15:55:29 -07:00
Tim Abbott 9749826e9c event_queue: Refactor code for checking if a message was mentioned.
Now, the two code paths do the same thing for this check.

It seems like there may be more work to do here, in that
wildcard_mentioned messages seem to not be eligible for sending
email/push notifications.  We probably want to add some logic there
for the user doing the mention to control whether or not it does.
2017-09-27 15:55:29 -07:00
Tim Abbott f68089fbe1 event_queue: Extract maybe_enqueue_notifications.
Conceptually, this code path should be shared between the
process_message handler and the garbage-collection hook.
2017-09-27 15:55:29 -07:00
Tim Abbott c3911e3fb6 test_classes: Improve error handling for send_message. 2017-09-27 15:55:29 -07:00
rht 2949d1c1e8 zerver: Remove the rest of absolute_import. 2017-09-27 10:02:39 -07:00
rht 7115a29bee zerver/webhooks: Remove absolute_import. 2017-09-27 10:00:39 -07:00
rht e239e97351 zerver/management: Remove absolute_import. 2017-09-27 10:00:39 -07:00
rht 41d8db373a zerver/worker: Remove absolute_import. 2017-09-27 10:00:39 -07:00
rht 1e87a4b68c zerver/tests: Remove absolute_import. 2017-09-27 10:00:39 -07:00
rht 74fd3d9f31 zerver/tornado: Remove absolute_import. 2017-09-27 10:00:39 -07:00
rht f43e54d352 zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
rht 15ca13c8de zerver/views: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott f0f0176517 test_auth_backends: Clean up Google auth subdomains handling.
This makes GoogleSubdomainLoginTest consistently access subdomains the
standard way, replacing the original hacky approach it had that
predated the library.
2017-09-26 22:20:07 -07:00
Tim Abbott 1c25bb5eba auth: Fix typo in invalid subdomain logging. 2017-09-26 22:20:05 -07:00
Steve Howell b340b28055 Extract get_service_bot_events().
There are several reasons to extract this function:

    * It's easy to unit test without extensive mocking.
    * It will show up when we profile code.
    * It is something that you can mostly ignore for
      most messages.

The main reason to extract this, though, is that we are about
to do some fairly complex splicing of data for the use case
of mentioning service bots on streams they are not subscribed to,
and we want to localize the complexity.
2017-09-26 18:49:03 -07:00
Robert Hönig 43422fa6f2 outgoing webhook: Notify bot owner on failure response. 2017-09-26 16:11:27 -07:00
Robert Hönig c77b245944 backend tests: Add 'AARON' as bot owner of OUTGOING_WEBHOOK_BOT. 2017-09-26 16:11:27 -07:00
Eeshan Garg 385eb4ce99 integrations: Add hubot to "Interactive bots" category. 2017-09-26 16:00:41 -07:00
Vishnu Ks 7fd4a71b7f test_emails: Generate emails instead of hardcoding.
Tweaked by tabbott to use require_GET.

Fixes #6344.
2017-09-26 15:02:47 -07:00
Vishnu Ks f58c87917e emails: Log emails that are queued or sent in dev environment.
Tweaked by tabbott to add some comments and clarify the code.
2017-09-26 15:02:47 -07:00
Alena Volkova abe4196232 urls: Change the method for adding alert words from PUT to POST. 2017-09-26 14:00:51 -07:00
Alena Volkova ca687e01d7 urls: Remove the old POST endpoint for alert words. 2017-09-26 14:00:51 -07:00
neiljp (Neil Pilgrim) ab4cbf81f0 mypy: Add assertions of get_realm results when parser requires realm. 2017-09-26 13:49:01 -07:00
Greg Price 7737326cf0 mypy: Add an Optional in fix_unreads command. 2017-09-26 13:49:01 -07:00
Greg Price c10aca1bbf soft deactivation: Simplify logic slightly in management command.
The `get_realm` method works properly regardless of whether the
'realm_id' option was set, so checking for that option first is
redundant.
2017-09-26 13:49:01 -07:00
neiljp (Neil Pilgrim) acf7e2a9ee mypy: Set conditional to be against result of get_realm, not input.
[Tweaked by greg to use explicit `is not None`.]
2017-09-26 13:49:01 -07:00
Tim Abbott e5ea598fe3 test_email_change: Cover the case of email already existing.
This fixes master failing tests.
2017-09-26 11:16:12 -07:00
neiljp (Neil Pilgrim) ef1242bb62 mypy: Split email validation error handling in json_change_settings. 2017-09-25 16:02:56 -07:00
neiljp (Neil Pilgrim) 133d679feb mypy: Avoid Message.is_status_message if rendered_content is None. 2017-09-25 16:02:56 -07:00
Greg Price dcf8d2d9ac mention: Tiny tweak to get loop fusion.
It's unlikely to be of any real consequence, but this code bugged me
in that it makes a whole set before throwing it away to make nearly
the same set.

Sadly Python's comprehensions lack a way to write these cleanly as one
comprehension; but with no extra code complexity we can make the
temporary a genexp, which does the job.
2017-09-25 16:02:56 -07:00
neiljp (Neil Pilgrim) 4be2ada055 mypy: Amend mention.py to ensure return of Set[Text]. 2017-09-25 16:02:56 -07:00
neiljp (Neil Pilgrim) e6daee441b mypy: Adjust board_actions.py to allow Optional[Text] values. 2017-09-25 16:02:56 -07:00
Rishi Gupta 9cec8f08d1 hotspots: Turn on hotspots for new users.
We need a migration to clear the tutorial_status for existing users,
so that we don't show hotspots to anyone who signed up for Zulip in
the month or so since we deleted the old tutorial.
2017-09-25 15:47:34 -07:00
hollywoodno ee7d4808fe emails: Add log for outgoing emails.
Fixes #5900.
2017-09-25 15:37:09 -07:00
Tim Abbott 4de194a265 addressee: Add missing Realm import for mypy. 2017-09-25 14:21:01 -07:00
Tim Abbott c11b1623dd addressee: Accept a realm object in legacy_build.
This fixes a bug where the internal_prep_message code path would
incorrectly ignore the `realm` that was passed into it.  As a result,
attempts to send messages using the system bots with this code path
would crash.

As a sidenote, we really need to make our test system consistent with
production in terms of whether the user's realm is the same as the
system realm.
2017-09-25 14:06:29 -07:00
Tim Abbott cc6d35c0f2 address: Convert for_private to accept a realm.
Again, we don't need the sender object, just the realm the sender is
in.
2017-09-25 14:00:47 -07:00
Tim Abbott 14c1660a55 addressee: Pass realm into get_user_profiles.
We don't access any attributes of the sender other than the realm, and
as it turns out, we in some cases want to use a different realm than
the sender's.
2017-09-25 14:00:46 -07:00
Tim Abbott 8e2c91b09c actions: Use internal_send_private_message. 2017-09-25 13:59:04 -07:00
Tim Abbott 0d2d9fc30d test_signup: Add checks for messages sent on invited user signup.
Previously, we didn't have a test that verified that these notices
were sent.
2017-09-25 13:58:41 -07:00
Tim Abbott 0e1518bb67 auth: Disable coverage check for an auth error path.
This is just a temporary change to get tests passing again, though
it's possible we'll be deleting this code path before long.
2017-09-25 12:06:10 -07:00
Tim Abbott f010ed117b Migrate several Google auth tests to subdomains test class.
The plan is to have everything expect subdomains, so it makes sense to
move these tests to the subdomains-only test class and style.

Most of the remaining GoogleLoginTest tests are now either duplicates
or basic API-level tests where subdomains are irrelevant.
2017-09-25 07:29:18 -07:00
Tim Abbott c62befe655 test_messages: Fix invalid subdomain in mirrored message test. 2017-09-25 07:20:58 -07:00
Tim Abbott 4674af0894 outgoing_webhook: Fix broken way of accessing realm.uri.
Previously, this accessed realm.uri via trying to use
zulip_default_context.  That doesn't make any sense, because
zulip_default_context expects an HttpRequest object, and those are
nowhere in sight in the code path.  We do, however, have the outgoing
webhook bot user involved in the event, and that's the object to
access realm.uri from here.
2017-09-25 07:05:02 -07:00
Tim Abbott 5159151e5f test_signup: Remove unnecessary realm_name/realm_subdomain arguments.
These arguments are only intended to be used for realm creation, and
they make the code more confusing.

We need to make a few changes after doing this, because some tests
were relying on these extra arguments causing the form to not submit
for their error handling.

We don't apply these changes to the LDAP tests, since fixing those
seems complicated.
2017-09-25 06:45:56 -07:00
Tim Abbott 632180274a mypy: Add imports needed for new migration. 2017-09-25 06:45:56 -07:00
Tim Abbott 163da73d2d test_signup: Hardcode REALMS_HAVE_SUBDOMAINS for mirror dummy tests.
This also fixes the fact that this code path was randomly setting a
bunch of fields only used in realm creation in the registration form.
2017-09-25 06:14:38 -07:00
Tim Abbott 69ea571fcf zerver: Ensure single-realm servers have '' string_id. 2017-09-25 06:14:00 -07:00
Robert Hönig 51657cadf0 outgoing_webhook: De-bug send_response_message().
This removes the unnecessary forwarder_user_profile argument
and adds an error message in case the recipient type is invalid.
2017-09-25 06:00:42 -07:00
Robert Hönig 15a1bf2b58 outgoing webhook: Log all non-200 responses. 2017-09-25 06:00:42 -07:00
Vishnu Ks f3b4de822b command: Create multiuse user invite generation command. 2017-09-22 19:48:46 -07:00
Eeshan Garg 11b7d219bd integrations: Render bots' docs alongside integration docs.
This commit implements support for rendering static files in
under static/generated/bots/ in the same manner as we render
our webhooks/integration documentation. Said static files are
generated by tools/setup/generate_zulip_bots_static_files.py
during provisioning.
2017-09-22 15:19:20 -07:00
Eeshan Garg 390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Tim Abbott 84926ff321 slow_queries: Log slow queries even if not reported via Zulip.
This should help make these logs more useful for debugging on an
arbitary Zulip system.
2017-09-22 14:11:07 -07:00
Tim Abbott ec61a070b4 signups: Add basic logging for new signup processing. 2017-09-22 14:11:07 -07:00
Tim Abbott f78dd33037 invites: Add logging for newly sent invitation emails. 2017-09-22 14:11:07 -07:00
Tim Abbott 8badd10a6c send_test_email: Remind users to restart the server after updates. 2017-09-22 13:32:22 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
Tim Abbott de9f00208d test_notifications: Fix unnecessary use of dirname. 2017-09-22 10:17:54 -07:00
Tim Abbott 3d63f981a1 test_notifications: Clarify code for new MissedMessages test. 2017-09-22 10:16:24 -07:00
Harshit Bansal e4c1aa37f3 notifications: Add tests for `relative_to_full_url()` function.
Fixes: #6544.
2017-09-22 10:04:33 -07:00
Harshit Bansal e453586741 notifications.py: Add regex for matching relative avatar URLs. 2017-09-22 10:04:33 -07:00
Harshit Bansal 660ad5824b notifications.py: Improve the regex for matching relative URLs.
Fixes: #6593.
2017-09-22 10:04:33 -07:00
Harshit Bansal c9878139b9 notifications.py: Extract `relative_to_full_url()` as a global function. 2017-09-22 10:04:33 -07:00
Vishnu Ks b4fedaa765 backend: Add support for multiuse user invite link. 2017-09-22 07:56:53 -07:00
Vishnu Ks 68ccfe78e6 registration: Add optional streams argument to send_registration_completion_email. 2017-09-22 07:51:58 -07:00
Vishnu Ks 3cc9feb676 models: Create MultiuseInvite model. 2017-09-22 07:51:58 -07:00
Tim Abbott 1edd137263 RealmAuditLog: Pass acting_user to do_reactivate_user. 2017-09-22 07:33:02 -07:00
Tim Abbott 92b70f9895 RealmAuditLog: Fix missing mypy annotation. 2017-09-22 07:33:02 -07:00
Tim Abbott dc843efd56 RealmAuditLog: Add support for stringifying entries. 2017-09-22 07:09:37 -07:00
Rishi Gupta 6ec3595b77 emails: Change enqueue_welcome_emails to take a user rather than user_id. 2017-09-22 06:20:33 -07:00
Rishi Gupta a7c8770f97 emails: Move enqueue_welcome_emails outside of signups queue.
The only thing this queue should do is sign you up for the newsletter, since
it is only populated if newsletter_data is not None.
2017-09-22 06:20:33 -07:00
Rishi Gupta 88bec16452 hotspots: Only narrow to PMs on first load.
After first load, the browser should set tutorial-status to started,
so this should automatically only happen once.

Fixes #6575.
2017-09-22 05:01:53 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott 7554958316 test_signup: Add a test for invite emails being cleared. 2017-09-21 06:15:11 -07:00
Tim Abbott f706f657c0 signup: Fix invitation emails not being cleared properly.
Previously, invitation reminder emails were only being cleared after a
successful signup if newsletter_data was available, since that was the
circumstance in which we were calling the relevant queue processor
code.  Now, we (1) clear them when a human user finishes signing up
and (2) correctly clear them using the 'address' field of
ScheduleEmail, not user_id.
2017-09-21 06:15:11 -07:00
Tim Abbott 660d8d4bc4 ScheduledJob: Add a __str__ function to make them more easily printed. 2017-09-21 06:05:18 -07:00
Steve Howell 9110ff935e tests: Restore 100% coverage to views/messages.py.
A recent change moved some untested code into messages.py, which
requires 100% coverage.  Now we test bogus stream names in
exclude_muting_conditions.
2017-09-20 13:02:22 -07:00
Steve Howell 428d3027c2 Only require ids for finding DefaultStream objects.
We don't need full Realm objects to find DefaultStream
objects for a realm.  So now a few functions related to
adding/removing default streams use realm_id for lookups.

Similarly, we don't need a full Stream object to find
out if a stream exists in DefaultStream, so we do id
lookups there as well.

This sets us up to use thinner objects in callers.
2017-09-20 10:31:33 -07:00
Steve Howell fd58d472a5 Use update_fields in do_deactivate_stream.
We are generally explicit about which fields we save.
2017-09-20 10:31:33 -07:00
Steve Howell f13cb94d84 Have exclude_topic_mutes() accept a stream id.
We want to convert stream names to stream ids as close
to the "edges" of our system as possible, so we let our
caller do the work of finding the stream id for a stream
narrow.
2017-09-20 10:31:33 -07:00
Steve Howell 7b2340decd Extract stream_name_in_use().
Checking to see if a stream exists is more idiomatic
if we just use exists() from Django.  We encapsulate it
for case insensitivity purposes.
2017-09-20 10:31:33 -07:00
Steve Howell 9046efb71a Use `prereg.users.set()` in do_invite_users.
This is a bit more idiomatic for many-to-many relationships.
2017-09-20 10:31:33 -07:00
Steve Howell 8ad7133351 Cache active_user_ids() more directly.
We now have a dedicated cache for active_user_ids() that only
stores a list of user_ids.

Before this commit, active_user_ids() used a cache of UserProfile
dictionaries, so it incurred unnecessary deserialization costs for
all the user fields that it sliced away in a list comprehension.

Because the cache is skinnier here, we also need to invalidate it
less frequently.  Basically, all we care about is new users, realm
deactivations, and user deactivations.

It's hard to measure how much this will improve performance, because
the speedup for any operation here is pretty minor, but we use this
function a lot, so hopefully it will make the overall system more
healthy.
2017-09-20 10:31:33 -07:00
Steve Howell cad3a35b6a Only require realm_id for active_user_ids().
This is mostly a preparatory commit for an upcoming optimization
related to stream data, but it probably does save us an
occasional DB hop to the realm table.
2017-09-20 10:31:33 -07:00
Steve Howell 26735eeeac Only require realm_id for get_active_user_dicts_in_realm().
This is a preparatory commit that will eventually allow us
to avoid fetching realm info that we don't need, in other
parts of the codebase.
2017-09-20 10:31:33 -07:00
Steve Howell 0966bf1a48 Simplify get_stream_cache_key().
Before this commit, we could pass in either a Realm object
or a realm_id to get_stream_cache_key().  Now we consistently
pass it a realm_id.
2017-09-20 10:31:33 -07:00
Steve Howell 0e24e6bdfa Add test_num_queries_with_streams() for home page. 2017-09-20 10:31:33 -07:00
Steve Howell 9773750375 tests: Count queries for home page. 2017-09-20 10:31:33 -07:00
Harshit Bansal 122167148a notifications: Fix the misalignment of emojis with the message text.
Fixes: #6579.
2017-09-20 10:30:35 -07:00
Greg Price 0c7dbd2e8a message send: Cut is_active from the values query in get_recipient_info.
This is unused since the query started filtering on is_active=True, in
51d4f16fe "Ignore inactive users in get_recipient_info()."
2017-09-19 20:08:39 -07:00
kunall17 1436d558a3 APNs: Add message body to notification payload.
(Edited by greg to leave the badge logic as is for now.)
2017-09-18 16:37:02 -07:00
Tim Abbott b4aa1336e7 logging: Avoid logging certain common 404 errors to error logs.
This should help keep /var/log/zulip/errors.log relatively clean on
production Zulip servers.
2017-09-16 08:07:06 -07:00
invinciblycool 5475c5cedb tests: Replace test_user_agent_parsing with a normal test.
Previously, this was its own separate test script; now it's a normal
part of the test suite.

Tweaked by tabbott to use a proper test method.

Fixes #6327.
2017-09-16 03:56:18 -07:00
Aditya Bansal ddf2075228 Make migration 0032 use an old version of user_avatar_path.
This fixes upgrading from very old Zulip servers (e.g. 1.4.3) all the
way to current.

Fixes: #6516.
2017-09-16 03:27:02 -07:00
Tim Abbott a33c98f23e push_notifications: Require the message.triggers fields to exist.
The code was already assuming it implicitly, and making it explicit
improves the readability.
2017-09-16 03:14:56 -07:00
Steve Howell c4b17f2f80 Optimize get_recipient_info() with get_ids_for().
The get_ids_for() function produces a 2x speedup for
1000 users.
2017-09-16 03:07:13 -07:00
Tim Abbott a1ddd934b4 bots: Fix calculation of bot domain with REALMS_HAVE_SUBDOMAINS.
Previously, the bot domain was calculated correctly in most
circumstances, but if you were using the root domain, it would be
e.g. ".chat.zulip.org", not "chat.zulip.org".  We fix this, with
perhaps more use of setting REALMS_HAVE_SUBDOMAINS than would be ideal
if we weren't about to set that True unconditionally.
2017-09-16 02:49:15 -07:00
Tim Abbott 0d5badb71a urls: Remove robots.txt blocking Zulip serves from search engines.
This was added back in 2012 and seems to have been accidentally not
updated since then.
2017-09-16 01:59:40 -07:00