Commit Graph

2387 Commits

Author SHA1 Message Date
Steve Howell 1b518f1983 Return mentioned users in get_user_info_for_message_updates().
The dictionary result for get_user_info_for_message_updates()
now has a `mention_user_ids` field that is a set of user ids
who were mentioned in a message.
2017-09-27 16:01:50 -07:00
Steve Howell 646abb57b7 refactor: Extract get_user_info_for_message_updates.
We'll want to expand this to get users that were mentioned in
the prior message, but this commit is just a refactoring.
2017-09-27 16:01:50 -07:00
Tim Abbott 06e6b3abe1 event_queue: Fixed stream_push_notify code path in missedmessage_hook.
This finishes implementing the stream_push_notify feature, which
previously didn't work if you were not idle when the message was sent.
2017-09-27 15:57:48 -07:00
Tim Abbott 2b1ea1555c event_queue: Simplify API of maybe_enqueue_notifications. 2017-09-27 15:57:47 -07:00
Tim Abbott d6732d0781 event_queue: Fix handling of private messages in missedmessage_hook.
Previously, we didn't correctly trigger email/push notifications for
PMs in these cases.
2017-09-27 15:57:47 -07:00
Tim Abbott bfa8f0efdb tests: Add an end-to-end test for missedmessage_hook.
This checks what arguments it passes into the enqueuing function.

Note, however, that the arguments are wrong for various cases, we'll
update the tests as we fix those bugs.
2017-09-27 15:57:47 -07:00
Tim Abbott 71fc8363b4 tests: Add a test suite for maybe_enqueue_notifications.
This ensures that as we expand the logic for under what circumstances
email and push notifications should be sent, we can be confident about
this code path always doing the right thing.
2017-09-27 15:55:29 -07:00
rht 1e87a4b68c zerver/tests: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott f0f0176517 test_auth_backends: Clean up Google auth subdomains handling.
This makes GoogleSubdomainLoginTest consistently access subdomains the
standard way, replacing the original hacky approach it had that
predated the library.
2017-09-26 22:20:07 -07:00
Steve Howell b340b28055 Extract get_service_bot_events().
There are several reasons to extract this function:

    * It's easy to unit test without extensive mocking.
    * It will show up when we profile code.
    * It is something that you can mostly ignore for
      most messages.

The main reason to extract this, though, is that we are about
to do some fairly complex splicing of data for the use case
of mentioning service bots on streams they are not subscribed to,
and we want to localize the complexity.
2017-09-26 18:49:03 -07:00
Robert Hönig 43422fa6f2 outgoing webhook: Notify bot owner on failure response. 2017-09-26 16:11:27 -07:00
Vishnu Ks 7fd4a71b7f test_emails: Generate emails instead of hardcoding.
Tweaked by tabbott to use require_GET.

Fixes #6344.
2017-09-26 15:02:47 -07:00
Vishnu Ks f58c87917e emails: Log emails that are queued or sent in dev environment.
Tweaked by tabbott to add some comments and clarify the code.
2017-09-26 15:02:47 -07:00
Alena Volkova abe4196232 urls: Change the method for adding alert words from PUT to POST. 2017-09-26 14:00:51 -07:00
Alena Volkova ca687e01d7 urls: Remove the old POST endpoint for alert words. 2017-09-26 14:00:51 -07:00
Tim Abbott e5ea598fe3 test_email_change: Cover the case of email already existing.
This fixes master failing tests.
2017-09-26 11:16:12 -07:00
Tim Abbott 0d2d9fc30d test_signup: Add checks for messages sent on invited user signup.
Previously, we didn't have a test that verified that these notices
were sent.
2017-09-25 13:58:41 -07:00
Tim Abbott f010ed117b Migrate several Google auth tests to subdomains test class.
The plan is to have everything expect subdomains, so it makes sense to
move these tests to the subdomains-only test class and style.

Most of the remaining GoogleLoginTest tests are now either duplicates
or basic API-level tests where subdomains are irrelevant.
2017-09-25 07:29:18 -07:00
Tim Abbott c62befe655 test_messages: Fix invalid subdomain in mirrored message test. 2017-09-25 07:20:58 -07:00
Tim Abbott 4674af0894 outgoing_webhook: Fix broken way of accessing realm.uri.
Previously, this accessed realm.uri via trying to use
zulip_default_context.  That doesn't make any sense, because
zulip_default_context expects an HttpRequest object, and those are
nowhere in sight in the code path.  We do, however, have the outgoing
webhook bot user involved in the event, and that's the object to
access realm.uri from here.
2017-09-25 07:05:02 -07:00
Tim Abbott 5159151e5f test_signup: Remove unnecessary realm_name/realm_subdomain arguments.
These arguments are only intended to be used for realm creation, and
they make the code more confusing.

We need to make a few changes after doing this, because some tests
were relying on these extra arguments causing the form to not submit
for their error handling.

We don't apply these changes to the LDAP tests, since fixing those
seems complicated.
2017-09-25 06:45:56 -07:00
Tim Abbott 163da73d2d test_signup: Hardcode REALMS_HAVE_SUBDOMAINS for mirror dummy tests.
This also fixes the fact that this code path was randomly setting a
bunch of fields only used in realm creation in the registration form.
2017-09-25 06:14:38 -07:00
Robert Hönig 15a1bf2b58 outgoing webhook: Log all non-200 responses. 2017-09-25 06:00:42 -07:00
Eeshan Garg 11b7d219bd integrations: Render bots' docs alongside integration docs.
This commit implements support for rendering static files in
under static/generated/bots/ in the same manner as we render
our webhooks/integration documentation. Said static files are
generated by tools/setup/generate_zulip_bots_static_files.py
during provisioning.
2017-09-22 15:19:20 -07:00
Eeshan Garg 390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Tim Abbott de9f00208d test_notifications: Fix unnecessary use of dirname. 2017-09-22 10:17:54 -07:00
Tim Abbott 3d63f981a1 test_notifications: Clarify code for new MissedMessages test. 2017-09-22 10:16:24 -07:00
Harshit Bansal e4c1aa37f3 notifications: Add tests for `relative_to_full_url()` function.
Fixes: #6544.
2017-09-22 10:04:33 -07:00
Vishnu Ks b4fedaa765 backend: Add support for multiuse user invite link. 2017-09-22 07:56:53 -07:00
Tim Abbott dc843efd56 RealmAuditLog: Add support for stringifying entries. 2017-09-22 07:09:37 -07:00
Rishi Gupta 6ec3595b77 emails: Change enqueue_welcome_emails to take a user rather than user_id. 2017-09-22 06:20:33 -07:00
Rishi Gupta a7c8770f97 emails: Move enqueue_welcome_emails outside of signups queue.
The only thing this queue should do is sign you up for the newsletter, since
it is only populated if newsletter_data is not None.
2017-09-22 06:20:33 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott 7554958316 test_signup: Add a test for invite emails being cleared. 2017-09-21 06:15:11 -07:00
Tim Abbott f706f657c0 signup: Fix invitation emails not being cleared properly.
Previously, invitation reminder emails were only being cleared after a
successful signup if newsletter_data was available, since that was the
circumstance in which we were calling the relevant queue processor
code.  Now, we (1) clear them when a human user finishes signing up
and (2) correctly clear them using the 'address' field of
ScheduleEmail, not user_id.
2017-09-21 06:15:11 -07:00
Steve Howell 9110ff935e tests: Restore 100% coverage to views/messages.py.
A recent change moved some untested code into messages.py, which
requires 100% coverage.  Now we test bogus stream names in
exclude_muting_conditions.
2017-09-20 13:02:22 -07:00
Steve Howell 428d3027c2 Only require ids for finding DefaultStream objects.
We don't need full Realm objects to find DefaultStream
objects for a realm.  So now a few functions related to
adding/removing default streams use realm_id for lookups.

Similarly, we don't need a full Stream object to find
out if a stream exists in DefaultStream, so we do id
lookups there as well.

This sets us up to use thinner objects in callers.
2017-09-20 10:31:33 -07:00
Steve Howell 8ad7133351 Cache active_user_ids() more directly.
We now have a dedicated cache for active_user_ids() that only
stores a list of user_ids.

Before this commit, active_user_ids() used a cache of UserProfile
dictionaries, so it incurred unnecessary deserialization costs for
all the user fields that it sliced away in a list comprehension.

Because the cache is skinnier here, we also need to invalidate it
less frequently.  Basically, all we care about is new users, realm
deactivations, and user deactivations.

It's hard to measure how much this will improve performance, because
the speedup for any operation here is pretty minor, but we use this
function a lot, so hopefully it will make the overall system more
healthy.
2017-09-20 10:31:33 -07:00
Steve Howell cad3a35b6a Only require realm_id for active_user_ids().
This is mostly a preparatory commit for an upcoming optimization
related to stream data, but it probably does save us an
occasional DB hop to the realm table.
2017-09-20 10:31:33 -07:00
Steve Howell 0966bf1a48 Simplify get_stream_cache_key().
Before this commit, we could pass in either a Realm object
or a realm_id to get_stream_cache_key().  Now we consistently
pass it a realm_id.
2017-09-20 10:31:33 -07:00
Steve Howell 0e24e6bdfa Add test_num_queries_with_streams() for home page. 2017-09-20 10:31:33 -07:00
Steve Howell 9773750375 tests: Count queries for home page. 2017-09-20 10:31:33 -07:00
Harshit Bansal 122167148a notifications: Fix the misalignment of emojis with the message text.
Fixes: #6579.
2017-09-20 10:30:35 -07:00
kunall17 1436d558a3 APNs: Add message body to notification payload.
(Edited by greg to leave the badge logic as is for now.)
2017-09-18 16:37:02 -07:00
invinciblycool 5475c5cedb tests: Replace test_user_agent_parsing with a normal test.
Previously, this was its own separate test script; now it's a normal
part of the test suite.

Tweaked by tabbott to use a proper test method.

Fixes #6327.
2017-09-16 03:56:18 -07:00
Tim Abbott a1ddd934b4 bots: Fix calculation of bot domain with REALMS_HAVE_SUBDOMAINS.
Previously, the bot domain was calculated correctly in most
circumstances, but if you were using the root domain, it would be
e.g. ".chat.zulip.org", not "chat.zulip.org".  We fix this, with
perhaps more use of setting REALMS_HAVE_SUBDOMAINS than would be ideal
if we weren't about to set that True unconditionally.
2017-09-16 02:49:15 -07:00
Tim Abbott 0d5badb71a urls: Remove robots.txt blocking Zulip serves from search engines.
This was added back in 2012 and seems to have been accidentally not
updated since then.
2017-09-16 01:59:40 -07:00
Tim Abbott e1498988d9 test_choose_realm: Hardcode REALMS_HAVE_SUBDOMAINS.
This is the only case that'll be important in the future, and this
is a nice checkpoint on the path to making REALMS_HAVE_SUBDMAINS=True.
2017-09-15 13:11:12 -07:00
Tim Abbott 064c8e39d6 backends: convert GitHub auth tests to consistently use zulip subdomain. 2017-09-15 13:09:02 -07:00
Tim Abbott f1133ab53e test_notifications: Fix HTML tests to assume REALMS_HAVE_SUBDOMAINS.
This is a step towards making REALMS_HAVE_SUBDOMAINS the default.
2017-09-15 12:28:38 -07:00
Steve Howell 1553dc00e0 Introduce StreamRecipient class.
This class encapsulates the mapping of stream ids to
recipient ids, and it is optimized for bulk use and
repeated use (i.e. it remembers values it already fetched).

This particular commit barely improves the performance
of gather_subscriptions_helper, but it sets us up for
further optimizations.

Long term, we may try to denormalize stream_id on to the
Subscriber table or otherwise modify the database so we
don't have to jump through hoops to do this kind of mapping.
This commit will help enable those changes, because we
isolate the mapping to this one new class.
2017-09-15 10:44:32 -07:00
Tim Abbott 15781dbc83 server_settings: Add email auth related features to data sent to clients.
This should make it possible for the mobile app to correctly allow
non-email addresses as usernames exactly when it makes sense to do so.
2017-09-15 10:30:20 -07:00
Supermanu 5f41f3c3cb backends.py: Expose backends that require email usernames 2017-09-15 10:29:02 -07:00
Supermanu 28beddfd76 backends.py: Enable auth with any ldap attributes as username.
This commit enables user to authenticate with any attribute set in
AUTH_LDAP_USER_SEARCH given that LDAP_EMAIL_ATTR is set to an email
attributes in the ldap server. Thus email and username can be
completely unrelated.

With some tweaks by tabbott to squash in the documentation and make it
work on older servers.
2017-09-15 10:28:41 -07:00
Rishi Gupta b9c8747bd0 hotspots: Use tutorial_status to manage whether we show hotspots. 2017-09-15 04:14:52 -07:00
Rishi Gupta ac48772732 hotspots.js: Fix timeout millisecond conversion. 2017-09-15 04:14:52 -07:00
Rishi Gupta a8deedbbb6 hotspots: Replace stream_settings with intro_streams and intro_topics. 2017-09-15 04:14:52 -07:00
Rishi Gupta 0677bd2a6d hotspots: Rename and update new_topic_button. 2017-09-15 04:14:52 -07:00
Rishi Gupta 4a383544af hotspots: Rename and update click_to_reply. 2017-09-15 04:14:52 -07:00
Vishnu Ks 22bab1c7ec events: Add upload_quota to initial state data. 2017-09-15 01:27:27 -07:00
Vishnu Ks 0ce62cd3e1 events: Add total_uploads_size to initial state data. 2017-09-15 01:27:20 -07:00
Steve Howell 7159f34bed Avoid getting realm emoji unless there is emoji syntax. 2017-09-15 01:09:08 -07:00
Steve Howell 8a941d03ae Avoid querying streams unless there are stream links. 2017-09-15 01:09:08 -07:00
Steve Howell c3032a7fe8 Avoid looking up emails when rendering messages.
We now fetch email -> id mappings for messages ONLY if
it potentially uses the !avatar(foo@example.com) syntax.
2017-09-15 01:09:08 -07:00
Steve Howell 4e7fce60ee Add possible_mentions() to speed up rendering.
We now triage message content for possible mentions before
going to the cache/DB to get name info.  This will create an
extra data hop for messages with mentions, but it will save
a fairly expensive cache lookup for most messages.  (This will
be especially helpful for large realms.)

[Note that we need a subsequent commit to actually make the speedup
happen here, since avatars also cause us to look up all users in
the realm.]
2017-09-15 01:09:08 -07:00
Tim Abbott 5722237f59 push: Rename received_pm to private_message.
This is a clearer name for this now more broadly used interface.
2017-09-14 05:41:37 -07:00
Sarah 97571a203d push: Add new formatting for stream message push and add tests.
This should make the push notifications for messages to streams with
the new stream push notifications setting enabled make sense.
2017-09-14 05:41:37 -07:00
Sarah c3a8138f74 user_settings: Add push notifications for all stream messages.
Add setting to enable push notifications for all stream messages.
2017-09-14 05:41:37 -07:00
Steve Howell d723be125a Optimize get_recipient_info() for sending messages.
This commit makes get_recipient_info() faster by never creating
Django ORM objects.  We use the ORM to create a values query
instead, and then we iterate over the rows to create various
collections of ids.

In order to avoid lots of code duplication, this commit unifies
how we query UserProfile for PMs and streams.  Prior to this
commit we were getting "wide" UserProfile objects out of
our memcached cache.  Now we just go to the database with our
list of userids.  The new approach at worst adds one hop to the
database for PMs, which aren't really a performance bottleneck
(compared to streams).  And the new approach actually saves a
hop when both partners aren't in cache (plus we don't pay the
penalty of hitting the cache itself).

The performance improvement here is easy to measure for messages
to streams with many users, even with all the other activity
that goes on inside do_send_messages().  I took test_performance()
in test_messages.py, set num_extra_users to 3000, and consistently
measured a ~20% speedup in do_send_messages().

This commit also eliminates fetching of emails.  We probably
could have done that in a prior commit, but in this commit it
is very explicit that we don't need it.  While removing email
from the query is a no-brainer, it actually had a negigible
impact on performance.  Almost all the savings here comes from
not create UserProfile objects.
2017-09-12 04:22:55 -07:00
Steve Howell 82b2bd8b65 Take user_ids in get_userids_for_missed_messages().
This helps us phase out the need for getting lots of UserProfile
objects.
2017-09-12 04:22:55 -07:00
Steve Howell ba397b5109 Use user_ids, not full objects, in render path.
There is no reason for either render_incoming_message() or
render_markdown() to require full UserProfile objects just to
triage alert words.

By only asking for user_ids, we save extra queries in two
callpaths and we make it easier to start using user_ids in
do_send_messages().
2017-09-12 04:22:55 -07:00
Steve Howell 262abe41ab Add a performance test for do_send_messages(). 2017-09-12 04:22:55 -07:00
Steve Howell b6bb7f2b1e Fix bug where we hard code realm for PushDeviceToken.
This had no test coverage, which is part of the reason it went
undetected, plus many instances probably only have one realm
with realm_id=1.
2017-09-08 12:32:17 -07:00
Steve Howell 69203c1c81 fix_unreads: Remove commit() call in fix().
The commit() call in fix() breaks migrations and tests (unless you
mock) due to outer transactions.

We now explicitly call commit() from the management command.
2017-09-07 07:06:03 -07:00
Steve Howell a2fe4178be Extract zerver/lib/fix_unreads.py.
This is a pure code move.
2017-09-07 07:06:03 -07:00
Steve Howell 848c0803bd Exclude muted topics from unread count. 2017-09-07 07:06:03 -07:00
Steve Howell f5edeb01ae Calculate idle users more efficiently when sending messages.
Usually a small minority of users are eligible to receive missed
message emails or mobile notifications.

We now filter users first before hitting UserPresence to find idle
users.  We also simply check for the existence of recent activity
rather than borrowing the more complicated data structures that we
use for the buddy list.
2017-09-07 06:59:44 -07:00
Steve Howell 4ac6bc46c7 Add MutedTopic model.
This commit completely switches us over to using a
dedicated model called MutedTopic to track which topics
a user has muted.

This includes the necessary migrations to create the
table and populate it from legacy data in UserProfile.

A subsequent commit will actually remove the old field
in UserProfile.
2017-09-02 09:19:51 -07:00
Steve Howell 06ca364049 minor: Test round-trip behavior for mutes.
Instead of peeking directly at the DB to verify our mutes are
set correctly, we now use the library function.  This prepares
us to modify the DB internals while preserving the tests.
2017-08-30 09:14:41 -07:00
Brock Whittaker 2140a4aa01 landing: Add /plans/ describing ways to use Zulip.
Note from tabbott: This isn't yet linked to and will need to go
through significantly more iteration, but it's a start.
2017-08-30 07:56:22 -07:00
Greg Price a4bcf1a64b APNs: Handle HTTP connection errors, and retry.
Should help with #6321 as at least a band-aid.
2017-08-29 15:27:41 -07:00
Greg Price 780e1ac5b2 push notifs: Add a simple test for the new APNs provider. 2017-08-29 15:27:41 -07:00
Steve Howell 0501570cd1 Remove POST-based API for setting topic mutes. 2017-08-29 16:53:38 -04:00
Steve Howell 0959c978c3 Fix lint error from recent subdomains commit.
We did a code sweep recently for subdomains (see
60be89d0).
2017-08-29 08:35:37 -07:00
Tim Abbott 60be89d00e test_push_notifications: Declare subdomains explicitly. 2017-08-28 23:19:07 -07:00
Tim Abbott 4a22316d90 test_decorator: Add explicit subdomains in tests. 2017-08-28 22:51:57 -07:00
Rishi Gupta c1997e759c password_reset: Change email to be appropriate for obtaining first password.
The situation if, for instance, the user signed up via google auth, and now
needs a password to get their API key.
2017-08-28 20:39:53 -07:00
Tim Abbott 50f5560bd1 accounts: Standardize URL for find_account.
This changes it to match the /accounts/ URL style for all of our other
auth code path endpoints.
2017-08-28 14:36:59 -07:00
Tim Abbott ac0d90e533 portico: Rename 'find_my_team' to 'find_account'. 2017-08-28 14:29:29 -07:00
Tim Abbott a0a1fe1512 settings: Rename SERVER_URI to ROOT_DOMAIN_URI.
This should be a lot less confusing.

See #6013 for discussion.
2017-08-28 14:09:28 -07:00
Umair Khan ecfafc05c0 registration: Use already_registered to show error.
Use this new variable to determine if the user already exists while
doing registration. While doing login through GitHub if we press
*Go back to login*, we pass email using email variable. As a result,
the login page starts showing the "User already exists error" if we
don't change the variable.
2017-08-28 07:02:11 -07:00
Steve Howell 73c30774cb admins: Add private streams to never_subscribed.
Admins need to know about private streams to delete them, even
if they are not subscribed.  We send the minimal info possible
to the client to allow them to have a UI for that.
2017-08-27 19:08:04 -07:00
Steve Howell 8ea9b80a8c Clean up test_never_subscribed_streams().
This basically extracts a few helper methods and makes the data
setup a bit more explicit.
2017-08-27 19:08:04 -07:00
Steve Howell 313f73258d Allow admins to delete private streams (backend only).
This is the backend piece.  Getting the UI right here is a bit
more complicated here, but this allows admins to use the API
to delete streams.
2017-08-27 19:08:04 -07:00
Preston Hansen 5a501784f2 digest emails: Add unit tests for digest email management.
Fixes #6266.
2017-08-27 13:10:14 -07:00
Vishnu Ks dc63f838d7 backend-tests: Add tests for get_users with all-users argument enabled. 2017-08-27 12:34:23 -07:00
Aditya Bansal 9d7e23c100 softdeactivation/management: Make specifying realm an optional arg. 2017-08-27 11:33:06 -07:00
Tim Abbott 1c8c5cc36f test_messages: Fix deactivation tests for new /me behavior. 2017-08-27 09:58:02 -07:00
Tim Abbott 92efe94a27 tests: Remove unnecessary apns mock. 2017-08-26 15:00:08 -07:00
Tim Abbott f0637cb01a push_notifications: Fix one last lint error. 2017-08-26 14:34:17 -07:00
Greg Price 613d093d7d push notifs: Implement APNs with new API.
And it works!

A couple of things still to do:

 * When a device token is no longer active, we'll get HTTP status 410.
   We should then remove the token from the database so we don't keep
   trying to push to it.  This is fairly urgent.

 * The library we're using has a nice asynchronous API, but this
   version doesn't use it.  This is OK now, but async will be
   essential at scale.
2017-08-26 14:16:05 -07:00
Greg Price d02101a401 APNs: Rip out the existing, broken implementation.
This code empirically doesn't work.  It's not entirely clear why, even
having done quite a bit of debugging; partly because the code is quite
convoluted, and because it shows the symptoms of people making changes
over time without really understanding how it was supposed to work.

Moreover, this code targets an old version of the APNs provider API.
Apple deprecated that in 2015, in favor of a shiny new one which uses
HTTP/2 to meet the same needs for concurrency and scale that the old
one had to do a bunch of ad-hoc protocol design for.

So, rip this code out.  We'll build a pathway to the new API from
scratch; it's not that complicated.
2017-08-26 14:16:05 -07:00
Tim Abbott f51f89a86e test_upload: Explicitly declare subdomains. 2017-08-26 13:45:27 -07:00
Tim Abbott d18f389bb5 test_messages: Set subdomain explicitly in most tests. 2017-08-26 13:45:27 -07:00
Tim Abbott f48bad2734 test_signup: Remove now-unnecessary mocking of subdomain. 2017-08-26 13:45:27 -07:00
Tim Abbott 9c85ccf70f test_signup: Use HostRequestMock.
PostRequestMock is less useful and going to break soon.
2017-08-26 13:45:27 -07:00
rht 437b2a3146 Update "MacOS" text to "macOS" 2017-08-26 09:00:42 -07:00
Tim Abbott 20c0d27317 test_narrow: Specify subdomains explicitly. 2017-08-25 16:09:52 -07:00
Tim Abbott edc095db6d test_subs: Explicitly declare subdomains. 2017-08-25 16:09:51 -07:00
Tim Abbott 6caca80ab7 test_presence: Specify subdomains for Zephyr tests. 2017-08-25 16:08:37 -07:00
Tim Abbott 7a5eb9dd9f tornado: Add support for testing with subdomains. 2017-08-25 15:42:58 -07:00
Tim Abbott acdeaca2a8 test_zephyr: Explictly declare subdomain in requests. 2017-08-25 15:42:50 -07:00
Tim Abbott 43cfe73908 tests: Make requests use the "zulip" subdomain by default.
Previously, we didn't pass customized HTTP_HOST headers when making
network requests.  As we move towards a world where everything is on a
subdomain, we'll want to start doing that.

The vast majority of our test code is written to interact with the
default "zulip" realm, which has a subdomain of "zulip".  While
probably longer-term, we'll wish this was the root domain, for now, we
need to make our HTTP requests match what is expected by the test
code.

This commit almost certainly introduces some weird bugs where code was
expecting a different subdomain but the tests doesn't fail yet.  It's
not clear how to find all of these, but I've done some grepping.
2017-08-25 15:42:49 -07:00
Tim Abbott ba64d4ee29 server_settings: Add additional subdomains test case.
This will help preserve 100% test coverage as we refactor to set
REALMS_HAVE_SUBDOMAINS=True unconditonally.
2017-08-25 15:42:43 -07:00
Tim Abbott 64e895533b test_docs: Add tests for different subdomain states.
This helps maintain 100% test coverage on the new test_classes code.
2017-08-25 15:42:43 -07:00
Tim Abbott 4229faf36e tests: Remove old six.PY2 code paths. 2017-08-25 11:05:16 -07:00
Tim Abbott b8e7369dee mypy: Remove type: ignores not needed in Python 3. 2017-08-25 11:04:20 -07:00
Umair Khan 9720d3a986 accounts_register: Use redirect_to_deactivation_notice. 2017-08-25 00:09:06 -07:00
Umair Khan 43c40f6144 Add tests for deactivation notice. 2017-08-25 00:09:06 -07:00
Tim Abbott 7d08ff69f0 tests: Remove most references to get_api_key.
This test helper doesn't really have value.
2017-08-24 23:30:46 -07:00
Tim Abbott e2ae096035 test_signup: Fix coverage of remaining find_my_team code. 2017-08-24 23:30:46 -07:00
Tim Abbott 59aae22f99 registration: Fix find_my_team handling of unusual users. 2017-08-24 23:17:08 -07:00
Tim Abbott ca7075a0a8 test_signup: Clean up find_my_team test suite. 2017-08-24 23:17:08 -07:00
Tim Abbott eeabed9119 models: Add new get_user_profile_by_api_key helper.
This results in a slight performance increase.
2017-08-24 23:17:08 -07:00
Tim Abbott 2aab6e0f49 forms: Replace is_inactive with more comprehensive check.
While we're at it, we clean up the old confusing error messages.
2017-08-24 23:16:31 -07:00
Tim Abbott 5fa7c3e0b2 test_classes: Clean up API for unsubscribing.
The old API required test authors to unnecessarily repeat themselves.
2017-08-24 21:37:57 -07:00
Tim Abbott eb720485c5 tests: Add and use new self.subscribe.
This new method cleans up the API for subscribing to something from a
test case.
2017-08-24 21:37:57 -07:00
Tim Abbott 60105fe38a registration: Remove accounts_home_with_realm_str.
This was the old mechanism for logging into a specific realm on a
server with more than one not using REALMS_HAVE_SUBDOMAINS.
2017-08-24 20:44:52 -07:00
Tim Abbott 80458945c7 tests: Remove unnecessary use of get_realm_by_email_domain.
That function was primarily intended for some limited registration
code paths, not the generic usage it was getting.
2017-08-24 20:22:56 -07:00
Aditya Bansal 6a2c83f051 notifications: Group messages by (recipient, sender) for PM's.
This fixes a issue with multiple PM's being clubbed into a single
missed message email.

Fixes #6224.
2017-08-24 19:58:55 -07:00
Tim Abbott 3d4893d906 settings: Rename SUBDOMAINS_HOMEPAGE to ROOT_DOMAIN_LANDING_PAGE.
This new setting name is a lot more readable.
2017-08-24 19:32:16 -07:00
Tim Abbott ab61fe5922 auth: Fix Google auth to not assume SUBDOMAINS_HOMEPAGE.
Previously, Google auth would redirect you back to the login page on
this code path.
2017-08-24 19:03:42 -07:00
Tim Abbott 8b0c0c33b5 login: Fix logging for login/register actions.
Previously, Zulip's server logs would not show which user or client
was involved in login or user registration actions, which made
debugging more annoying than it needed to be.
2017-08-24 16:29:27 -07:00
Tim Abbott 5206dad373 decorator: Fix is_browser_view handling of e.g. mobile apps.
We may end up with some endpoints that could be the browser or the
mobile apps, and we want the right default to happen here.
2017-08-24 16:26:57 -07:00
Tim Abbott 7ffc107061 decorator: Rename is_json_view to is_browser_view.
This improves clarity, now that we're planning to start logging in the
login/registration pages, which aren't browser views.
2017-08-24 16:25:08 -07:00
Vishnu Ks f9a6dffe1a management: Add add_user_list_args function to ZulipBaseCommand. 2017-08-24 14:29:32 -07:00
Steve Howell 87c4961597 Add zerver/lib/topic_mutes.py
This is mostly pure code extraction.

It also removes some dead code in update_muted_topic, where
were updating muted_topics spuriously before calling
do_update_muted_topic.
2017-08-24 14:20:35 -07:00
Tim Abbott f46f3871b4 integrations: Add support for using root subdomain.
This fixes up the URL/link construction to make the root subdomain work.
2017-08-23 23:18:36 -07:00
Tim Abbott d44ac5c320 auth: Clean up google_oauth2_test arguments. 2017-08-23 20:50:03 -07:00
Tim Abbott d3e3c704d4 do_create_realm: Remove unnecessary second return value.
Unlike creating a stream, there's really no reason one would want to
call the function to create a realm while uncertain whether that realm
already existed.
2017-08-23 20:07:17 -07:00
Tim Abbott 8a65fdce49 subdomains: Fix over-aggressive redirect to find_my_team.
If the root domain is a valid subdomain, we don't want to redirect
everyone away from it.
2017-08-23 20:07:17 -07:00
Tim Abbott 397340b454 test_events: Fix a missing mypy import. 2017-08-23 20:04:20 -07:00
Vishnu Ks 8418f0d30e dev_login: List realms and show only users in the selected realm. 2017-08-23 19:27:32 -07:00
Tim Abbott 225debdbc5 test_events: Fix a misplaced type annotation. 2017-08-23 19:19:14 -07:00
Steve Howell f797604e66 Add test coverage for unreads count (stream muting). 2017-08-23 18:00:59 -07:00
Vishnu Ks 59790f37fc models: Create get_notifications_stream method in class Realm. 2017-08-23 17:50:34 -07:00
Steve Howell 2b4faaa847 Support non-search queries in /json/messages/matches_narrow.
For filters like has:link, where the web app doesn't necessarily
want to guess whether incoming messages meet the criteria of the
filter, the server is asked to query rows that match the query.

Usually these queries are search queries, which have fields for
content_matches and subject_matches.  Our logic was handling those
correctly.

Non-search queries were throwing an exception related to tuple
unpacking.  Now we recognize when those fields are absent and
do the proper thing.

There are probably situations where the web app should stop hitting
this endpoint and just use its own filters.  We are making the most
defensive fix first.

Fixes #6118
2017-08-23 01:07:57 -07:00
Tim Abbott 06f8fd529d realm: Add clear error message for too-long realm name.
This fixes an exception we saw in production, as well as fixing the
form to not offer ultra-long names that we don't allow.
2017-08-22 22:04:52 -07:00
Tim Abbott 7c37fc8909 auth: Improve robustness of not sending duplicate login emails.
Previously, we had a very not-robust check on the URL, which might
have caused cases like Google auth registration to not do the right
thing.
2017-08-22 16:26:30 -07:00
Umair Khan a5c05f9812 push_notification: Retry event in case of error.
Fixes #5301
2017-08-22 11:16:48 -07:00
Steve Howell 81e3f489f2 Use sender realm in user_profiles_from_unvalidated_emails.
This change is mostly based on a similar commit from hackerkid
in a feature branch.  It borrows both code and ideas.  Some of
it's my own stuff, as I was working on a newer branch.

We now call get_user_including_cross_realm_email() inside of
user_profiles_from_unvalidated_emails(), instead of using
get_user_profile_by_email.

This requires a few of our callers to pass down sender into us.

One consequence of this change is that we change the symptoms
for trying to send to emails outside of your realm.  In some
cases, we simply raise an error that an email is invalid to us
instead of getting into the deeper validate_recipient_user_profiles
check.
2017-08-22 10:42:15 -07:00
Steve Howell 54edecd510 Replace adddressee.for_email() with for_user_profile().
This requires us to change not just the immediate caller, but
also some of their callers, to pass user_profile objects around
instead of emails.
2017-08-22 10:42:15 -07:00
Steve Howell 30d37d1270 Add Addressee class.
This class simplifies the calling sequence to methods like
check_message and _internal_prep_message, and it's also more
type safe.

Checking for message types is encapsulated with calls to is_stream()
and is_private().  There are also shortcut constructors when you
know that the type of the address (stream vs. private), which is often.
2017-08-22 10:42:15 -07:00
Tim Abbott 58edf75962 api: Allow incoming webhook bots to use the send_message API. 2017-08-22 10:06:34 -07:00
Steve Howell e12da79732 create_realm: Redirect to a nice error page for SMTP errors.
Fixes #3850
2017-08-21 15:39:26 -07:00
Steve Howell 99e08fe49a accounts_home: Redirect to nice error page for SMTP errors. 2017-08-21 15:39:26 -07:00
Steve Howell ac80d1df0a Add /config-error/smtp endpoint for SMTP errors. 2017-08-21 15:39:26 -07:00
Aditya Bansal 7a2667ce53 soft_deactivation: Fix flaky tests by seeding message for users.
In this we basically seed a single message for the user which will
be soft deactivated by sending a stream message / group PM to
ensure that is has at least one UserMessage row, since in real
world every human user will always have at least one User Message
row.
2017-08-18 07:02:11 -04:00
Tim Abbott ef722c5688 tests: Remove unused ujson imports. 2017-08-17 09:05:52 -07:00
Umair Khan 6e87108d46 result.json: Upgrade test_users. 2017-08-17 09:03:35 -07:00
Umair Khan 212571ab86 result.json: Upgrade test_subs. 2017-08-17 09:03:35 -07:00
Umair Khan eb04fc006e result.json: Upgrade test_settings. 2017-08-17 09:03:35 -07:00
Umair Khan d44c1e7070 result.json: Upgrade test_realm_emoji. 2017-08-17 09:03:35 -07:00
Umair Khan 7b85e40ef5 result.json: Upgrade test_realm_domains. 2017-08-17 09:03:35 -07:00
Umair Khan 90099ffeb2 result.json: Upgrade test_reactions. 2017-08-17 09:03:35 -07:00
Umair Khan 351426df8b result.json: Upgrade test_presence. 2017-08-17 09:03:35 -07:00
Umair Khan 4c8933b844 result.json: Upgrade test_narrow. 2017-08-17 09:03:35 -07:00
Umair Khan 7bb9ac0014 result.json: Upgrade test_messages. 2017-08-17 09:03:35 -07:00
Umair Khan 025f7247cf result.json: Upgrade test_external. 2017-08-17 09:03:35 -07:00
Umair Khan 2b7bb6e965 result.json: Upgrade test_events. 2017-08-17 09:03:35 -07:00
Umair Khan de5f43ea47 result.json: Upgrade test_bots. 2017-08-17 09:03:35 -07:00
Umair Khan 1c09f95e0e result.json: Upgrade test_auth_backends. 2017-08-17 09:03:35 -07:00
Steve Howell 47bcedbc1b Fix server searches for is:mentioned and is:alerted.
Before this change, server searches for both
`is:mentioned` and `is:alerted` would return all messages
where the user is specifically mentioned (but not
at-all mentions).

Now we follow the JS semantics:

    is:mentioned -- all mentions, including wildcards
    is:alerted  -- has an alert word

Here is one relevant JS snippet:

        } else if (operand === 'mentioned') {
            return message.mentioned;
        } else if (operand === 'alerted') {
            return message.alerted;

And here you see that `mentioned` is OR'ed over both mention flags:

    message.mentioned = convert_flag('mentioned') || convert_flag('wildcard_mentioned');

The `alerted` flag on the JS side is a simple mapping:

    message.alerted = convert_flag('has_alert_word');

Fixes #5020
2017-08-16 11:29:12 -07:00
Tim Abbott 8600efcf36 test_bugdown: Fix duplicated test name. 2017-08-16 11:29:12 -07:00
Tim Abbott d2288154f6 bugdown: Fully remove mentions matching @name.
Given typeahed and the fact that this only worked if the person had a
full name that didn't contain whitespace, this side effect of the
original @shortname mentionfeature that we removed was experienced by
users as a bug.

Fixes #6142.
2017-08-16 11:18:09 -07:00
Vaida Plankyte 52046d537a auth.py: Add config_error page for misconfigured github/google auth.
Significantly modified by tabbott to use a better system, pass tests,
and clean up the content.
2017-08-16 10:05:19 -07:00
Tim Abbott 46b5c942aa test_narrow: Fix remaining case of search tests flaking with "lunch".
We apparently were using the default of num_before=1, not
num_before=0, which meant that if the very last randomly generated
message was one by cordelia mentioning lunch,
test_get_messages_with_search would fail because there were actually 3
matches.
2017-08-16 07:37:06 -07:00
Umair Khan 3afa8ad6a6 test_users: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan d051bfbcb4 test_urls: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan d3ced34152 test_uploads: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan b9785cd4d6 test_subs: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan b14603249f test_realm_filters: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan bab6893dbb test_realm_emoji: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan ed12419061 test_reactions: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan 02aab6a43f test_presence: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan 70996de415 test_messages: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan 0e119c0ec2 test_bugdown: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan 2b6c6d823d test_bots: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan 366d2927c3 test_attachments: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Umair Khan 64c961f5de test_alert_words: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Tim Abbott 7c4eb43f34 test_notifications: Fix reply-to email for test suite.
It turns out that moving NOREPLY_EMAIL_ADDRESS DEFAULT_SETTINGS
unexpectedly changed its value in the test suite.
2017-08-15 18:33:15 -07:00
Brock Whittaker 83184d5c60 /about/: Merge with /authors, redesign page.
This adds the authors to the Zulip repository on GitHub from
/authors/ along with re-styling the page to fit the same
aesthetic as /for/open-source/ and other product-pages.
2017-08-15 16:09:01 -07:00
Tim Abbott 353a61b1d8 test_signup: Clean up re-fetching of user objects. 2017-08-15 12:52:23 -07:00
Jack Zhang 182570d5f3 Deduplicate header markup for portico and portico-help templates. 2017-08-15 12:15:21 -07:00
Tim Abbott 5a63ea0170 decorator: Extract validate_account_and_subdomain and deduplicate.
This fixes the significant duplication of code between the
authenticate_log_and_execute_json code path and the `validate_api_key`
code path.

These's till a bit of duplication, in the form of `process_client` and
`request._email` interactions, but it is very minor at this point.
2017-08-15 10:54:23 -07:00
Tim Abbott 25ec30dcf1 decorator: Deduplicate validate_api_key by using new helper function.
This should eliminate a bunch of duplicated code between these two
code paths.
2017-08-15 10:54:23 -07:00
Tim Abbott 3c37795675 decorator: Add localhost/Tornado hack to access_user_by_api_key.
This should make it possible to share that code with other code paths.
2017-08-15 10:54:23 -07:00
Tim Abbott 842bf77efb unread: Convert mark_topic_as_read to use stream IDs.
This is safer in its handling of potentially renamed streams.
2017-08-15 10:40:02 -07:00
Tim Abbott f3a41ac796 unread: Convert mark_stream_as_read to use stream IDs.
The logic is simpler and more robust.
2017-08-15 10:40:02 -07:00
Steve Howell 89f9017686 Create new endpoints for marking streams/topics as read.
The new endpoints are:
    /json/mark_stream_as_read: takes stream name
    /json/mark_topic_as_read: takes stream name, topic name

The /json/flags endpoint no longer allows streams or topics
to be passed in as parameters.
2017-08-15 10:17:29 -07:00
Steve Howell 60cc8fd58a Extract do_mark_stream_messages_as_read.
This function optimizes marking streams and topics as read,
by using UserMessage.where_unread(), which uses a partial
index on the "read" flag.

This also simplifies the code path for ordinary message
flag updates.

In order to keep 100% line coverage, I simplified the
logging in update_message_flags, so now all requests
will show the "actually" format.

This is an interim step toward creating dedicated endpoints
for marking streams/topics as reads, so we do error checking
with asserts for flag/operation, so we don't introduce a
temporary translation string.
2017-08-15 10:09:10 -07:00
Tim Abbott 9081f2cf44 reactions: Store the emoji codepoint in the database.
This is the first part of a larger migration to convert Zulip's
reactions storage to something based on the codepoint, not the emoji
name that the user typed in, so that we don't need to worry about
changes in the names we're using breaking the emoji storage.
2017-08-15 09:29:27 -07:00
Tim Abbott 3409806b88 test_narrow: Fix nondeterministic test failures with "lunch".
We recently changed the populate_db data set to include more variable
message content, which happened to include the possibility of the word
"lunch" appearing in the test messages.  This caused occasional
failures of the search tests that looked for messages containing
"lunch" starting at the beginning of time, not the beginning of the
test.
2017-08-15 09:20:45 -07:00
Aditya Bansal 25d9c7eea1 soft_deactivation: Add helper functions.
This commits adds new helper functions which are:
* get_users_for_soft_deactivation(): This function can be used to
fetch a list of human users which pass the criteria of minimum
inactivity period (in days) passed as a parameter to the function.
* do_soft_activate_users(): Given a list of users this function
reactivates them and help them catch up with the missing message
rows for them in the UserMessage table.
2017-08-15 08:33:16 -07:00
Aditya Bansal 0cb909b978 events: Fill in missing messages for a returing soft_deactivated user. 2017-08-15 08:33:16 -07:00
Aditya Bansal 658a14d0bb soft_deactivation: Stop creating UserMessage rows when soft deactivated. 2017-08-15 08:33:16 -07:00
Aditya Bansal 57858596ab soft_deactivation: Add new function 'add_missing_messages'.
This function will help us in creating undisturbed experience for
returning soft deactivated users.

Tweaked by tabbott to fix minor performance and clarity issues.
2017-08-15 08:29:51 -07:00
Aditya Bansal dc5510ce44 soft_deactivation: Add helper functions for deactivating users. 2017-08-15 08:29:28 -07:00
Steve Howell 274938bf5f Eliminate approximate_unread_count().
We use page_params.unread_msgs.count in the web
app instead.

Fixes #1300.
2017-08-14 12:39:00 -07:00
Jack Zhang 1e6570fa5d Update integrations doc context relative urls. 2017-08-10 10:21:53 -07:00
Jack Zhang 63ad7b6769 Rename /integrations/doc API endpoint as /integrations/doc-html. 2017-08-10 10:21:53 -07:00
Steve Howell c7b9044ee5 Fix apply_unread_message_event() for mentions.
We were exiting this function in certain cases before updating
mentions. This bug was always there, but it was flaky in terms
of database setup whether the tests would fail, so now the
relevant test sends three consecutive messages.

We also avoid putting duplicate message ids in mentions.
2017-08-10 05:09:04 -04:00
neiljp (Neil Pilgrim) 357c9ed016 mypy: Pass empty dict to makeRecord for args, instead of None. 2017-08-09 18:04:21 -07:00
Tim Abbott 1d10941712 register: Don't display field to enter password unless needed.
This should significantly improve the user experience for new users
signing up with GitHub/Google auth.  It comes complete with tests for
the various cases.  Further work may be needed for LDAP to not prompt
for a password, however.

Fixes #886.
2017-08-09 13:44:57 -07:00
Umair Khan 2c9a57161a registration: Add test for password less remote reg. 2017-08-09 13:44:57 -07:00
Umair Khan 4a3fde023f google: Respect is_signup argument.
This allows us to go to Registration form directly. This behaviour is
similar to what we follow in GitHub oAuth. Before this, in registration
flow if an account was not found, user was asked if they wanted to go to
registration flow. This confirmation behavior is followed for login
oauth path.
2017-08-09 13:44:57 -07:00
Umair Khan bb0eb76bf3 github: Don't ask for password in registration. 2017-08-09 13:44:57 -07:00
Vishnu Ks 6b46762384 management: Fix typo in get_realm CommandError message. 2017-08-08 14:13:19 -07:00
neiljp (Neil Pilgrim) b782db48e1 mypy: Remove superfluous older 'type: ignore' annotations. 2017-08-08 11:27:51 -07:00
neiljp (Neil Pilgrim) 67f1a72aae mypy: Ensure events test fails only due to settings absence. 2017-08-07 21:27:50 -07:00
neiljp (Neil Pilgrim) 334348cbca mypy: Use namedtuples for request objects in test_messages.py. 2017-08-05 17:53:18 -07:00
Tim Abbott fc0144f6e4 test_decorators: Annotate fake Request as HttpRequest. 2017-08-05 17:53:18 -07:00
neiljp (Neil Pilgrim) 6717f361ec mypy: Amend tornado_redirected_to_list calls to use Mapping. 2017-08-05 12:08:59 -07:00
Tim Abbott dd49bec93c Fix changing email addresses back after email change.
We apparently were not correctly clearing the user_profile's email
address from caches when changing email addresses, which meant that
trying to look up the old email in the user_profile caches would still
work.

Fixes #6035.
2017-08-05 10:49:44 -07:00
Rishi Gupta 251bd3e577 registration.py: Remove first_in_realm from accounts_register.
The parent commit implies first_in_realm == realm_creation.
2017-08-05 10:17:14 -07:00
Steve Howell 63f7b9a579 Remove "all" option for flag-updating endpoint.
The "all" option for 'message/flags' was dangerous, as it could
apply to any of our flags.  The only flag it made sense for, the
"read" flag, now has a dedicated endpoint.
2017-08-04 14:10:46 -07:00
Steve Howell 541156792e Add /mark_all_as_read endpoint.
This change simplifies how we mark all messages as read.  It also
speeds up the backend by taking advantage of our partial index
for unread messages.  We also use a new statsd indicator.
2017-08-04 14:10:46 -07:00
Umair Khan b2a2664715 rate_limiter: Upgrade clear_user_history to generic API.
Changes the name of the function to change_history as well.
2017-08-02 18:01:39 -07:00
Vaida Plankyte 02d65a67af frontend: Edit confirm_continue_registration.html to be clearer.
Fixes #5707.
2017-08-02 08:51:25 -07:00
Rishi Gupta 082c3a3c93 portico: Add for-working-groups-and-communities.md. 2017-08-02 00:19:34 -07:00
Rishi Gupta 94f7a5704f portico: Add for/companies. 2017-08-02 00:19:34 -07:00
Tim Abbott 32a5a54ba5 test_docs: Fix test for the redesigned /features page. 2017-08-02 00:09:32 -07:00
Rishi Gupta 852bbfa16a tutorial: Remove tutorial_finale.
Was already orphaned, since skip is always true. This is just removing the
code.
2017-08-01 22:38:22 -07:00
Rishi Gupta a51d8d4251 realm creation: Change initial realm messages and streams. 2017-08-01 22:38:22 -07:00
Tim Abbott a9fa1a5527 api: Migrate /json/messages_in_narrow off legacy API.
This completes the major endpoint migrations to eliminate legacy API
endpoints from Zulip.

There's a few other things that will happen naturally, so I believe
this fixes #611.
2017-07-31 13:08:06 -07:00
Tim Abbott ed10ce70d4 invites: Migrate to non-legacy API format.
This has the side effect of making it possible for a bot user to
invite users with its API key, which seems useful.
2017-07-31 13:08:06 -07:00
Tim Abbott 6a50e13156 uploads: Remove legacy /json/upload_file endpoint.
This migrates Zulip to use the equivalent API endpoint that has been
present for a while.
2017-07-31 13:08:06 -07:00
Tim Abbott 1e5aee054b settings: Migrate main settings-change code to API.
This was one of the few major remaining endpoints that were still on
the old-style legacy API.
2017-07-31 13:08:06 -07:00
Tim Abbott 9bf02fb913 custom_profile: Prevent bot users from accessing endpoint. 2017-07-31 13:08:06 -07:00
Tim Abbott 93261e14b7 hotspots: Prevent bot users from accessing endpoint. 2017-07-31 13:08:06 -07:00
Anna Liao 6a3c8ca2c5 push_notifications: Add human users only decorator.
This is part of #4517.
2017-07-31 13:08:06 -07:00
Tim Abbott 47eb7ee003 push_notifications: Fix incorrect URL being used in bouncer test.
This was actually testing the validation in the direct push
notifications code path, not the push notification boucner code path.
2017-07-31 13:08:06 -07:00
Tim Abbott 4b7c23f133 markdown: Rename bugdown-data.json to a more reasonable name. 2017-07-28 17:54:01 -07:00
Jack Zhang 11c27015f0 hotspots: Refactor backend for hotspots v2. 2017-07-28 16:34:13 -07:00
vaibhav 691aff55a3 bots: Add UI to select Slack compatible interface for webhooks.
interface_type select menu will be used to choose the interface
for outgoing webhooks. It will be displayed only when the selected
bot type is OUTGOING WEBHOOK type. The default value is GENERIC
interface type (1).
2017-07-28 16:22:55 -07:00
Tim Abbott 7778a66171 test_messages: Add a test for unauthorized mirroring of PMs. 2017-07-28 15:55:34 -07:00
Tim Abbott 8d0b743a6f test_messages: Add a test for PMs to deactivated users. 2017-07-28 15:53:33 -07:00
Tim Abbott a9e987f31d test_messages: Add a test for private messages copying self. 2017-07-28 15:49:27 -07:00
Aditya Bansal 452bbd9104 test_helpers: Extract stub for request_event_queue & get_user_events. 2017-07-28 14:50:55 -07:00
Steve Howell e6e3bbb780 Add a "mentions" section to unread message ids. 2017-07-27 16:14:26 -07:00
Jason Michalski 4f0110e081 Add unread_msgs to the initial state data.
We are adding a new list of unread message ids grouped by
conversation to the queue registration result. This will allow
clients to show accurate unread badges without needing to load an
unbound number of historic messages.

Jason started this commit, and then Steve Howell finished it.

We only identify conversations using stream_id/user_id info;
we may need a subsequent version that includes things like
stream names and user emails/names for API clients that don't
have data structures to map ids -> attributes.
2017-07-27 16:14:25 -07:00
Abhijeet Kaur 1deb58b178 bots: Add complete test-coverage for bot_lib.py file.
Also, add error handling for get_bot_handler instead of
throwing an assertion error.
2017-07-27 15:50:29 -07:00
Steve Howell 69fd967c5d topics history: Simplify data in /topics endpoint.
In anticipation of have all unread message ids available to the
web app in page_params (via a separate effort), we are simplifying
the /topics endpoint to no longer return unread counts.

Instead we have a list of tiny dictionaries with these fields:

    name - name of the topic
    max_id - max message id for the topic (aka most recent)

The items in the list are order by most-recent-topic-first.
2017-07-27 14:26:22 -07:00
Harshit Bansal 10265dbbd9 test_reactions: Migrate tests to use default test realm emoji. 2017-07-27 18:59:02 +00:00
Harshit Bansal 543cb11d17 test_realm_emoji: Migrate tests to use default test realm emoji. 2017-07-27 18:59:02 +00:00
Harshit Bansal 8fa861a020 test_realm_emoji: Add test.
Add a test for checking that non-admins can upload a custom emoji when
`add_emoji_by_admins_only` is set to False.
2017-07-27 18:59:02 +00:00
Harshit Bansal b683af2cbb test_realm_emoji: Fix some typos. 2017-07-27 18:59:02 +00:00
Harshit Bansal 4737363f0c test_bugdown: Migrate tests to use the default test realm emoji. 2017-07-27 18:59:02 +00:00
Harshit Bansal f34d4e8f19 test_notifications: Migrate tests to use the default test realm emoji. 2017-07-27 18:59:02 +00:00
Harshit Bansal eeb25b6b4b populate_db: Add a test realm emoji while populating databases.
Add `green_tick` realm emoji which can be used while performing tests.
2017-07-27 18:57:06 +00:00
Tim Abbott 3ff199c613 test_home: Fix test for /apps/ page. 2017-07-26 10:37:54 -07:00
Tim Abbott c9b2ef82c9 test_bugdown: Fix test for Twitter emoji.
Our work on the Twitter emoji added a new instance of our emoji syntax
between when we wrote 2d92f9dc0b and
rebase+merged it.
2017-07-26 10:20:25 -07:00
Tim Abbott 2e74dbf2d9 apps: Fix the documentation tests for /apps/. 2017-07-26 10:08:16 -07:00
digi0ps 2d92f9dc0b emoji: Change emoji image titles to use spaces instead of underscores.
Previously, the emoji images title display `🍼`.
This commit changes the title to display `baby bottle`.
2017-07-26 09:27:17 -07:00
Vishnu Ks 6551640c67 confirmation: Use ConfirmationKeyException in get_object_from_key.
Fixes #5739.
2017-07-25 18:44:30 -07:00
Rishi Gupta 817e574d86 emails: Change Zulip Missed Messages to sentence case. 2017-07-25 17:42:33 -07:00
Greg Price 11a7a17087 streams: Remove a redundant field in the `json_stream_exists` result.
This route is called only in `js/compose.js`, to handle autosubscribe.
That code doesn't check this "exists" field, because there's no need
-- the same information is already carried in whether the result was
success or failure.  So just eliminate it.

This makes the logic here a little simpler.  It also eliminates
another usage of the `data` parameter to `json_error`.  I have half a
mind to eliminate that parameter, in favor of making `JsonableError`
subclasses whenever there's structured data to include, in particular
to get the benefits of typing.  There are a couple of places where
that change isn't locally a clear win, but this is not one of them.
2017-07-25 15:33:11 -07:00
Greg Price 28b4234227 admin: Make an error about auth settings not mimic auth errors.
This error isn't saying that any kind of authentication or
authorization failed -- it's just a validation error like
any other validation error in the values the user is asking to
set.  The thought of authentication comes into it only because
the setting happens to be *about* authentication.

Fix the error to look like the other validation errors around it,
rather than give a 403 HTTP status code and a "reason" field that
mimics the "reason" fields in `api_fetch_api_key`.
2017-07-25 15:33:11 -07:00
Harshit Bansal a03ce01688 bugdown: Fix the rendering of emojis in twitter link previews.
Process the unicode emojis in twitter link previews and render them
properly. Before this we were not processing the unicode emojis in
twitter link previews and hence on the systems which don't have
fonts for displaying them they were rendered as blank boxes.

Fixes: #5427.
2017-07-24 17:32:05 -07:00
Harshit Bansal 127d1f995d test_bugdown.py: Add test for `url_to_a()`. 2017-07-24 17:31:08 -07:00
Harshit Bansal f2c04576bd models: Fix the URL validation code in `RealmFilter` model. 2017-07-24 17:31:08 -07:00
Abhijeet Kaur 6f60c65a65 embedded bots: Add tests for verification of embedded bot services.
Add test to check if the embedded bot service being used is in the
registry or not.
Add test to check if the bot being added to the registry has a valid
bot corresponding to it.
Move 'get_bot_handler' to 'zerver/lib/bot_lib.py' as it is an independent
function, not related to the 'EmbeddedBotWorker' class that it was
previously a part of.
2017-07-24 17:14:14 -07:00
Greg Price 9faa44af60 JsonableError: Optionally carry error codes and structured data.
This provides the main infrastructure for fixing #5598.  From here,
it's a matter of on the one hand upgrading exception handlers -- the
many except-blocks in the codebase that look for JsonableError -- to
look beyond the string `msg` and pass on the machine-readable full
error information to their various downstream recipients, and on the
other hand adjusting places where we raise errors to take advantage
of this mechanism to give the errors structured details.

In an ideal future, I think all exception handlers that look (or
should look) for a JsonableError would use its contents in structured
form, never mentioning `msg`; but the majority of error sites might
continue to just instantiate JsonableError with a string message.  The
latter is the simplest thing to do, and probably most error types will
never have code looking for them specifically.

Because the new API refactors the `to_json_error_msg` method which was
designed for subclasses to override, update the 4 subclasses that did
so to take full advantage of the new API instead.
2017-07-24 16:41:22 -07:00
Greg Price e18baff32c JsonableError: Rename message from `error` to `msg`.
The whole thing is an error, so "message" is a more apt word for the
error message specifically.  We abbreviate that as `msg` in the actual
HTTP responses and in the signatures of `json_error` and friends, so
do the same here.
2017-07-24 16:41:22 -07:00
Elliott Jin fcb889c5c0 outgoing webhooks: Remove process_failure interface method. 2017-07-24 14:10:14 -07:00
Elliott Jin fdc9294a6e outgoing webhooks: Don't call *_with_message if message is None. 2017-07-24 14:10:14 -07:00
Elliott Jin 3d815bcea9 outgoing webhooks: Return None instead of "" in process_success. 2017-07-24 14:10:14 -07:00
Elliott Jin a7713840d4 outgoing webhooks: Remove underscore from test class names. 2017-07-24 14:10:14 -07:00
Elliott Jin 6a61a8a431 outgoing webhooks: Consolidate interfaces into lib/outgoing_webhook.py 2017-07-24 14:10:14 -07:00
vaibhav 4c50c4fc76 Add outgoing webhook interface for Slack. 2017-07-24 14:10:14 -07:00
vaibhav e5be29bc3c Add tests for Generic Interface. 2017-07-24 14:10:14 -07:00
vaibhav ff63f0b1d3 Return response string instead of dict() from process_success, process_failure. 2017-07-24 14:10:14 -07:00
James Rowan bba90c5096 emails: Make missed message email come from 'Zulip Missed Messages.' 2017-07-21 14:24:03 -07:00
Jack Zhang e915321f89 registration: Remove organization type selection in realm creation. 2017-07-21 13:09:06 -07:00
Eeshan Garg 6fe3748826 decorator: Remove redundant tests for api_key_only_webhook_view.
We do not need to test the exception message being logged in every
test case where an exception is raised by a webhook function.
Testing it once should be enough; this makes the tests less
verbose.
2017-07-20 10:30:03 -07:00
Rishi Gupta 9782911aca portico: Add initial why zulip page. 2017-07-19 11:53:06 -07:00
Eeshan Garg a24ea56c53 decorator: Handle invalid JSON in api_key_only_webhook_view.
Exception logging within api_key_only_webhook_view fails when
ValueError is raised if the request.body passed to ujson.loads
isn't valid JSON. In this case, we now just convert the payload
to a string and log that. This allows us to inspect JSON payloads
that aren't being decoded properly.
2017-07-19 10:42:35 -07:00
Sarah eef9f15910 test_settings: Refactor to use UserProfile property_types framework. 2017-07-18 15:49:20 -07:00
Rishi Gupta 394f85eb76 confirmation: Change confirmation keys to have length 24. 2017-07-17 23:18:47 -07:00
Rishi Gupta 5d168c90f9 confirmation: Remove ConfirmationManager.
Also adds Confirmation.type, and cleans up the rest of Confirmation to look
more like the model definitions in zerver.

In the migration, all existing confirmations adopt the type
USER_REGISTRATION, to be conservative. In a few commits, different
confirmation types will have different validity periods, and
USER_REGISTRATION will have the shortest default.
2017-07-17 23:18:47 -07:00
Rishi Gupta 0f4b71b766 confirmation: Liberate get_link_for_object from ConfirmationManager. 2017-07-17 23:18:47 -07:00
Aditya Bansal f2d6194ae1 actions: Start logging subscription activities in RealmAuditLog. 2017-07-17 17:23:41 -07:00
Brock Whittaker 38440d6ba4 Change product page nav and hero styling. 2017-07-17 16:12:36 -07:00
Rishi Gupta ea8c1c3cad emails: Use ScheduledEmail.user in send_future_email. 2017-07-17 16:05:38 -07:00
Rishi Gupta aa845e7f60 models: Replace ScheduledJob with ScheduledEmail.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
2017-07-17 16:05:38 -07:00
Vishnu Ks f9245e26d9 Make user_id, realm_id mandatory in _get_unversioned_avatar_url. 2017-07-17 21:01:30 +00:00
Rishi Gupta 898269bbac email: Change send_email to raise exception on failure.
More in line with how we do error handling in the rest of Zulip.
2017-07-16 16:56:39 -07:00
Rishi Gupta f1aaf4cc33 emails: Remove the display_email function.
No longer needed, since this now only appears in build_email.
2017-07-16 16:56:39 -07:00
Rishi Gupta eacdb0b302 emails: Change welcome emails to use to_user_id. 2017-07-16 16:56:39 -07:00
Rishi Gupta b0d325b8c5 emails: Change send_future_email to accept a to_user_id.
Also changes digest emails to use a to_user_id instead of a to_email.
2017-07-16 16:56:39 -07:00
neiljp (Neil Pilgrim) 8611a2cafa tidying: Combine multiple 'import typing' lines onto one line. 2017-07-16 16:43:21 -07:00
Harshit Bansal 45bc220796 notifications: Fix relative stream links in missed message emails.
Fixes: #5310.
2017-07-16 20:51:08 +00:00
Durga Akhil Mundroy 146dfa6f0b org-permissions: Add allow_edit_history organiztion setting.
This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization.  It controls access through 2
key mechanisms:

* For long-ago edited messages, get_messages removes the edit history
  content from messages it sends to clients.

* For newly edited messages, clients are responsible for checking the
  setting and not saving the edit history data.  Since the webapp was
  the only client displaying it before this change, this just required
  some changes in message_events.js.

Significantly modified by tabbott to fix some logic bugs and add a
test.
2017-07-16 10:10:06 -07:00
James Rowan 21b5536353 emails: Update subject lines of emails. 2017-07-14 15:20:04 -07:00
Vaida Plankyte 28ea174ed9 backend: Implement high contrast mode display setting. 2017-07-14 14:53:24 -07:00
Steve Howell 475eb21a5e Revert commits related to client_message_id.
I pushed a bunch of commits that attempted to introduce
the concept of `client_message_id` into our server, as
part of cleaning up our codepaths related to messages you
sent (both for the locally echoed case and for the host
case).

When we deployed this, we had some strange failures involving
double-echoed messages and issues advancing the pointer that appeared
related to #5779.  We didn't get to the bottom of exactly why the PR
caused havoc, but I decided there was a cleaner approach, anyway.
2017-07-14 12:13:35 -07:00
Steve Howell 8fbb55df85 Introduce client_message_id on the server.
We are deprecating local_id/local_message_id on the Python server.
Instead of the server knowing about the client's implementation of
local id, with the message id = 9999.01 scheme, we just send the
server an opaque id to send back to us.

This commit changes the name from local_id -> client_message_id,
but it doesn't change the actual values passed yet.

The goal for client_key in future commits will be to:
    * Have it for all messages, not just locally rendered messages
    * Not have it overlap with server-side message ids.

The history behind local_id having numbers like 9999.01 is that
they are actually interim message ids and the numerical value is
used for rendering the message list when we do client-side rendering.
2017-07-13 23:42:27 -04:00
Jack Zhang 6e2f90c8c9 Improve /integrations page load time.
Prior to this commit, 7 megabytes of images (through 253 individual requests)
were heavily slowing down the initial load. With this commit, we load only the
logos (60 or so images).

Documentation and images for the individual integration sub-pages is requested
separately using the /integrations/doc/ endpoint, which returns HTML.
2017-07-13 20:37:09 -07:00
James Rowan 71bc40d829 emails: Update subject for invitation emails. 2017-07-13 18:47:21 -07:00
James Rowan 88b8d98d34 emails: Update subject for missed message emails. 2017-07-13 18:47:17 -07:00
James Rowan 92c2d93ea3 tests: Enforce new custom sender names and addresses. 2017-07-13 14:50:36 -07:00
Vishnu Ks 098569ccb4 test_classes: Use get_user in unsubscribe_from_stream. 2017-07-13 14:13:03 -07:00
Vishnu Ks 99e78cbead test_classes: Use get_user in get_streams function. 2017-07-13 14:13:03 -07:00
Vishnu Ks 466c3f92f9 test_subs: Replace iago@zulip.com with self.example_email('iago'). 2017-07-13 14:13:03 -07:00
Sarah db3539438c test_events: Simplify testing code for bool user settings.
Use bool_change if the user_display setting property_type is bool, so that no additional code needs to be added to test_events for new boolean user display settings.
2017-07-13 14:07:54 -07:00
Cory Lynch 8afdb06e2a search: Add test case for searching for a link.
Fixes #2038
2017-07-13 09:23:34 -07:00
Tim Abbott 6a914c83ef docs: Fix indentation of DocPageTest. 2017-07-12 16:28:38 -07:00
Aditya Bansal 6100d43339 test_subs/test_users: Fix breaking tests due to regressions. 2017-07-13 01:56:26 +05:30
Tim Abbott 5acc7506ac test_users: Fix GetProfileTest to test its original purpose.
This test accidentally got refactored to use some test suite library
functions that ended up somewhat defeating its purpose.
2017-07-12 13:03:38 -07:00
Vishnu Ks 63586480a5 test_subs: Replace hamlet@zulip.com with self.example_email('hamlet'). 2017-07-12 12:49:12 -07:00
Vishnu Ks 6e133cc77a test_subs: Eliminate redundant class variable realm.
The SubscriptionAPITest class variables `realm` and `test_realm` stores
the same information and are redundant. I have eliminated all occurances
of self.realm and replaced with self.test_realm.
2017-07-13 00:45:24 +05:30
Vishnu Ks ed0423784d test_realm_emoji: Replace get_user_profile_by_email.
Replace with get_user.
2017-07-13 00:45:24 +05:30
Vishnu Ks ba4ea7dd8a notifications.py: Replace get_user_profile_by_email.
Replace with get_user.
2017-07-13 00:45:24 +05:30
Aditya Bansal 84e53f960d pep8: Add compliance with rule E261 to tests/test_upload.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 1b5e3190e3 pep8: Add compliance with rule E261 to tests/test_subs.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 2643a40176 pep8: Add compliance with rule E261 to tests/test_signup.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 59b26e582a pep8: Add compliance with rule E261 to tests/test_realm.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 0970d29ec1 pep8: Add compliance with rule E261 to tests/test_narrow.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ce70ded488 pep8: Add compliance with rule E261 to tests/test_messages.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal 8c233198f8 pep8: Add compliance with rule E261 to tests/test_events.py. 2017-07-11 15:17:32 -07:00
Aditya Bansal ee97ea1356 pep8: Add compliance with rule E261 to tests/test_bugdown.py. 2017-07-11 15:17:32 -07:00
Harshit Bansal e926f72bea notifications: Fix broken rendering of realm emoji in missed message emails.
We were using relative URLs for realm emojis in missed message emails.
Since the email server is not same as the Zulip server and doesn't
have the realm emoji files, they were rendered as broken images. This
commit fixes them to use absolute URL.

Fixes: #5692.
2017-07-08 10:54:06 -07:00
Rishi Gupta b3654c7b7e confirmation: Remove vestiges of the MitUser table.
The MitUser table was removed in df525ad.

confirm_mituser.html could have been accessed through the last few lines of
confirmation/views.py:
    templates.insert(0, 'confirmation/confirm_%s.html'
                     % (obj._meta.model_name,))

The commit message on df525ad suggests there was another way
confirm_mituser.html could have been called, but I don't currently see
evidence for it in the code.
2017-07-07 18:53:00 -07:00
Rishi Gupta 8fed9eeb75 confirmation: Make host a required argument in get_link_for_object.
Removes some lines of test from test_email_change.py. The relevant code path
was never utilized by the code itself, just by the tests.
2017-07-07 18:53:00 -07:00
Vishnu Ks 8954c0c726 management: Use Realm.objects.get instead of get_realm.
The Realm.DoesNotExist exception will never be raised if we pass a
string_id as in that case zerver.models.get_realm function would be
used for fetching. realm.zerver.models.get_realm uses filter query
so the exception will not be raised even if there are no realm which
matches the query.

Tests added by tabbott.
2017-07-07 15:02:12 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Vishnu Ks 92e7255217 page_params: Include password min length and quality.
The change password form http://localhost:9991/#settings/your-account
don't have data-min-length and data-min-quality attributes. The
account_settings.handlebar which has the change password form is
rendered client side. So we have to pass the value of min length
and quality in page params to set the data-min-length and
data-min-quality attributes.
2017-07-07 14:48:06 -07:00
Tim Abbott 982667acf5 management: Add new library for specifying realms.
This new library is intended to make it easy for management commands
to access a realm or a user in a realm without having to duplicate any
of the annoying parsing/extraction code.
2017-07-07 13:57:57 -07:00
Tim Abbott 05c190a97c Fix new-login emails for iOS not identifying OS correctly.
Previously, we were only checking for common strings in iOS browsers,
not for the explicit string that we use with the Zulip app.
2017-07-07 13:34:25 -07:00
neiljp (Neil Pilgrim) f1a8148242 mypy: Set remote_username parameter to Optional[Text] based on a test. 2017-07-07 10:56:33 -07:00
neiljp (Neil Pilgrim) a84d7b8581 mypy: Specify type of 'user_profile = None' as Optional[UserProfile]. 2017-07-07 10:56:33 -07:00
neiljp (Neil Pilgrim) 0958298d88 mypy: Add missing type parameter in test_upload.py:create_user. 2017-07-07 10:56:33 -07:00
neiljp (Neil Pilgrim) db0b985e82 mypy: Use specific kwargs types. 2017-07-07 10:56:33 -07:00
Vishnu Ks 9067890120 urls: Use POST for zerver.views.user_settings.set_avatar_backend.
The set_avatar_backend is changed from PUT to POST becuase this
is not idempotent.
2017-07-07 10:31:43 -07:00
Vishnu Ks df5df76961 urls: Use POST for zerver.views.realm_emoji.upload_emoji.
The upload_emoji endpoint is changed from PUT to POST because
this endpoint is not idempotent.
2017-07-07 10:31:43 -07:00
Vishnu Ks cf2dec539f urls: Use POST for zerver.views.realm_icon.upload_icon.
This upload_icon endpoint is changed from PUT to POST becuase
this endpoint is not idempotent.
2017-07-07 10:31:43 -07:00
Tim Abbott 0606ba88df push_notifications: Verify that token types are valid.
We only have two types of push notification tokens, so we should
validate that in the bouncer code path.
2017-07-07 10:28:36 -07:00
Tim Abbott eaa33cce73 push_notifications: Validate format of APNS tokens.
This fixes a bug where we would previously not validate the format of
APNS tokens before writing them to the database, which could lead to
exceptions in the push notifications system if a buggy mobile app
submitted invalid format tokens.
2017-07-07 10:28:36 -07:00
Jack Zhang 37a80d5cc4 portico: Redesign and rewrite /integrations text headings. 2017-07-06 15:28:50 -07:00
Steve Howell 155f3d42c8 test: Fix test_i18n.test_json_error().
The refer_friend endpoint is about to be deprecated, so we
test with invite_users instead, and I tried to make the
dummy value returned by the mocked gettext function a little
more clear.
2017-07-06 11:30:55 -04:00
Umair Khan 638b32542d auth: Don't show deactivation notice to mirror dummies. 2017-07-05 23:50:53 -07:00
Rishi Gupta 1291ac9eff emails: Groundwork to personalize reply-to name of missed message emails. 2017-07-05 15:33:01 -07:00
James Rowan 388cef900e tests: Enforce that activation emails come from NOREPLY address. 2017-07-05 15:19:00 -07:00
Harshit Bansal 3c9705ce98 actions.py: Instead of deleting a realm emoji just deactivate it.
Instead of removing an emoji from the database, just mark them as
deactivated so that they can't be used further but can be rendered
properly in reactions and messages.

Fixes: #4750.
2017-07-05 13:00:15 -07:00
James Rowan d88e7308bf emails: Add a FromAddress class to control access to certain settings emails.
No change in behavior.

Also makes the first step towards converting all uses of
settings.ZULIP_ADMINISTRATOR and settings.NOREPLY_EMAIL_ADDRESS to
FromAddress.*.

Once everything is converted, it will be easier to ensure that future
development doesn't break backwards compatibility with the old style of
settings emails.
2017-07-04 14:25:01 -07:00
James Rowan 368bd66d8b emails: Refactor send_email functions to take both a sender name and address.
This will allow for customized senders for emails, e.g. 'Zulip Digest' for
digest emails and 'Zulip Missed Messages' for missed message emails.

Also:
* Converts the sender name to always be "Zulip", if the from_email used to
  be settings.NOREPLY_EMAIL_ADDRESS or settings.ZULIP_ADMINISTRATOR.

* Changes the default value of settings.NOREPLY_EMAIL_ADDRESS in the
  prod_setting_template to no longer have a display name. The only use of
  that display name was in the email pathway.
2017-07-04 14:25:01 -07:00
Rishi Gupta a8af0b6d91 email_mirror: Change missed message noreply to not have a display name.
It's better to see "noreply@..." when replying to a message that you can't
reply to than "Zulip".
2017-07-04 14:25:01 -07:00
Tim Abbott c2ee259987 settings: Simplify testing code for bool realm settings.
Now, when we add a new realm setting with the bool type, we don't need
to edit the tests at all.
2017-07-04 14:23:05 -07:00
Franziska von der Goltz 98847937f4 realm: add mandatory topic setting on org level.
Lets organizations require users to specify a topic the discussion.

Fixes: #5164.
2017-07-04 14:09:21 -07:00
Aditya Bansal caa8a3f7fc Make test_google_oauth2_token_failure() work without internet. 2017-07-04 12:45:27 -07:00
Tim Abbott 59d2eae946 signals: Fix browser reporting for new Electron logins.
Previously, we were checking for "Zulip" entries in User-Agent strings
at the very end of the list, not the beginning.  A Zulip User-Agent
should have priority, since it includes the User-Agent strings for a
lot of other browsers, and also it will always be the most specific
value.
2017-07-03 10:12:20 -07:00
Harshit Bansal 4cb8ac100a emoji: Fix the title and alt text for unicode emojis.
Unicode emojis when rendered should display canonical short name.
Similarly, the alt text should be of the format `:<short_name>:`.
For both of these we currently display the actual unicode symbol.
As some systems don't have the fonts necessary for displaying them
properly, they are rendered as empty square blocks. This commit also
ensures that the markup generated for emoji generated by canonical
name and by an unicode emoji is same.

Fixes: #5555.
2017-07-01 11:14:24 -07:00
Rishi Gupta b55dce197d actions.py: Clear ScheduledJobs of users in do_deactivate_{user,realm}. 2017-07-01 08:44:47 -07:00
Pweaver (Paul Weaver) 2c10e7c0db Change from and noreply emails to use EXTERNAL_HOST domain.
This eliminates another mandatory setting from what one needs to
configure to setup a Zulip server.
2017-07-01 08:09:25 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
vaibhav 7f9b4d3a94 Outgoing Webhook System: Add usage of Interfaces in DoRestCall. 2017-06-28 11:11:21 -04:00
Yago González 2f4091b38d api: Fix caps in JSON validation error message. 2017-06-27 08:33:56 -04:00
Umair Khan acf0aad4a2 signals.py: Bring coverage to 100%. 2017-06-26 16:14:14 -07:00
Umair Khan 6357737978 get_device_browser: Fix Opera detection.
Starting from version 15, Opera uses 'OPR/' in its user agent string.
We need to move Opera detection before Chrome detection because
Opera's user agent string satisfies Chrome conditions as well.
Ref: https://dev.opera.com/blog/opera-user-agent-strings-opera-15-and-beyond/
2017-06-26 16:14:14 -07:00
Umair Khan 14402b0b97 get_device_browser: Move Edge detection to the top.
The user agent strings of Edge satisfy Chrome condition as well.
Ref: https://msdn.microsoft.com/en-us/library/hh869301(v=vs.85).aspx
2017-06-26 16:14:14 -07:00
Umair Khan dc78a7888f login_or_register_remote_user: Send login emails for mobile.
Fixes #5389
2017-06-26 16:14:14 -07:00
Umair Khan 34a91be9a2 api_fetch_api_key: Send new login emails for mobile. 2017-06-26 16:14:14 -07:00
Umair Khan 7e43cd5624 get_device_browser: Fix browser string for mobile.
Now get_device_browser() returns 'Zulip' for 'ZulipAndroid' and
'ZulipMobile'.
2017-06-26 16:14:14 -07:00
Umair Khan b9c69004e4 google_oauth2_test: Pass headers in all requests.
Currently we only pass headers in the first client_get call but
sometimes the effective request which reaches the view is through
a later call to the client_get in this function. Due to which
headers are not passed.
2017-06-26 16:14:14 -07:00
Vishnu Ks 00668c7bbd registration.py: Send pms from welcome-bot on user creation. 2017-06-25 11:23:45 -04:00
Vishnu Ks 9b7585a436 bots: Make welcome-bot cross realm. 2017-06-25 11:23:45 -04:00
Greg Price c9e35c459a landing: Write up why open source projects should use Zulip
This isn't very slick, but it should get the main points down,
and it's past time we got something like this up.  Definitely
needs in the future another pass at the text, and also some images
(screenshots, etc.) and styling.
2017-06-22 16:26:50 -07:00
Sarah 205d523ab2 test_events: Refactor to use do_set_user_display_settings_test.
Remove individual setting tests and replace with
test_set_user_display_settings.
2017-06-22 08:00:56 -04:00
sinwar 8ff2e5a22b requirements: Upgrade moto to latest version.
Fixes #5454
2017-06-21 19:05:50 -04:00
James Rowan 19ffb84ac2 Make notification email for new login give browser and os.
notify_new_login.subject: include browser and os in subject line.
test_new_users: test for subject line to include browser/os.
2017-06-21 14:36:41 -04:00
Umair Khan a4afca7b73 ldap: Don't authenticate if realm is None.
Fixes #5431
2017-06-21 10:14:34 -04:00
Umair Khan 005037830c test_auth_backends: Fix indentation. 2017-06-21 10:14:34 -04:00
Sampriti Panda 5dc053d6fb bots: Add validation to add_bot_backend to prevent empty short names
Fixes #5487
2017-06-21 10:11:08 -04:00
Umair Khan 4f223c19d8 sign-in: Show proper error for deactivated user.
Show a clear error message when a user tries to sign in with
a deactivated account.

Fixes #4757
2017-06-20 11:38:58 -04:00
Harshit Bansal dfbf8b8086 bugdown: Remove shortname mention syntax.
We used shortnames for mentioning users before we had autocomplete
feature. Since we now have autocomplete typeahead, this syntax is
no more useful and just causes problems. This commit removes the
shortname mention syntax.

Fixes: #4189.
2017-06-19 16:57:59 -04:00
Jack Zhang af3b7d4c98 message-edit: Patch html tags after edits on new lines. 2017-06-19 16:43:11 -04:00
vaibhav 33c0c00cd6 Outgoing webhook System: first Iteration of outgoing webhook UI. 2017-06-19 16:05:54 -04:00
Aditya Bansal f32c1892ff preview.py: Fix error raised on uploading file with unicode filename. 2017-06-19 14:58:44 -04:00
Cory Lynch 552c39a036 fenced_code.py: Improve test coverage for special chars. 2017-06-19 06:53:25 -04:00
Cory Lynch b4419528f7 Update narrow.py, messages.py to support is:unread. 2017-06-19 06:51:13 -04:00
vaibhav 42af09663d validators: Add check_url validator.
This just uses Django's existing validator, and reformats its
interface to work with has_request_variables.
2017-06-15 15:03:26 -07:00
Tim Abbott 93ea4128ac auth: Require the realm_name field in RegistrationForm.
Previously, the only required field in RegistrationForm was the full
name (and possibly ToS, depending on settings).  This meant that if
LDAP was configured, realm creation would break, because the form
would be valid the first time one landed on it, before the user even
filled it out!

The correct fix is to make the extra fields required in
RegistrationForm in the event that we're doing realm creation.

It's possible that a cleaner fix would be to use a subclass.

With a test from Umair Waheed Khan.

Fixes #5387.
2017-06-15 11:04:25 -07:00
Abhijeet Kaur af7e08acb0 bots: Add UI to view bot types of existing bots in "Your bots".
Tweaked by tabbott for more standard internationalization.
2017-06-15 10:08:31 -07:00
kunall17 d8310724b6 Use absolute URLs in GCM push notifications.
This will eventually allow us to simplify the logic required for the
mobile apps processing GCM notifications.
2017-06-15 03:56:59 -07:00
Harshit Bansal 7950f82f27 realm_emoji: Make sure that a deactivated realm emoji can't be used.
A deactivated realm emoji should neither be accepted further as a
reaction nor its further occurences in a message be rendered as an
emoji. However, all the old occurences should continue to render
normally.
2017-06-15 02:59:57 -07:00
Tim Abbott 1043cd7a2b docs: Fix tests for error pages.
Without the trailing /, these return a 30x error.
2017-06-14 21:21:36 -07:00
Cory Lynch 78de70688e dev: Add URL to preview error 404/5xx pages. 2017-06-14 21:18:45 -07:00
Cory Lynch cb4dadfdcd tools: Create /devtools page for useful dev URLs.
Fixes #5179.
2017-06-14 20:17:56 -07:00
Umair Khan 399973167c testing: Add testing framework for Tornado. 2017-06-13 15:04:04 -07:00
Rishi Gupta dbac262509 hotspots: Change hotspots to have a name and a description. 2017-06-13 12:59:33 -07:00
Aditya Ramesh a77aa79da9 emails: Add html versions of emails that do not have them.
Fixes #4691.
2017-06-12 23:02:28 -07:00
Rishi Gupta 9a02739008 emails: Remove invitation_mit. 2017-06-12 23:02:28 -07:00
Rishi Gupta 3332ebaf22 emails: Remove confirmation_registration_mit.
Not enough new Zephyr signups to justify the carrying cost :(.
2017-06-12 23:02:28 -07:00
Rishi Gupta 215568aae2 actions.py: Remove transaction.atomic from do_start_email_change_process.
Don't think it's necessary?
2017-06-12 23:02:28 -07:00
Steve Howell 1ba7d1ec68 Clean up var names in test_user_getting_add_peer_event(). 2017-06-10 10:43:57 -07:00
Rishi Gupta 1d702d13a4 emails: Replace context variable zulip_support with support_email. 2017-06-10 01:35:03 -07:00
Rishi Gupta 15b967fc3e emails: Move support_email into a common context. 2017-06-10 01:25:44 -07:00
Rishi Gupta 056489c247 emails: Move verbose_support_offers into a common context.
Server settings should just be added to the context in build_email, so that
the individual email pathways (and later, the email testing framework)
doesn't have to worry about it.
2017-06-10 01:16:03 -07:00
Rishi Gupta 7fae8fb53d test_emails: Make fixture values more helpful for visual review.
Also fixes the fact that we were pulling assets from chat.zulip.org.
2017-06-09 23:34:24 -07:00
Tim Abbott 2514ee4294 test_docs: Add test coverage for new /emails/ endpoint. 2017-06-09 16:25:39 -07:00
Akhil 4986027126 org-settings: Add backend for notifications stream.
Realm.notifications_stream is not a boolean, Text or integer field, and
thus doesn't fit into the do_set_realm_property framework. Added function
to update it in actions.py. Altered the view, realm.py, to accept
stream-id. Also, notifications stream can be disabled by sending a
negative id.
2017-06-09 16:03:52 -07:00
Steve Howell 96db97d4f7 minor: Sort imports in test_events.py. 2017-06-09 13:10:43 -05:00
Jack Zhang 98a785bb25 message_edit: Replace highlighting replaced text with delete / insert.
Rationale: For the more off-to-the-side edit history view, changes
are easier to digest by highlighting deleted content in red followed
immediately by added and changed content in green.

TODO: Toggle for showing the edited messages without highlighting;
deleted content would not be shown in this view.
2017-06-07 11:06:15 -07:00
Tim Abbott 4efdf85f62 bots: Add test for invalid bot type. 2017-06-06 21:16:27 -07:00
Abhijeet Kaur 60ff82ed7c bots: Add UI for creating different types of bot.
Add 'Type of bot' option for bots by adding dropdown option in
settings->"Your bots".  For now, this allows creating incoming webhook
bots in addition to default bots.

This will enable users to add a bot as an incoming webhook
(in addition to add full-featured bots).

With various minor tweaks and cleanups by tabbott.

Fixes #2186.
2017-06-06 21:11:22 -07:00
Greg Price 04b363f3a2 apps: Redirect /apps -> zulipchat.com/apps, except on upstream itself.
This page describes software the user will get from upstream for
their own devices, independent of what's on the server they're
using.  So it should live in a place maintained together with
that other software, rather than be distributed and versioned
with the server.

The use of ZILENCER_ENABLED to tell the difference is rather a hack
but is currently how we do this in the small handful of similar
spots; see #5245.

Fixes #5234.
2017-06-06 11:21:20 -07:00
Tim Abbott d2079cbb2e streams: Fix leaking private streams after last user is removed.
When the last user on a private stream is removed, the stream is no
longer possible to administer, and thus should be marked as
deactivated, so that default streams entries are removed and it no
longer appears in the UI as a non-administerable broken stream.
2017-06-04 10:40:41 -07:00
Aditya Bansal 25bb21238f pep8: Add compliance with rule E261 test_unread.py. 2017-06-04 15:07:39 +05:30
Aditya Bansal bf1e3a0125 pep8: Add compliance with rule E261 test_logging_handlers.py. 2017-06-04 15:07:26 +05:30
Aditya Bansal 4410b5889a pep8: Add compliance with rule E261 test_auth_backends.py. 2017-06-04 15:06:52 +05:30
Tommy Ip 8d6af030c0 bugdown: Fix modal_link.
The `data-toggle` property prevented the new style of overlay modals
from launching, and regardless, isn't a future-proof options for how
this should work.
2017-06-03 18:41:19 -07:00
Tim Abbott 1549f8773e portico: Always display org info when a server has only 1 realm.
Previously, we were incorrectly using the get_unique_open_realm
function to determine whether we're in the (common) single-realm
server case and should just display an org-info-enabled login form on
the homepage.

Now, we use a slightly different function extracted from
get_unique_open_realm that doesn't check whether the realm is
invite-only.

Fixes #4841.
2017-06-02 15:00:22 -07:00
Tim Abbott 1f48fa2767 invite: Fix invite_by_admins_only to be enforced in backend.
This is CVE-2017-0896.

Apparently, this setting never actually was wired up to anything other
than hiding the UI widget.

Huge thanks to Ibram Marzouk from the HackerOne community for finding
this security bug.
2017-06-01 17:24:29 -07:00
Tim Abbott 73f8653da6 test_messages: Increase time allowed for bulk send.
This fixes an occasional test flake we've been seeing recently.
2017-06-01 00:00:36 -07:00
Aditya Bansal a0faf3364a pep8: Add compliance with rule E261 test_reactions.py. 2017-05-31 17:07:15 -07:00
Aditya Bansal 7bb669ccb4 pep8: Add compliance with rule E261 test_email_mirror.py. 2017-05-31 17:07:15 -07:00
Umair Khan 5d794d08dd test_bulk_message_fetching: Add debug code.
This test fails on self.assertTrue(delay < 0.001 * num_ids, error_msg)
randomly. This commit adds debug code to see what the real values of
paramters are.
2017-05-31 09:27:30 -07:00
Umair Khan 71f97b7bcb testing: Invalidate cache before counting queries.
To get accurate count of the queries, we should make sure that
caches don't come into play. If we count queries while caches are
filled, we will get a lower count. Caches are not supposed to be
persistent, so our test can also fail if cache is invalidated
during the course of the unit test.

This commit solves the problem with Stream cache. This cache comes
into play when we use `get_stream` function. If cache is valid,
we will not issue queries to Stream and Recipient table. I think
the problem was one of those rare occasions when the Stream cache
got invalidated during the course of the test, due to which query
count was increased by 2. After this commit, we intentially invalidate
the Stream cache.
2017-05-30 17:28:41 +05:00
K.Kanakhin 2434f2d96c messages: Add support for admins deleting messages.
This makes it possible for Zulip administrators to delete messages.
This is primarily intended for use in deleting early test messages,
but it can solve other problems as well.

Later we'll want to play with the permissions model for this, but for
now, the goal is just to integrate the feature.

Note that it saves the deleted messages for some time using the same
approach as Zulip's message retention policy feature.

Fixes #135.
2017-05-29 21:59:38 -07:00
Harshit Bansal 298e23b447 realm_emoji.py: Allow an user to delete an emoji uploaded by them.
If a realm is configured to allow any user to upload an emoji,
then, an emoji author must be allowed to delete an emoji uploaded
by them.
2017-05-29 20:21:25 -07:00
vaibhav 9cf9837f12 webhooks: Add outgoing webhook bot user to development database. 2017-05-29 16:01:23 -07:00
Tim Abbott 4893779001 test_events: Avoid coverage errors in LogEventsTest. 2017-05-29 15:26:33 -07:00
Maxim Averin a4c3f571db Switch change_tos_version to use RealmAuditLog. 2017-05-29 15:24:01 -07:00
Maxim Averin 685fb16c39 Switch change_full_name to use RealmAuditLog.
This requires adding an `acting_user` parameter to the
`do_change_bot_owner` function.
2017-05-29 15:22:08 -07:00
Steve Howell 1f4e1ece3a Fix flaky time-based test.
For testing waiting period thresholds, we need to
explicitly set date_joined for Hamlet to be now.
2017-05-29 11:54:06 -06:00
Ron Shafii 8cc1f13d24 test: Add test_notify_realm_of_new_user. 2017-05-28 19:14:52 -07:00
Ethan Estrada 91e6f00d57 Test that `log_event` auto creates missing log dir. 2017-05-28 18:24:00 -07:00
Ethan Estrada a11c578d57 Test for early return when "LOG_EVENT_DIR" is `None`. 2017-05-28 18:21:50 -07:00
Reid Barton ccb4c5c26f bots: Move zephyr-related files to api/integrations/zephyr/. 2017-05-26 15:07:02 -07:00
Tim Abbott f5373c46f2 lint: Fix trailing newlines in outgoing webhook tests. 2017-05-25 16:28:11 -07:00
Elliott Jin 4907f24603 bots: Add additional service bot tests. 2017-05-25 15:00:51 -07:00
Elliott Jin af49a23013 bots: Generalize service bot tests to also test embedded bots. 2017-05-25 15:00:51 -07:00
Elliott Jin cd8c77bf91 bots: Clean up order and naming of service bot tests. 2017-05-25 15:00:51 -07:00
Elliott Jin 49adeee8c8 bots: Move service bot tests into separate file. 2017-05-25 15:00:51 -07:00
Vishnu Ks bb98e35aa0 Replace othello@zulip.com with example_email('othello'). 2017-05-24 19:37:36 -07:00
Vishnu Ks 52d2d6c4fa Replace prospero@zulip.com with example_email('prospero'). 2017-05-24 19:37:36 -07:00
Vishnu Ks c4db3b7d1c Replace cordelia@zulip.com with example_email('cordelia'). 2017-05-24 19:37:36 -07:00
Vishnu Ks 961b35d52e Replace iago@zulip.com with example_email('iago'). 2017-05-24 19:37:36 -07:00
Vishnu Ks 5230eaef1c Replace hamlet@zulip.com with example_email('hamlet'). 2017-05-24 19:37:36 -07:00
Vishnu Ks b65f692fcf Make LoginEmailValidatorTestCase use ZulipTestCase instead of TestCase. 2017-05-24 19:23:48 -07:00