Commit Graph

3112 Commits

Author SHA1 Message Date
Steve Howell 33f576f514 tests: Add coverage for email/stream encode/decode logic. 2018-08-13 13:23:29 -07:00
Rishi Gupta 4813bac98b signup: Add opayq.com to disposable domain whitelist. 2018-08-13 10:56:47 -07:00
Rishi Gupta 228e237d51 tests: Fix incorrect comment in test_invite_using_disposable_email. 2018-08-13 10:56:47 -07:00
Steve Howell 46e25896c7 tests: Test handling of bogus widget_content. 2018-08-13 10:37:35 -07:00
Steve Howell 297f086b6a tests: Add coverage for service bot events.
We test the "skipping" logic a couple different ways.
2018-08-13 10:37:35 -07:00
Steve Howell 90844700fb tests: Test for unexpected bot types. 2018-08-13 10:37:35 -07:00
Shubham Padia e21e8c1bae compose: Hide subscribe button and change text for waiting period users.
Fixes #10124.
Users in the waiting period category cannot subscribe other users to
a stream. When a user tries to mention another unsubscribed user, a
warning message appears with a subscribe button on it to subscribe
the other user.
This commit removes the subscribe button and changes the warning text
for users in the waiting period category.
2018-08-13 10:18:35 -07:00
Shubham Padia feb2cdf378 onboarding: Change logic for preventing new login emails for a new user.
Issue: When you created a new organization with /new, the "new login"
emails were emailed. We previously had a hack of adding the
.just_registered property to the user Python object to attempt to
prevent the emails, and checking that in zerver/signals.py. This
commit gets rid of the .just_registered check.
Instead of the .just_registered check, this checks if the user has
joined more than a minute before.
A test test_dont_send_login_emails_for_new_user_registration_logins
already exists.

Tweaked by tabbott to introduce the constant JUST_CREATED_THRESHOLD.

Fixes #10179.
2018-08-13 10:16:41 -07:00
Steve Howell 3a2422a149 tests: Test corner cases for bot-related PMs. 2018-08-11 14:51:26 -07:00
Steve Howell 7e32e2bf81 tests: Cover ValueError logic in extract_recipients. 2018-08-11 14:51:26 -07:00
Steve Howell 134678849d tests: Test sending messages outside of your realm. 2018-08-11 14:51:26 -07:00
Steve Howell 05763dcb1d tests: Cover stream name collisions in deactivation test. 2018-08-11 14:51:26 -07:00
Steve Howell 413a0174f4 Extract a zephyr.py library.
Right now it only has one function, but the function
we removed never really belonged in actions.py, and
now we have better test coverage on actions.py, which
is an important module to get to 100%.
2018-08-11 14:51:26 -07:00
Steve Howell 71d46083da tests: Ensure streams get created for internal messages. 2018-08-11 14:51:26 -07:00
Steve Howell e9a31232df tests: Test RuntimeError logic for realm check. 2018-08-11 14:51:26 -07:00
Steve Howell 101446ec52 tests: Add coverage for message truncation logic. 2018-08-11 14:51:26 -07:00
Aditya Bansal 0ed2b1d574 thumbnails: Fix bug with '/user_avatar/' links in Zulip messages.
In this commit we fix a bug due to which url preview images for urls
to custom emojis, realm icons or user avatars appeared broken when
such urls would be part of a Zulip message.
2018-08-10 17:17:50 -07:00
Aditya Bansal 985b8bb843 thumbnails: Refactor to include '/user_uploads/' in encrypted url.
This is a preparatory commit to fix a bug in which a user posts
a link of custom emoji, user avatar or realm icon in a Zulip
message.
In this commit we are just adjusting the url generation in the
backend to have the '/user_uploads/' in the encrypted url generated
which the user is supposed to be redirected to and therefore
essentially reaching thumbor with the encrypted url.
This is necessary because 'user_uploads' and 'user_avatars' (or any
other item under 'user_avatars' endpoint) have a different folder
location under the local file storage backend. 'user_uploads'
endpoint's stuff is stored in a 'files' directory whereas stuff
'user_avatars' endpoint's stuff is stored in a 'avatars' directory.
Thumbor needs to know from which directory a particular local file
needs to be retrieved and therefore the zthumbor/loaders.py adds
a prefix location for the directory.

Since in an upcoming commit we are going to add user_avatars
directory location 'avatars' folder as a prefix this preparatory
commit helps simply doing the changes.
2018-08-10 17:17:50 -07:00
Rhea Parekh 3ff339c294 slack import: Add support for uploads in messages through 'files' keyword.
It appears that Slack just changed their export format, and how uses
this `files` list for user-uploaded files.
2018-08-10 16:20:36 -07:00
Rhea Parekh 20bca1409f import: Set emoji records 'last_modified' value in 'import_uploads_s3'.
The 'last_modified' value in emoji records is
needed for uploading the file to the S3 backend.
We set the same in the function 'import_uploads_s3'.

We also have to remove the keyword 'last_modified'
while building the RealmEmoji dict, as it is not
a field which exists in RealmEmoji objects.
2018-08-10 16:20:36 -07:00
Tim Abbott da8f4bc0e9 push notifications: Add support for removing GCM push notifications.
This uses the recently introduced active_mobile_push_notification
flag; messages that have had a mobile push notification sent will have
a removal push notification sent as soon as they are marked as read.

Note that this feature is behind a setting,
SEND_REMOVE_PUSH_NOTIFICATIONS, since the notification format is not
supported by the mobile apps yet, and we want to give a grace period
before we start sending notifications that appear as (null) to
clients.  But the tracking logic to maintain the set of message IDs
with an active push notification runs unconditionally.

This is designed with at-least-once semantics; so mobile clients need
to handle the possibility that they receive duplicat requests to
remove a push notification.

We reuse the existing missedmessage_mobile_notifications queue
processor for the work, to avoid materially impacting the latency of
marking messages as read.

Fixes #7459, though we'll need to open a follow-up issue for
using these data on iOS.
2018-08-10 13:58:39 -07:00
Tim Abbott 58307f80aa event_queue: Stop mocking push notifications in most tests.
Historically, queue_json_publish had a special third argument that was
basically its default mock behavior in the test suite.  We've been
migrating away from that model, because it was confusing and resulted
in poor test coverage of our queue worker code paths; this was one of
the last holdouts.

As it turns out, we don't exercise this code path in a way that
impacts tests much; the main downside of this change is a likely small
penalty to performance of the full test suite when sending private
messages.
2018-08-10 13:58:39 -07:00
Tim Abbott c3c7b33351 tests: Move clear_client_event_queues_for_testing to ZulipTestCase.
Following recent testing flakes that were traced down to this not
having been called causing `receiver_is_off_zulip` to depend on test
ordering, it makes sense to centralize this.

I think it should always have been in ZulipTestCase; it appears the
reason it wasn't from the beginning was that originally only
test_events.py interacted with it, and do_test there still needs to
call this directly (because it can be called multiple times within a
single test).  And then we did the wrong thing as expanded use of
Tornado event_queue code in tests to more of the codebase.
2018-08-10 13:58:39 -07:00
Tim Abbott 93c12fad28 test_events: Use clear_event_queue_for_testing more systematically.
This prevents these unit tests from accidentally leaking data outside
their boundaries.

Verified using a test that fails after test_events without this change.
2018-08-10 13:58:39 -07:00
Tim Abbott 8ceb3b88ad test_tornado: Fix leaking of event queue state.
Apparently, we weren't calling the proper clear functions inside the
Tornado tests, which resulted in unexpected behavior in other tests
that were relying on the Tornado event queue system being empty.

(In this case, a new test for mobile push notifications that assumed
receiver_is_off_zulip() was always true failed after this was run).
2018-08-10 13:58:39 -07:00
Vishnu Ks d64ba5d2be billing: Update the charged amount when user changes plan in upgrade page. 2018-08-10 13:52:35 -07:00
Rhea Parekh cf60b8821d outgoing webhooks: Warn user that PMs are not supported in Slack-format webhook.
Private messages are not supported in Slack-format webhook.
Instead of raising a NotImplementedError, we warn the user
that PM service is not supported by sending a message to the
user.

Added tests for the same.

Fixes #9239
2018-08-09 17:44:26 -07:00
Rhea Parekh 2357b1e145 tests: Rename 'event' to 'stream_message_event' for clarity. 2018-08-09 17:42:10 -07:00
Tim Abbott 83300c329b register: Fix period at end of heading. 2018-08-09 17:17:45 -07:00
Shubham Padia bdaff17e28 narrow: Use is_private flag index for `is:private`.
This implements a significant performance optimization for users
clicking the `Private messages` narrow in the Zulip UI, especially for
those users who do not have 50 recent private messages in an
organization with a lot of stream message traffic (because then
previously, postgres needed to scan through a huge amount of history
to find enough private messages).

The database index powering it can also support many other queries we
might want to do in the future to support "recent conversations" type
features.

Fixes #6896.
2018-08-09 16:10:01 -07:00
Shubham Padia 9dcac3479e actions.py: Set is_private flag in do_send_messages. 2018-08-09 16:08:04 -07:00
Shubham Padia a524d425ad actions.py: Block client interaction with flags in the NON_API_FLAGS.
Raise error if flag is present in NON_API_FLAGS or is not present in
UserMessage.flags.
2018-08-09 16:08:03 -07:00
Shubham Padia 2f895afdaa models: Do not leak `is_private` UserMessage flag to the API.
See the comment for why this is correct; basically, this flag is used
only for internal accounting, and would only confuse API clients.
2018-08-09 16:03:30 -07:00
Tim Abbott 25b646feaf decorator: Fix error message for accessing deactivated accounts.
The previous message was potentially a lot more ambiguous about
whether this was something about presence.  "Deactivated" makes it
explicit that some action was taken to deactivate the account.
2018-08-09 15:58:31 -07:00
Rhea Parekh 26300110ca import: Fix rendered_content in imported messages.
After the messages have been imported, set the rendered_content of the
messages instead of leaving its value to be 'None'.

This is important to ensure that:
(1) Performance for users is good after completing the import.
(2) The database's full-text indexes have all of the imported messages
(which only happens properly when Message rows have their
rendered_content field edited).

Fixes #9168.
2018-08-09 15:12:53 -07:00
Eeshan Garg 9e2a369e42 render_markdown_path: Add test for templates with absolute paths.
In certain cases we have to load a template directly because it
isn't in Jinja2's recognized template directories. This commit
adds a test to make sure that absolute paths are recognized
if they are pure Markdown files.
2018-08-09 14:34:52 -07:00
Shubham Padia d409555b2f auth: Add function for generating test ldap_dir to backends.py.
Generates ldap_dir based on the mode and the no. of extra users.
It supports three modes, 'a', 'b' and 'c', description for which
can be found in prod_settings_templates.py.
2018-08-09 13:46:44 -07:00
Steve Howell dc70bc1d3f tests: Make retention tests less time-sensitive.
We now update all test messages to have a pub_date
of "now" in the setUp() function in TestRetentionLib.

We've seen tests flake on query counts before this
patch.  It's not certain that the test flaked due
to time-related glitches, but it seems the most
plausible explanation.
2018-08-09 06:13:40 -04:00
Yago González 298aa0fcbf mobile: Make otp_encrypt_api_key accept API keys.
Since otp_encrypt_api_key only encrypts API keys, it doesn't require
access to the full UserProfile object to work properly. Now the
parameter it accepts is just the API key.

This is preparatory refactoring for removing the api_key field on
UserProfile.
2018-08-08 16:45:40 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Yago González 13b9dd33fa test_decorators: Explain typo-looking expression.
The validate_api_key sentence may look a bit confusing since we are
using webhook_bot's email address but default_bot's API key.

At first sight, and without any context on these tests, it may look like
that's just a typo, but we do want it to be like it is right now because
that way the API key used doesn't correspond to the provided email
address (triggering some untested parts of our backend logic).
2018-08-08 16:35:17 -07:00
Harshit Bansal 9057e543ac emoji: Disable support for letting users switch emojisets.
Due to copyright issues with potentially displaying Apple emojisets on
non-apple devices, as well as iamcal dropping support for the emojione
emojiset (see https://github.com/iamcal/emoji-data/pull/142), we are
dropping (perhaps temporarily) support for allowing users to switch
emojisets in Zulip.

This commit just hides the feature from the user but leaves most of
the infrastructure in place so that in the future if we decide to
re-enable the support we will not need to redo the infrastructure work
(some JS-side code is deleted, mostly because we'll want to re-add the
feature using the do_settings_change infrastructure anyway).

The most likely emoji set to add is the legacy "blobs" Google emoji
set, since it seems popular with some users.

Tweaked by tabbott to remove some additional JS code and update the
changelog.
2018-08-08 15:48:04 -07:00
Shubham Dhama 4d07e485ff test_event_queue: Make subscription management code readable.
This test refactor makes the subscription/stream settings changes use standard
APIs and thus be easier to follow (and more robust to subtle re-fetching bugs).

This is a follow-up to #9181.
2018-08-08 12:39:19 -07:00
Shubham Padia 17d81d46dc Realm filters: Allow `#`(hash) based URLs in filter patterns.
Fixes #9817.
Use case: When people try to add a filter to a site that uses
a hash based router, which makes the URL like:
`http://some-site.domain.com/#/c/$(id)s`.
2018-08-08 11:06:33 -07:00
Yago González 5566646c45 user_groups: Handle renaming to existing names.
Renaming a user group to a name shared by other group wasn't a scenario
handled by the backend, and the server errored whenever this was
attempted.

Now a json_error is returned, letting the user know that a user group
with that name already exists.
2018-08-08 11:03:47 -07:00
Yago González 14bc5c7d5c messages: Don't require "anchor" when "use_first_unread_anchor" is set.
The use_first_unread_anchor parameter allows automatically setting the
anchor to the first message that hasn't been read in this narrow.
Therefore it isn't necessary to specify an anchor when this parameter is
enabled.

Note from Tim: Arguably, we should think about making
`use_first_unread_anchor` the default behavior when anchor is
unspecified, but that's for later consideration.
2018-08-08 10:30:08 -07:00
Yago González e1662024d1 api docs: Load the OpenAPI file only when needed.
We found out in #9953 that, appparently, loading the OpenAPI file was
taking abut a 5% of the Zulip server startup time.

Since in many cases (especially in development) having the file loaded
won't be necessary at all, we read it on the first time data from the
OpenAPI spec is needed.

Tweaked by tabbott to add a test.
2018-08-08 09:00:28 -07:00
Yago González 25d2efb9ca api docs: Live reload the OpenAPI spec on update.
Automatically detect if the OpenAPI spec file has been modified since
the last time it was loaded into memory, and if it has, automatically
reload it to have the latest version.

This feature is designed with development environments in mind. The main
benefit is being able to see the changes made to the OpenAPI document
without needing to restart the development server, which is tedious and
slows the documentation workflow down.
2018-08-08 08:54:25 -07:00
Rhea Parekh 18a4904437 import: Move 'build_attachment' to import_util. 2018-08-07 16:45:42 -07:00
Rhea Parekh b6ccc0bc52 import: Move 'build_defaultstream' to import_util. 2018-08-07 16:45:42 -07:00