Commit Graph

2544 Commits

Author SHA1 Message Date
Eklavya Sharma fb1d00fc6c Upgrade python-twitter to latest version.
Fixes #1145.
2016-07-17 10:23:25 -07:00
Eklavya Sharma 4130170da8 bugdown: Fix string handling.
Use unicode strings where needed and correctly encode/decode strings.
2016-07-17 10:23:25 -07:00
Eklavya Sharma abe7faa2f9 bugdown: Annotate some variables. 2016-07-17 10:23:25 -07:00
Steve Howell 60bd20da61 Remove zerver/lib/query.py (dead code). 2016-07-16 14:51:15 -07:00
Rishi Gupta 09754c9861 Remove JsonableErrors from actions.do_update_message.
There were a bunch of authorization and well-formedness checks in
zerver.lib.actions.do_update_message that I moved to
zerver.views.messages.update_message_backend.

Reason: by convention, functions in actions.py complete their actions;
error checking should be done outside the file when possible.

Fixes: #1150.
2016-07-15 13:55:49 -07:00
Rishi Gupta d529a94e4d Add realm setting to time-limit editing of message content.
This is controlled through the admin tab and a new field in the Realms table.
Notes:
* The admin tab setting takes a value in minutes, whereas the backend stores it
  in seconds.
* This setting is unused when allow_message_editing is false.
* There is some generosity in how the limit is enforced. For instance, if the
  user sees the hovering edit button, we ensure they have at least 5 seconds to
  click it, and if the user gets to the message edit form, we ensure they have
  at least 10 seconds to make the edit, by relaxing the limit.
* This commit also includes a countdown timer in the message edit form.

Resolves #903.
2016-07-15 13:55:49 -07:00
Eklavya Sharma 29673411df Make backend tests (hackishly) pass on python3.4.
Replace bot_owner=bot_owner by bot_owner_id=bot_owner.id while
creating a UserProfile object.
2016-07-15 11:13:33 -07:00
Eklavya Sharma 8c0d7311ac zerver/decorator.py: Use BytesIO to read request.
Use BytesIO instead of StringIO to get a file-like object on the
request's body.
2016-07-15 11:13:33 -07:00
Umair Khan 8dabc97d9e Add tests for frontend i18n regexes. 2016-07-15 14:48:09 +05:00
Umair Khan a07a810a2e Fix regex for capturing frontned strings.
Fixes: #1155
2016-07-15 14:48:05 +05:00
Steve Howell 9d7716f368 Add message.topic_name() helper function.
This little helper will facilitate adding a new Topic
table in the future.
2016-07-14 09:54:56 -07:00
Steve Howell a5b9e3b893 Add test_patch_bot_avatar().
Fixes: #1002
2016-07-13 22:53:46 -07:00
Steve Howell 21fb7693d2 Add test_helpers.client_patch_multipart(). 2016-07-13 22:53:40 -07:00
Steve Howell 538962f3ca Add test_add_bot_with_username_in_use(). 2016-07-13 22:53:40 -07:00
Steve Howell 468faf5724 Add test_add_bot_with_bad_username(). 2016-07-13 22:53:40 -07:00
Steve Howell de4cc80aa0 Add test_add_bot_with_too_many_files(). 2016-07-13 22:53:40 -07:00
Steve Howell e9ddf28b2c Add test_add_bot_with_user_avatar(). 2016-07-13 22:53:40 -07:00
Tomasz Kolek f689fbfa4d Replace old Trello integration with a webhook integration.
Fixes: #709.

[With tweaks on the documentation by tabbott]
2016-07-13 21:00:27 -07:00
Steve Howell 701bb7a59f Remove test_helpers.DummySession 2016-07-13 19:08:28 -07:00
Steve Howell 616e49e2e8 Remove test_helpers.DummyTornadoRequest 2016-07-13 19:08:28 -07:00
Steve Howell 0060ea7903 Remove test_helpers.DummyStream 2016-07-13 19:08:28 -07:00
Steve Howell 191ac80475 Remove callback logic from test_helpers.DummyHandler. 2016-07-13 19:08:28 -07:00
Eklavya Sharma d7ea2b8a67 zerver/tests/test_upload.py: Use byte strings.
Some string literals are incorrectly marked as `str` strings.
Change them to byte strings.
2016-07-13 16:00:46 -07:00
Eklavya Sharma a7d2dab28f Fix non-deterministic output from Taiga integration.
Taiga's webhook integration would give output events in a random
order which caused test failures on python 3 (seems like python
3 is more prone to non-deterministic failures).  Fix that by
sorting the outputs obtained from events before concatenating them.
2016-07-13 16:00:46 -07:00
Eklavya Sharma 5382aeb385 Render PagerDuty message using ujson.dumps.
Use ujson.dumps to render raw messages sent by the PagerDuty
integration instead of using pprint.pformat.  pprint.pformat
gives different results on python 2 and 3.
2016-07-13 16:00:46 -07:00
Eklavya Sharma 4f633bcd0b zerver/lib/notifications.py: Fix string encoding/decoding.
Correctly encode and decode strings in convert_html_to_markdown.
It wasn't possible to use universal_newlines=True since
Popen.communicate doesn't encode/decode strings correctly on
python 2.
2016-07-13 16:00:46 -07:00
Eklavya Sharma e6502710b6 Change exception.message to str(exception).
The 'message' attribute in Exception has been deprecated.
It has been removed in python 3.
2016-07-13 16:00:46 -07:00
Eklavya Sharma 993558c680 Get mogrified SQL queries as text.
zerver.lib.test_helpers.queries_captured returns a byte string,
while we want the queries to be text.  So decode the captured
queries.
2016-07-13 16:00:46 -07:00
Tim Abbott c0a6672471 Update path to language_options.json in production.
The previous code didn't correctly transport language_options.json to
the production environment.
2016-07-13 12:30:45 -07:00
Umair Khan 395e053ce3 Revert "Revert "Extract reply from email.""
This reverts commit f1ba3ded42.
2016-07-13 11:24:18 -07:00
Umair Khan f15dfc69fb Make code Python 3 compatible.
Make convert_html_to_markdown function Python 3 compatible.
2016-07-13 11:24:18 -07:00
Tim Abbott 34a251adb1 upload: Fix exception uploaded files with unknown content type.
It turns out our detected content_type can actually be None, which
meant calling force_text on it broke uploading some files.
2016-07-13 10:57:55 -07:00
Umair Khan 0aae0eab49 Switch to PyAPNS for sending push notifications.
Switch to [PyAPNS](https://github.com/djacobs/PyAPNs).

Fixes #538.
2016-07-13 10:55:07 -07:00
Umair Khan 636466ff8b Fix unreliable tests for missed messages.
Generate random token using getrandints.

Resolves the proximal issue discussed in #1212.
2016-07-13 10:51:21 -07:00
Steve Howell c069c8f1e7 Add test_update_api_key_for_invalid_user(). 2016-07-13 07:51:00 -07:00
Steve Howell aea79517f5 Test bot_owner logic in /json/users
This fixes a small gap in our coverage for get_members_backend.
2016-07-12 23:01:56 -07:00
Steve Howell 108ce82571 Add test_api_with_insufficient_permissions(). 2016-07-12 22:16:00 -07:00
Steve Howell cee47e7f82 Add test_api_with_nonexistent_user(). 2016-07-12 22:16:00 -07:00
Steve Howell f43be3bd8f Add test_updating_non_existent_user(). 2016-07-12 22:16:00 -07:00
Tim Abbott 647cead0d1 slow queries: Include full log line in slow query log.
The extra data is useful, and I think this won't make the lines annoying long.
2016-07-12 19:12:49 -07:00
Tim Abbott 305189956b update_message_flags: Log number of messages updated. 2016-07-12 19:12:49 -07:00
Eklavya Sharma e28b038f1d Use assert_in_response by fixing line-wrapping in templates. 2016-07-12 17:52:47 -07:00
Steve Howell 3923f94a2b Remove unused code path in avatar() endpoint. 2016-07-12 17:37:02 -07:00
Steve Howell 3995a5d8eb Add test coverage for /avatar/<email> 2016-07-12 17:37:02 -07:00
Tomasz Kolek d0066c37ff Add bitbucket2 integration.
Bitbucket changed the format of their API.  The old format is still
useful for BitBucket enterprise, but for the main cloud verison of
Bitbucket, we need a new BitBucket integration supporting the new API.
2016-07-12 17:34:34 -07:00
Eklavya Sharma c98c3d5f8d Re-enable some backend tests on python 3. 2016-07-12 14:06:29 -07:00
Eklavya Sharma d740a87d04 zerver/tests: Use unicode strings.
* Use unicode strings for strings containing non-ASCII characters.
* Decode response content when text output is expected.
2016-07-12 14:06:29 -07:00
Eklavya Sharma 161c27d0e9 Add methods to AuthedTestCase to test response content.
Add methods assert_equals_response and assert_in_response to
AuthedTestCase.  These methods make it convenient to check if
a string equals the contents of an HttpResponse's body or if a
string is a substring of the contents of an HttpResponse's body.
2016-07-12 14:06:29 -07:00
Eklavya Sharma 2080ff6c2a Decode response.content everywhere except in tests.
response.content is binary data, but code usually assumes it to
be text.  Fix this by decoding response.content where required.
Don't do this in tests yet.
2016-07-12 14:06:29 -07:00
Steve Howell 45d1eefc52 Add test_create_user_backend().
This tests provides full line coverage on the
create_user_backend view.
2016-07-12 12:29:24 -07:00
Eklavya Sharma 73a3c9fa47 On python 3, skip expected failures in backend tests. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 4868cd9969 zerver/lib/test_helpers.py: Add skip_py3 decorator. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 0a9c600c8b Show skipped tests in test runner. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 958335bdb3 Change all default values in models to unicode. 2016-07-11 21:30:32 -07:00
Tomasz Kolek d7c7279523 Add is_status_message method to Message model. 2016-07-11 12:09:06 -07:00
Tim Abbott e21bc11cfd Extract attachment_url_to_path_id. 2016-07-10 18:07:37 -07:00
Tim Abbott 48ae178d0b Refactor attachment_url_re to be a common value in upload.py. 2016-07-10 18:04:58 -07:00
rahuldeve c5756e4fa4 Update attachment tracking on message update. 2016-07-10 18:01:59 -07:00
Tim Abbott 3647973069 update_message_backend: Reorganize validation checks.
The new organization more clearly does the permission check first.
2016-07-10 18:01:59 -07:00
Tim Abbott 05632b68e1 edit: Add validation for setting empty content. 2016-07-10 18:01:59 -07:00
Tim Abbott 0d418d5695 Add some message editing tests for invalid input. 2016-07-10 18:01:59 -07:00
Tim Abbott 211a166abc message edit: Move new topic non-empty check to view. 2016-07-10 18:01:59 -07:00
Eklavya Sharma 0900ca5353 Replace assertItemsEqual by assertEqual.
This is needed because assertItemsEqual doesn't exist in python 3.
2016-07-10 17:33:11 -07:00
Rishi Gupta 43c2f35776 Add realm setting to disable message editing.
This is controlled through the admin tab and a new field in the Realms
table.  This mirrors the behavior of the old hardcoded setting
feature_flags.disable_message_editing.  Partially resolves #903.
2016-07-10 11:57:24 -07:00
Rishi Gupta 07e7230ae1 models.py: Add post_save to Message.
Flushes message from cache after a message.save(). Needed for tests
where we directly manipulate Message objects in the database.
2016-07-10 11:42:18 -07:00
Eklavya Sharma f1ba3ded42 Revert "Extract reply from email."
This reverts commit f1f48f305e.

The use of sklearn unfortunately caused a substantial slowdown to the
Zulip provisioning process, which didn't seem worth it for a
relatively minor feature.
2016-07-10 11:30:30 -07:00
Eklavya Sharma 9161ddaee0 zerver/middleware.py: Handle binary data in errors.
In write_log_line, error_content can be binary_type and
error_content_iter can be a Sequence of binary_type.  Handle
this this in a python 3 compatible way.  Also change annotations
to reflect this fact.
2016-07-10 11:30:13 -07:00
Steve Howell c671881713 decorators: Extract is_local_addr(). 2016-07-09 17:58:48 -07:00
Steve Howell 8e528569a7 Clean up rate_limit() for deployments that opt out.
If settings.RATE_LIMITING is False, short circuit rate
limiting earlier in rate_limit().  This change particularly
avoids inspect request.user and possibly spamming the error
log for sites that don't care about rate limiting.
2016-07-09 17:58:20 -07:00
Eklavya Sharma 801bcdd956 zerver/tests/tests.py: Fix non-deterministic failure.
Replace occurences of list(d.keys()) by sorted(d.keys()).
2016-07-09 17:54:54 -07:00
Eklavya Sharma 1f3ce7cf38 zerver/worker/queue_processors.py: Open file in binary.
Open a file in binary mode instead of text mode.
2016-07-09 17:54:54 -07:00
Eklavya Sharma 2f2e543a0e zerver/tests/tests.py: Replace assertItemsEqual.
Replace assertItemsEqual(a, b) by assertEqual(sorted(a), sorted(b))
because assertItemsEqual has been removed in python 3.
2016-07-09 17:54:54 -07:00
Steve Howell 89105e41d7 Added TestInternalNotifyView. 2016-07-09 07:47:12 -07:00
Steve Howell ce14a3551d Added test_get_client_name(). 2016-07-09 07:37:13 -07:00
Steve Howell 69f18c26fc Improve test_api_key_only_webhook_view().
This test now covers all lines of api_key_only_webhook_view(),
including the error logic and rate limiting logic.
2016-07-09 07:37:13 -07:00
Steve Howell 3d095beb63 Extracted client_is_exempt_from_rate_limiting(). 2016-07-09 07:37:13 -07:00
Steve Howell 49543b9ec4 Added RateLimitTestCase tests. 2016-07-09 07:37:13 -07:00
Steve Howell c680c6a981 Removed unused to_non_negative_float() function. 2016-07-09 07:37:13 -07:00
Umair Khan b7ec66fc96 Remove templates, styles and js through storage. 2016-07-09 07:33:35 -07:00
Umair Khan 043ae8ad65 Upgrade to Django-Pipeline==1.6.8. 2016-07-09 07:09:55 -07:00
Umair Khan 2d243c0703 get_all_templates should not return __init__.py. 2016-07-09 07:09:55 -07:00
Tim Abbott 1e2d38e790 Move stringify_message_dict into to_dict_uncached. 2016-07-08 17:58:56 -07:00
Tim Abbott 72e948d19a Remove now-unused message_cache_key message cache.
Originally this cache was used to transmit data from Django to Tornado
(and also for general message caching purposes), but now nothing
actually reads from this cache, so we can eliminate it.
2016-07-08 17:58:56 -07:00
Tim Abbott 8d5ec14b31 cache_helpers: Fill to_dict cache instead of old message cache.
Apparently, the message cache we were filling was completely useless
and unused, and furthermore, the cache we were filling as part of
restarting the server was also totally useless, since it didn't have
the messages users would be requesting.
2016-07-08 17:58:56 -07:00
Taranjeet a8a4caf2c0 zerver: Fix lines with length greater than 120. 2016-07-08 11:41:43 -07:00
Umair Khan f1f48f305e Extract reply from email. 2016-07-08 10:58:25 -07:00
Umair Khan 75bd3541ea Add tests for new email policy.
- Update test_extra_context_in_missed_stream_messages
- Add test_extra_context_in_personal_missed_stream_messages
- Add test_extra_context_in_huddle_missed_stream_messages
2016-07-08 10:58:25 -07:00
Umair Khan 8538ba8ea8 Remove do_send_missedmessage_events function. 2016-07-08 10:58:25 -07:00
Umair Khan 06355105f5 Missed message emails should come from user email.
Fixes #448
FIxes #612
2016-07-08 10:58:25 -07:00
Eklavya Sharma 3e9349df4f zerver/decorator.py: Use force_bytes instead of encode.
The value type of request.META is str, not text type.
So use force_bytes on the data instead of encode('utf-8').
2016-07-08 01:24:30 +05:30
Eklavya Sharma 4cf7641ab1 zerver/tests/test_i18n.py: Ignore due to incomplete stubs.
In python 3, http.cookies has incomplete stubs.
2016-07-07 12:42:51 -07:00
Eklavya Sharma 6c3f1bb967 beanstalk.py: Encode and decode strings correctly. 2016-07-07 12:42:51 -07:00
Eklavya Sharma 26b8e7357a zerver/views/messages.py: Operate on bytes in highlight_string. 2016-07-07 12:42:51 -07:00
Eklavya Sharma 83640ed0cd runtornado.py: Ignore due to incorrect stubs. 2016-07-07 10:09:35 -07:00
Eklavya Sharma 6fd8906358 rename_stream.py: Fix broken code.
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
2016-07-07 10:08:29 -07:00
Eklavya Sharma c679c180f5 enqueue_file.py: Add type hint for mypy. 2016-07-07 10:07:51 -07:00
Eklavya Sharma 896c18a57b email-mirror.py: Make it pass on mypy in python 3.
* Replace filter by list comprehension.

* Add '# type: ignore' to statements which use attributes from
  modeule `posix`, since stubs for posix are missing on python 3.
2016-07-07 10:07:28 -07:00
Eklavya Sharma 17cb6e00bd create_stream.py: Fix broken code.
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
2016-07-07 10:06:39 -07:00
Eklavya Sharma 7956fcbf0d zerver/lib/str_utils.py: Allow specifying encoding. 2016-07-07 10:06:22 -07:00
Eklavya Sharma 63d55bdd86 zerver/views/__init__.py: decode b64encoded ccache.
Convert b64encoded ccache to `str` before passing to
subprocess.check_call.
2016-07-07 10:02:08 -07:00