Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
12,821 Commits 28 Branches 143 Tags 606 MiB
Commit Graph

3100 Commits

Author SHA1 Message Date
Steve Howell 7a429d1e30 export: Add sanity_check_stream_data(). 2016-08-12 15:27:22 -07:00
Steve Howell ec86e475b4 export: Add Config.post_process_data 2016-08-12 15:12:01 -07:00
Steve Howell 0c2c331905 export: Flip how we fetch stream subscription data. 
We now get stream subscriptions BEFORE stream recipients.
 2016-08-12 15:12:01 -07:00
Steve Howell 70a916aae3 export: Flip how we fetch user subscription data. 
We now get user subscriptions BEFORE user recipients.
 2016-08-12 15:12:01 -07:00
Steve Howell 2a2ce6ada1 export: Remove hard-to-maintain code comment. 
Subsequent changes are gonna make the top-down/bottom-up comment
no longer valid.
 2016-08-12 15:12:01 -07:00
Steve Howell 90494cfec2 export: Note user-owned bot limitations. 2016-08-12 15:10:49 -07:00
Tim Abbott ed01842f95 notifications: Fix missed message emails with multiple senders. 
Previously, missed message emails with multiple senders would
incorrectly have a "," outside the quoted sender name part of the from
address string, resulting in confusing email output.
 2016-08-12 12:28:06 -07:00
Tim Abbott 386c6c2a31 settings: Refactor SYSTEM_ONLY_REALMS configuration. 2016-08-12 12:12:53 -07:00
Tim Abbott f0e495831e forms: Fix check for whether terms of service are enabled. 2016-08-12 11:04:45 -07:00
Steve Howell 6fdd42c08b export: Create convenient soft links. 2016-08-12 10:48:33 -07:00
Steve Howell 70b68ddcc3 export: Use a config for export_single_user(). 2016-08-12 10:37:41 -07:00
Steve Howell c69a5bdec3 export: Handle more tables via export_from_config(). 
This commit introduces the ability to do custom fetches
and to essentially use temp tables for intermediate results.

(The temp table stuff deals with recipients/subscriptions
having three different flavors--user, stream, and huddle.)
 2016-08-12 10:37:35 -07:00
Steve Howell f471a1779e export: Handle simple exports with export_from_config(). 
This handles the simple tables that don't need custom fetches.
 2016-08-12 09:54:57 -07:00
Steve Howell 682155778d export: Add export_with_config(). 
Subsequent commits will start to use this.
 2016-08-12 09:54:57 -07:00
Steve Howell b0e6d20321 export: Write stats.txt for `./manage.py export <realm>`. 2016-08-12 09:06:10 -07:00
Rishi Gupta 7910a6e134 Terms of Service: Add ability to update TOS and have users re-sign. 
Most directly useful for the migration to zulipchat.com.

Creates a new field in UserProfile to store the tos_version, as well as two
new settings TOS_VERSION and FIRST_TIME_TOS_TEMPLATE. We check for a version
mismatch between what the user has signed and the current
settings.TOS_VERSION whenever the user hits the home page, and redirect them
if needed.

Note that accounts_accept_terms.html and
zerver.views.accounts_accept_terms were unused before this commit
(they date from c327446537)
 2016-08-11 23:37:48 -07:00
Tim Abbott 11cac86a58 settings: Add support for specifying a remote redis password. 2016-08-11 16:54:22 -07:00
Steve Howell df3aa39be3 export: Extract write_data_to_file(). 2016-08-11 15:51:22 -07:00
Steve Howell f29b32bbb2 export: Clarify message exporting code. 
The function to create the message partial files has been
renamed to export_partial_message_files().  It now gets its own
list of user profile ids and recipient ids from the response,
so that we can de-clutter do_export_realm().
 2016-08-11 15:51:22 -07:00
Steve Howell 5cd915694a export: Extract launch_user_message_subprocesses(). 
This is the last in a series of commits that makes it
so that do_export_realm() mostly delegates work out
to other functions.
 2016-08-11 15:21:30 -07:00
Steve Howell b383f5ca5d export: Extract fetch_user_profile_cross_realm(). 2016-08-11 15:21:30 -07:00
Steve Howell fee2106c6f export: Extract fetch_huddle_objects(). 
This also removes the dead codepath for include_private=False.
 2016-08-11 15:21:30 -07:00
Steve Howell a6235f6a60 export: Add comments to export_single_user(). 
(This is a bit of a prefactoring to hopefully create a nice
diff in a subsequent commit.)
 2016-08-11 15:21:30 -07:00
Steve Howell 6e7fe76cf4 export: s/avatar_bucket/processing_avatars 
The name avatar_bucket was confusing for a boolean, and
in some places it was used for non-S3 paths.

I considered the more concise 'is_avatar', but that
was still confusing when you are processing multiple
files, because you think it's a calculated property
on one file instead of an overall codepath switch.

I also considered splitting up some functions, but
there is a lot of common logic between handling
file uploads and avatars that's not trivial to extract
into helpers, especially on the S3 side.
 2016-08-11 15:21:30 -07:00
Steve Howell 3dab366733 export: Clean up names of upload/avatar export functions. 
I did some minor moving around of code that made us have
one fewer function without any additional conditional
logic. The names are more explicit about saying
"from_local" and "from_s3".  Also, there is less clutter
now in do_export_realm(), which is evolving into more of
a dispatcher and less of a worker.
 2016-08-11 15:21:30 -07:00
Steve Howell 9582a83d10 export: Fix usage in export_single_user command. 2016-08-11 15:21:30 -07:00
Steve Howell d62a351107 export: Add sanity_check_output(). 2016-08-11 15:21:30 -07:00
Tim Abbott 95aee366c8 Enable local browser error reporting if ZILENCER_ENABLED. 2016-08-11 13:49:18 -07:00
Umair Khan af9f308f0e Send double columned language list in page_params. 2016-08-10 16:19:28 -07:00
Umair Khan 73e5289736 Add default_language_name in page_params. 2016-08-10 16:19:28 -07:00
Umair Khan ddaf1a2b67 Return unformatted list from get_language_list. 2016-08-10 16:19:28 -07:00
Steve Howell 06b0df5efc export: Remove spurious select_related() call for Client. 2016-08-10 14:16:17 -07:00
Steve Howell cb59a11f0a export: Extract get_primary_ids(). 2016-08-10 14:16:17 -07:00
Steve Howell 90e9083b81 export: Extract filter_by_realm(). 2016-08-10 14:16:17 -07:00
Steve Howell 4b6b1b8ad4 export: Extract filter_by_users(). 2016-08-10 14:16:17 -07:00
Steve Howell db9edfce34 export: Use DATE_FIELDS in fix_datetime_fields(). 
Now we only call this once per table and use DATE_FIELDS to
look up the data fields.
 2016-08-10 14:16:17 -07:00
Steve Howell 35c59fc4d7 export: Clean up export_messages(). 
This is pretty minor cleanup, but it makes it a little more
explicit what we're writing to the shard file, and it allows
us to use a more specific mypy type when calling
floatify_datetime_fields.
 2016-08-10 14:16:17 -07:00
Steve Howell 1d1f36c0b8 export: Always use subprocesses to export UserMessage. 
We no longer have an in-process code path to export
UserMessage rows.  We want to only maintain the
subprocess code, which we'll always use in production,
and which will work fine in dev.
 2016-08-10 14:16:17 -07:00
Steve Howell 78bbefbf94 export: Create import_attachments. 2016-08-10 14:16:17 -07:00
Rag Sagar 2fef36f15a Add realm-level default language setting. 
Adds a new field default language in the zerver_realm model.
This realm level default language will be used as default language
for newly created users. Realm level default language can be
changed from the administration page.

Fixes #1372.
 2016-08-09 17:38:29 -07:00
Steve Howell 7ec6a394fe export: Filter Attachment objects by realm. 2016-08-09 16:47:14 -07:00
Steve Howell cecfaa7761 export: Extract import_message_data(). 2016-08-09 16:47:14 -07:00
Steve Howell 5386ed280e export: Extract update_id_map(). 
We also use a vanilla dictionary instead of a defaultdict, so
that we explicitly initialize what tables are being re-mapped.
 2016-08-09 16:47:14 -07:00
Steve Howell 217ef8a4d2 export: Split fix_foreign_keys() into two functions. 
We now have convert_to_id_fields for the simple case, and
re_map_foreign_keys for the more complex case. I also
renamed some parameters and variables.
 2016-08-09 16:47:14 -07:00
Umair Khan e1090118c5 Percent translated should be an integer. 2016-08-09 16:19:07 -07:00
Rag Sagar 5771e2eb25 Move i18n functions from zerver/views/__init__.py to zerver/lib/i18n.py 2016-08-09 16:10:24 -07:00
Rag Sagar 28f0079432 Use a constant to set UserProfile default_language field max_length attribute. 2016-08-09 16:10:24 -07:00
Steve Howell dd88ffccfd export: Extract make_raw() in lib/export.py. 2016-08-09 15:58:27 -07:00
Steve Howell 09fa343bdd export: Use DATE_FIELDS in floatify_datetime_fields. 
This avoids a little bit of code duplication, plus it should
make it a little easier to add new date fields in the future.
 2016-08-09 15:58:27 -07:00
Steve Howell c14ab3c91f export: Add annotations to zerver/lib/export.py. 
I also fixed some small things like removing unnecessary return
statements, and adding a TODO.

In some cases I explicitly cast stuff at run-time to set() or
str() to appease mypy, as well as make it clear to somebody
reading the code that the callee might not respect ordering
or tolerate unicode.
 2016-08-09 15:58:27 -07:00
Steve Howell f18cc4ae3a export: Added export_avatars_local_helper(). 2016-08-09 15:58:27 -07:00
Umair Khan 86920a297c Mock out loggers in push notification tests. 
Fixes: #1575
 2016-08-09 22:16:09 +05:00
Umair Khan 7a84f12a38 Verify mock calls without strict order. 
In push notification tests, call assert_has_calls with `any_order=True`
so that the order of the calls does not matter.
 2016-08-09 22:15:37 +05:00
Tim Abbott 42b63f5caa test_templates: Fix failure to report a user-facing error. 2016-08-09 09:54:44 -07:00
Tim Abbott f3365f4089 test_templates: Add integrations_dict to context. 
This fixes the fact that this test was (silently) failing.
 2016-08-09 09:54:36 -07:00
Umair Khan fe812a89bf mypy: Type of remote_username should be text_type. 2016-08-09 09:19:24 -07:00
Umair Khan 8c249f859e Push Notification: Add tests for GCM. 2016-08-09 16:13:02 +05:00
Umair Khan acd76eb604 Annotate push_notifications.py. 2016-08-08 15:17:02 -07:00
Umair Khan 9c13e9a072 Annotate test_push_notifications. 2016-08-08 15:17:02 -07:00
Umair Khan 35dd1ffb13 Pass password as string to authenticate. 2016-08-08 15:17:01 -07:00
Umair Khan 266c791e64 Email should be a unicode. 2016-08-08 15:17:01 -07:00
Umair Khan 503ad774f5 Tokens should be unicode. 2016-08-08 15:17:01 -07:00
Umair Khan 0dd7d0dda1 push_notifications: Change name to 'identifier'. 2016-08-08 15:15:29 -07:00
Umair Khan fdbd132ba4 Annotate zerver/context_processors.py. 2016-08-08 15:15:29 -07:00
Umair Khan 00eab4e526 Annotate zerver/views/__init__.py. 2016-08-08 15:15:29 -07:00
Umair Khan 25dd1aeb5c Annotate zerver/lib/actions.py. 2016-08-08 15:13:29 -07:00
Umair Khan ef62506fb1 Annotate zerver/models.py. 2016-08-08 15:13:29 -07:00
Umair Khan e34b7a0691 Fix type annotation in message_cache_items. 2016-08-08 15:13:29 -07:00
Umair Khan ec910d96fc Annotate zerver/storage.py. 2016-08-08 15:12:12 -07:00
Umair Khan 0dfc757447 Annotate test_auth_backends.py. 2016-08-08 15:12:12 -07:00
Tim Abbott 6264ff7039 Add new Zulip realm import/export tool. 
The previous export tool would only work properly for small realms,
and was missing a number of important features:
* Export of avatars and uploads from S3
* Export of presence data, activity data, etc.
* Faithful export/import of timestamps
* Parallel export of messages
* Not OOM killing for large realms

The new tool runs as a pair of documented management commands, and
solves all of those problems.

Also we add a new management command for exporting the data of an
individual user.
 2016-08-08 14:58:18 -07:00
Tim Abbott e7adaf8db1 utils: Add mkdir_p implementation. 2016-08-08 14:36:50 -07:00
Tim Abbott 54a6bdc3ad views: Fix mypy annotations in get_invitee_emails_set. 
Apparently this buggy annotation slipped through into master.
 2016-08-05 15:04:16 -07:00
Tomasz Kolek 64b5d0ce64 Add support for inviting emails in "name" <email> format. 
Often, users will copy email addresses with a name (rather than pure
email addresses) into the Zulip "invite users" UI.  Previously, that
would throw an error.

This change also adds a get_invitee_emails_set function for parsing
emails content and a test suite for this new feature.

Fixes: #1419.
 2016-08-05 14:48:04 -07:00
Umair Khan d33fc0046f Add APNS feedback server tests. 2016-08-05 10:36:57 -07:00
Umair Khan 33450d1a0b Add APNS push notification tests. 2016-08-05 10:36:57 -07:00
Umair Khan 40054ce26c Add APNS response listener test case. 2016-08-05 10:36:57 -07:00
Umair Khan df8fc30b75 Create MockRedis class. 
Redis is not available on Travis and creating a mock Redis server looks
easier than using `mock` module.
 2016-08-05 10:36:57 -07:00
Umair Khan 2ab796aeb5 Add tests for APNS message. 2016-08-05 10:36:57 -07:00
Umair Khan 882bb5558b Push to APNS in a loop. 2016-08-05 10:36:57 -07:00
Umair Khan 4e8ca0a326 Make sure apns response code is an integer. 2016-08-05 10:36:57 -07:00
Umair Khan 5c41eae7ec Don't send connection to apns response listener. 2016-08-05 10:36:57 -07:00
Umair Khan ffaf7cb2ba Create get_connection function for APNS. 2016-08-05 10:36:57 -07:00
Umair Khan 1d2bdfdfb1 Save user id instead of user profile. 2016-08-05 10:36:57 -07:00
Umair Khan 0cac1aa135 For APNS frame identifier generate random int. 2016-08-05 10:36:57 -07:00
Umair Khan 0b5dc56b8d Redis key expiry should be an integer. 2016-08-05 10:36:57 -07:00
Umair Khan cbd53fbac8 Don't import OrderedDict. 2016-08-05 10:36:57 -07:00
Umair Khan 0b88deb640 APNS_SANDBOX should be a bool. 
Fixes: #1480
 2016-08-05 10:36:57 -07:00
Tim Abbott dcb92d41b8 Annotate test_messages.py. 2016-08-04 15:57:04 -07:00
Tim Abbott 39f1024740 test_messages: Fix remaining mypy errors. 2016-08-04 15:57:03 -07:00
Tim Abbott 88550edd9e Annotate queue_processors.py. 2016-08-04 15:57:03 -07:00
Tim Abbott 4e8054b84a Refactor queue worker tests to match actual API. 2016-08-04 15:57:03 -07:00
Tim Abbott 1e54897ca7 process_queue: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 4ddc567c50 views/__init__: Add missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 641aa08721 compilemessages: Add mypy annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a787dee48b makemessages: Add missing annotations and fix string types. 2016-08-04 15:57:03 -07:00
Tim Abbott 2da6b6bfd8 views.messages: Add some more annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 2f13b0b18a actions: Fix various string annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 4f782bc186 test_events: Fix annotation gaps. 2016-08-04 15:57:03 -07:00
Tim Abbott 2eb5baa024 zendesk: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott d7d8d6b4d2 parallel: Add missing annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott d00719f204 Annotate IFTTT tests. 2016-08-04 15:57:03 -07:00
Tim Abbott d449f6ba72 Annotate bitbucket2 tests. 2016-08-04 15:57:03 -07:00
Tim Abbott 5253c044e7 test_helpers: Add some missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a7086d3d8a trello: Fix accidentally commented type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 4840e493b2 decorator: Add missing annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott ef5b5bb45b views/streams: Add missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 92062b9526 process_queue: Add missing annotation. 2016-08-04 15:57:02 -07:00
Tim Abbott cc11229377 test_auth_backends: Add missing type annotations. 2016-08-04 15:57:02 -07:00
Tim Abbott 0689485666 Add lint check for malformed type annotations missing :. 
Also fix the 2 annotations that weren't being checked because of this.
 2016-08-04 15:53:23 -07:00
Tim Abbott af823f7a76 Annotate circleci integration. 2016-08-04 15:53:23 -07:00
Tim Abbott 55611cd21a Annotate test_i18n.py. 2016-08-04 15:53:23 -07:00
Tim Abbott 028efdcb78 integrations: Add detailed module docstrings. 2016-08-04 12:13:25 -07:00
Tomasz Kolek 09d081d9bd Replace hardcoded integration-lozenges to generating ones. 2016-08-04 11:41:18 -07:00
Tomasz Kolek 0e33b8bd4d Add integrations library. 
Define Integration and WebhookIntegration classes.
Change webhook part of integration's guide.
Replace hardcoded webhook urls to generating
based on WEBHOOKS list.
 2016-08-04 11:39:20 -07:00
Umair Khan b38ca0c690 Add tests for GitHub team and organization auth. 2016-08-03 12:01:15 -07:00
Umair Khan fc1d8e217f Don't pass `return_data` to mock functions. 
GitHubAuthBackend already passes the `return_data` keyword argument
to the `do_auth` function.
 2016-08-03 12:01:15 -07:00
Tim Abbott 5f275c9868 Re-enable mobile app advertisements in /apps.html. 2016-08-02 16:45:11 -07:00
Rishi Gupta 948ea7663c ModelReprMixin: Fix handling of missing __unicode__ function. 
The old behavior was to raise an exception, but Django was catching
the exception and doing unexpected things. For instance, in the
manage.py shell, printing out a ModelReprMixin object (with
__unicode__ not implemented) would result in nothing being printed,
rather than it raising a error or otherwise alerting the programmer as
to what was going on.
 2016-08-02 11:46:01 -07:00
Umair Khan 2f6293027d Add test for empty email gateway pattern. 2016-07-31 20:39:09 -07:00
Umair Khan 2c07f1b19a Use NOREPLY_EMAIL_ADDRESS if email gateway not enabled. 
This fixes a regression where missed message emails would not be sent
at all in the event that EMAIL_GATEWAY_PATTERN was unset.

The overall experience still isn't great, but it's better than crashing.

Fixes: #1411

[commit message expanded by tabbott]
 2016-07-31 20:38:18 -07:00
Rag Sagar b9f2397073 Add validation and tests for default language setting. 2016-07-31 17:44:10 -07:00
Christie Koehler 31efecf03d Add HelloWorld webhook integration + Walkthough. 
Improves webhook integration docs and Hello World webhook.  Includes
many suggested improvements from @timabbott and @tomaszkolek.
 2016-07-29 21:38:40 -07:00
Steve Howell 2a37dafcbb Write untested_url_report.txt. 2016-07-29 21:26:51 -07:00
Steve Howell 8b13530712 Add tools/analyze-url-coverage 2016-07-29 21:23:33 -07:00
Steve Howell 21f83afe3a Add --url-coverage option to ./tools/test-backend. 2016-07-29 21:23:33 -07:00
Tim Abbott 297566510c test_activity: Increase expected query count by 1. 
This is required because the Zulip UserProfile caching logic has
changed with 8e5e6a06f2, and sometimes
the current user will not be in the cache.
 2016-07-29 21:15:10 -07:00
Tim Abbott f7083b4079 test_templates: Fix Python 3 compatibility. 2016-07-29 21:11:27 -07:00
Tim Abbott 51672f9ddc Remove now-unused zulip_com template context variable. 2016-07-29 20:47:42 -07:00
acrefoot b7ccf64c79 Move Zulip's TOS to TERMS_OF_SERVICE markdown file. 
This reimplements zulip.com/terms using the new markdown system.
 2016-07-29 20:47:42 -07:00
acrefoot e568dbc76f Add TERMS_OF_SERVICE setting. 
This new setting allows the server administrator to add a custom
Terms of Service page by supplying the path to a markdown file.

Also adds a test.
 2016-07-29 20:47:31 -07:00
Tim Abbott 700e803840 settings: Add new DEBUG_RATE_LIMITING setting. 
This exists primarily in order to allow us to mock settings.DEBUG for
the purposes of rate limiting, without actually mocking
settings.DEBUG, which I suspect Django never intended one to do, and
thus caused some very strange test failures (see
https://github.com/zulip/zulip/pull/776 for details).
 2016-07-29 19:32:02 -07:00
Eklavya Sharma 5691ca61b0 Add tests for email_mirror management command. 2016-07-29 19:13:52 -07:00
Steve Howell 9fb071947d Remove expected_run_time from @slow test decorator. 2016-07-29 16:41:21 -07:00
Steve Howell 438a118ea5 Simplify enforce_timely_test_completion(). 
This will lead to minor differences in the warnings that
people see when they run tests that are slow.  We call out
the slowness a little more clearly from a visual standpoint,
and we simplify the calculation of the slowness threshold.

We still allow more time for tests with the `@slow` decorator
to run, but we don't use their expected_run_time.
 2016-07-29 16:41:21 -07:00
Steve Howell f67b56702a Remove @slow decorators for tests that are no longer slow. 
These tests aren't exactly zippy, but they don't even make
the top 100 slow tests.
 2016-07-29 16:41:21 -07:00
Steve Howell 4556bf528f Add --report-slow-tests option to tools/test-backend. 2016-07-29 16:41:20 -07:00
Tim Abbott 6b2b7ab3ff Rename GitHubBackend to GitHubAuthBackend for consistency. 2016-07-29 12:49:10 -07:00
Umair Khan d43b031a32 Add tests to verify GitHub backend. 2016-07-29 12:49:10 -07:00
Umair Khan 80d62de40a Add GitHub authentication. 
Fixes: #1042
 2016-07-29 12:49:10 -07:00
Umair Khan 7531d9679b Sort language dropdown list. 2016-07-29 11:58:41 -07:00
Umair Khan 2edb79776e Calculate translation percentages. 
Fixes: #1397
 2016-07-29 11:58:41 -07:00
Umair Khan 0aa72cb347 Point registration to correct url. 
While logging through GitHub if the realm of the user doesn't
exist then we are redirected to registration page but the action
points to the complete url of the GitHub oAuth overflow.
 2016-07-28 15:27:22 -07:00
Eklavya Sharma 3b3b5c7c16 Rename management command email-mirror to email_mirror. 
All other zulip management command names have underscores, so
rename email-mirror to email_mirror.

This will also make it possible to import this module, which will
help in writing tests for it.
 2016-07-28 14:52:09 -07:00
Eklavya Sharma b98b049377 Fix string handling in email_mirror queue_processor. 2016-07-28 14:52:09 -07:00
Eklavya Sharma 6e9bc44123 Re-implement email-mirror using imaplib. 
Switch from twisted to imaplib to gain python 3 compatibility and
make code easier to understand.
 2016-07-28 14:52:09 -07:00
Tim Abbott 832951d2dd Fix IFTTT integration documention on 'content' field. 2016-07-28 14:05:45 -07:00
Steve Howell ffe364c719 Don't pass UserProfile object to client_put(). 
This was in AdminCreateUserTest.test_create_user_backend().
For end to end tests we are logged in, and we need to verify
that our decorators add UserProfile to the parameters of
the view on our behalf, so that we don't get false positives.

In an upcoming commit, we will want to be able to serialize
the parameters for client_put to produce url coverage reports,
so that is another reason not to pass in the UserProfile
object.  (That was how this was discovered.)
 2016-07-28 13:57:28 -07:00
Umair Khan 8e5e6a06f2 Delete cache entry for user profile. 
Our flush functions update user profile cache entries which can cause
confusing race conditions (see e.g. #1257).  To resolve this, we move
all the user_profile flush functions to delete the entry instead of
updating it -- it will then be fetched as part of the next request
that needs to access the user object.

There are still races here, and there is perhaps an argument that a
better fix for this would be to re-fetch the object and then put it
into the cache, but this resolves the main cache correctness problem
we had with the previous implementation.

Fixes: #1322.
 2016-07-28 13:43:14 -07:00
Steve Howell 713797a65c Add lint checks for self.client.{get,patch,put,post,delete}. 2016-07-27 20:50:54 -07:00
Steve Howell 985de51903 Use self.client_delete() wrapper in more places. 2016-07-27 20:49:33 -07:00
Steve Howell 6937f63fd5 Add client_get() test helper. 2016-07-27 20:49:33 -07:00
Steve Howell 38f2a2f475 Add client_post() test helper. 
This makes us more consistent, since we have other wrappers
like client_patch, client_put, and client_delete.

Wrapping also will facilitate instrumentation of our posting code.
 2016-07-27 20:49:32 -07:00
Steve Howell 51bae8abc4 Remove stub() test helper. 2016-07-27 14:12:10 -07:00
rahuldeve ec6c9ba436 Add test for removing an uploaded file via message editing. 
Fixes #1373.
 2016-07-27 14:10:15 -07:00
rahuldeve 4267dcbb68 Change the way referred attachments are removed on message update. 
This fixes incorrect behavior when two or more attachments are removed
in the same message.
 2016-07-27 14:08:59 -07:00
Tim Abbott df525ad1c5 Remove old MitUser model and related code. 
The MitUser model caused a constant series of little problems for
users with mit.edu email addresses trying to sign up for different
Zulip servers.

The new implementation just uses conditionals on the realm object when
selecting the confirmation template to use.
 2016-07-26 20:30:12 -07:00
Tim Abbott ed6c134cf4 Clean up and document clean_email code path. 2016-07-26 20:30:12 -07:00
Tim Abbott 375551aaa6 Clean up most hardcoding of mit.edu domain checks. 
This moves all this code to be gated on a few virtual realm settings.
 2016-07-26 20:30:12 -07:00
Tim Abbott 70543e059a bugdown: Remove hardcoding of mit.edu for zephyr_mirror realm config. 2016-07-26 20:30:12 -07:00
Tim Abbott c17676b00c Cleanup MIT hardcoding for disabling presence. 2016-07-26 20:30:12 -07:00
Tim Abbott 884f50cdd7 validate_user_access: Assert user_profile is not None. 
This function is only called in cases where user_profile isn't None,
and the code reads better if we just check that first rather than
checking it on every line that accesses user_profile.
 2016-07-26 20:30:12 -07:00
Tim Abbott d1adbd798b get_subscribers_to_streams: Pass through requesting_user. 2016-07-26 20:30:12 -07:00
Tim Abbott 691ad7fcfc maybe_get_subscriber_emails: Pass user_profile through. 2016-07-26 20:30:12 -07:00
Tim Abbott f790a04102 Remove unused get_subscriber_ids. 2016-07-26 20:30:12 -07:00
Tim Abbott 1771b8275c Remove unused get_other_subscriber_ids. 2016-07-26 20:30:12 -07:00
Steve Howell eb11e02309 Add MirroredMessageUsersTest. 2016-07-26 20:01:36 -07:00
Steve Howell 3fa745ebe3 Remove untested code in is_public_stream(). 
We were calling valid_stream_name(), but more appropriate
validation checks for stream names happen elsewhere in the codepath.
 2016-07-26 18:10:00 -07:00
Steve Howell 81cd9d33e4 Add test_non_string_narrow_operand_in_dict(). 2016-07-26 18:10:00 -07:00
Tim Abbott b2d1c18268 views: Move push token views to new push_notification.py. 2016-07-26 14:44:24 -07:00
Tim Abbott 6dc5681171 views: Move presence views to presence.py. 2016-07-26 14:29:32 -07:00
Tim Abbott 62affa53c9 views: Move update_realm to new realm.py views file. 2016-07-26 14:29:24 -07:00
Umair Khan 7c138c6e33 Import is_inactive from proper file. 2016-07-26 12:26:55 -07:00
Eklavya Sharma e0dbaf1031 Use universal_newlines=True in subprocess.check_output. 
In python 3, subprocess uses bytes for input and output if
universal_newlines=False (the default).  It uses str for input and
output if universal_newlines=True.

Since we mostly deal with strings, add universal_newlines=True to
subprocess.check_output.
 2016-07-26 12:06:41 -07:00
Eklavya Sharma 5ce80eb4ba Update types of images accepted as avatars. 
* Reject SVG in frontend because backend doesn't accept it.
* Add JPEG format in backend tests for avatar upload.
 2016-07-26 16:36:47 +05:30
Eklavya Sharma a2a8b54f6e Upgrade pillow. 
Update test images because pillow's resize algorithm has changed.
 2016-07-26 16:36:07 +05:30
rahuldeve 5ed2b01bc7 upload: Fix single attachments not being claimed on message edit. 
Fixed an attachments not being claimed if just one new attachment is
added (aka the common case).
 2016-07-25 17:56:27 -07:00
rahuldeve e06d5fbec9 Pass Message object instead of message dict to do_claim_attachments. 2016-07-25 17:54:36 -07:00
rahuldeve c61d149837 Pass Message object instead of message dict to claim_attachment. 2016-07-25 17:51:17 -07:00
Steve Howell 57c436b32d Add TODOs for #1379 (bugdown mutual dependency) 2016-07-25 14:59:32 -07:00
Ashish Kumar 8398dac025 Fix 'Invalid stream' error message in get_subscription_or_die(). 2016-07-25 14:48:48 -07:00
Ashish Kumar 56048919f5 Increase test coverage of streams API by covering JSON errors. 2016-07-25 14:48:48 -07:00
Ashish Kumar 84566c750f Add test_successful_subscriptions_add_with_announce(). 2016-07-25 14:45:29 -07:00
Ashish Kumar a143677ea2 Add test_json_get_subscribers(). 2016-07-25 14:45:28 -07:00
Ashish Kumar 9104468926 Add test_existing_subscriptions_autosubscription(). 2016-07-25 14:43:27 -07:00
Steve Howell 2846e23cfc Extracted test helper: get_and_check_messages(). 
This new helper combines two old helpers, one of which was misnamed
and the other of which was always called after the first, so it
made sense to just combine the helpers.

Fixes: #1386
 2016-07-24 19:43:56 -07:00
Steve Howell 1da9325476 Require non-empty full_name or password in json_change_settings 2016-07-23 14:53:15 -07:00
Steve Howell 14717f414c Add default for full_name in json_change_settings. 2016-07-23 14:53:15 -07:00
Steve Howell f27ad0d800 Add test_changing_nothing_still_returns_success(). 
(in ChangeSettingsTest)
 2016-07-23 14:53:15 -07:00
Steve Howell 728ac09df9 Remove ChangeSettingsTest.post_with_params(). 
The individual tests now explicitly show all the parameters
they are passing to the endpoint, which makes them easier
to troubleshoot.
 2016-07-23 14:53:15 -07:00
Steve Howell f01b10f97a Improve test coverage for use_first_unread_anchor parameter. 
This commit adds these two tests:

test_use_first_unread_anchor_with_some_unread_messages
test_use_first_unread_anchor_with_no_unread_messages

The new tests add coverage to the conditional logic in
get_old_messages_backend() that looks at first_unread_result
when use_first_unread_anchor is set to True.
 2016-07-23 14:39:58 -07:00
Steve Howell 08f6ae7c14 Improved and renamed test_use_first_unread_anchor(). 
The test is now called test_use_first_unread_anchor_with_muted_topics().
Before this commit, the test exercised setting
use_first_unread_anchor to True, but it didn't inspect the
most relevant query affected by the flag.  Now it does.

This test is still kind of hard to read, and it's far from ideal,
but I'm reluctant to remove it from the test suite.
 2016-07-23 14:39:58 -07:00
Steve Howell ee2007136d Test "empty list" logic in exclude_muting_conditions(). 2016-07-23 14:39:58 -07:00
Eklavya Sharma 196e021a7f Add automated tests for zerver/lib/type_debug.py. 2016-07-23 14:38:28 -07:00
Eklavya Sharma 5b06edc402 zerver/lib/type_debug.py: Handle container types better. 2016-07-23 14:35:39 -07:00
Eklavya Sharma a632c71cce Add zerver/lib/type_debug.py. 
This file has a decorator print_types, which makes the decorated
function print the types of its arguments whenever it is called.
 2016-07-23 14:35:39 -07:00
Taranjeet Singh 9bea1f46c7 zerver/views/webhooks/bitbucket2.py: Fix line with length greater than 120. 2016-07-23 14:33:11 -07:00
Tim Abbott eba2d470dc Fix removing attachments when editing a message. 
Fixes #1373.
 2016-07-22 14:48:47 -07:00
Tim Abbott 4de203a23f Fix MIT users joining a Zulip realm. 2016-07-22 14:27:36 -07:00
Eklavya Sharma 09e4e5aea7 Add links after `type: ignore`s. 2016-07-22 13:59:31 -07:00
Eklavya Sharma 674f6999e1 Improve annotations of decorators. 2016-07-22 11:14:33 -07:00
Eklavya Sharma 4b7d94564a zerver/views/streams.py: Annotate variables to help mypy. 2016-07-22 11:14:33 -07:00
Eklavya Sharma 07e36d87a2 Remove return where no return value is expected. 
zerver.lib.notifications.send_future_email has return type None.
So replace `return send_future_email(...` by `send_future_email(...`.
 2016-07-22 11:14:33 -07:00
Krit Karan a61ef3bc0e Reformat the content variable in ``do_refer_friend()``. 
Fixes #1162.
 2016-07-22 10:36:35 -07:00
Eklavya Sharma c59740385d zerver/lib/str_utils.py: Replace ValueError by TypeError. 
When a parameter of a wrong type is passed to a `force_*` method
in str_utils.py, raise a TypeError.
 2016-07-21 23:14:00 +05:30
Umair Khan dce48d58b6 Cache translations. 
Fixes: #1158
 2016-07-21 10:23:31 -07:00
hackerkid b6620cfa57 Replace UserProfile with HttpRequest in logged_in_and_active. 2016-07-20 20:08:00 -07:00
Tim Abbott c791037166 narrow: Document BuildNarrowFilterTest. 2016-07-20 14:18:40 -07:00
acrefoot 0dfb76fc5e Change HTTP verb for create_user_backend to PUT (#1326) 2016-07-20 12:57:47 -07:00
Steve Howell 2484d870b4 Add test_build_narrow_filter(). 2016-07-20 11:04:00 -07:00
Tim Abbott 19b860ceec Rename local_settings.py symlink to prod_settings.py. 2016-07-19 20:59:59 -07:00
Tim Abbott e876f12b83 settings: Extract settings.WELCOME_EMAIL_SENDER. 2016-07-19 15:28:41 -07:00
Tim Abbott 7725c62892 Use VERBOSE_SUPPORT_OFFERS in day1 emails. 2016-07-19 15:28:41 -07:00
Tim Abbott 3ea4cbb5c3 settings: Extract settings.VERBOSE_SUPPORT_OFFERS. 2016-07-19 15:28:41 -07:00
Tim Abbott ff1faffecd settings: Extract EMAIL_GATEWAY_EXTRA_PATTERN_HACK. 2016-07-19 15:28:41 -07:00
Tim Abbott 82f78621dd settings: Extract CUSTOM_LOGO_URL. 2016-07-19 15:28:41 -07:00
Tim Abbott de679a23c9 settings: Extract settings.REGISTER_LINK_DISABLED. 2016-07-19 15:28:41 -07:00
Tim Abbott c25b077224 settings: Extract settings.SHOW_OSS_ANNOUNCEMENT. 2016-07-19 15:28:41 -07:00
Tim Abbott fb386da552 settings: Extract settings.SAVE_FRONTEND_STACKTRACES. 2016-07-19 15:28:41 -07:00
Tim Abbott 522ccf8eb2 settings: Extract settings.STAGING_ERROR_NOTIFICATIONS. 2016-07-19 15:28:41 -07:00
Tim Abbott 1f4c9eefe4 settings: Extract settings.ZILENCER_ENABLED. 2016-07-19 15:28:41 -07:00
Tim Abbott 0aaa55fb8f settings: Extract settings.SHARE_THE_LOVE. 2016-07-19 15:28:41 -07:00
Tim Abbott afaac85dc6 Move get_sqlalchemy_connection to its own file. 2016-07-19 15:28:41 -07:00
Steve Howell 83679a7775 Test search queries and highlight_string(). 
This increases test coverage by exercising highlight_string().
It also gives deeper test coverage to NarrowBuilder.by_search(),
which had test coverage before, but only in terms of inspecting
the SQL that was generated.  This test actually runs the SQL
under the hood.

This partly fixes #1006.
 2016-07-18 16:39:19 -07:00
Tim Abbott 50c3e42f0e Fix test failure due to recent merge interaction. 
599b15cb84 broke master because it had
previously only been tested before HomeTest was created.
 2016-07-18 16:37:25 -07:00
Kartik Maji 599b15cb84 Allow fetching subscribers for streams the user has never subscribed to. 
This allows the frontend to fetch data on the subscribers list (etc.)
for streams where the user has never been subscribed, making it
possible to implement UI showing details like subscribe counts on the
subscriptions page.

This is likely a performance regression for very large teams with
large numbers of streams; we'll want to do some testing to determine
the impact (and thus whether we should make this feature only fully
enabled for larger realms).
 2016-07-18 16:24:19 -07:00
Taranjeet Singh 5462341cb4 zerver/views/webhooks/github.py: Fix lines with length greater than 120. 2016-07-18 15:10:41 -07:00
Taranjeet Singh 3031214718 zerver/views/messages.py: Fix line with length greater than 120. 2016-07-18 15:05:52 -07:00
Tim Abbott 3c591aa724 compilemessages: Don't try to include zh-CN and zh_CN in language options. 2016-07-18 15:02:43 -07:00
Tim Abbott 55a98a41d8 compilemessages: Improve error handling for unknown locale. 2016-07-18 15:02:43 -07:00
Taranjeet Singh 84660a5087 zerver/views/__init__.py: Fix lines with length greater than 120. 2016-07-18 14:51:06 -07:00
Taranjeet Singh 37ea785b8f zerver/forms.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh 9f3f82d06d zerver/lib/socket.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh 5499136bfd zerver/views/webhooks/teamcity.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh 7199ee8f08 zerver/views/webhooks/taiga.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh 3e17011f9c zerver/views/webhooks/pivotal.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh c437659cd9 zerver/views/webhooks/pingdom.py: Fix line with length greater than 120. 
zerver/views/webhooks/pingdom.py: Fix indetation for function args.
 2016-07-18 14:42:39 -07:00
Taranjeet Singh 7ddb254d2e zerver/views/webhooks/circleci.py: Fix line with length greater than 120. 
zerver/views/webhooks/circleci.py: Fix indentation for function args.
 2016-07-18 14:42:39 -07:00
Taranjeet Singh 1323685140 zerver/views/webhooks/beanstalk.py: Fix line with length greater than 120. 
zerver/views/webhooks/beanstalk.py: Capture string in a temp variable.
 2016-07-18 14:42:39 -07:00
Taranjeet Singh 569d14a826 zerver/views/webhooks/airbrake.py: Fix line with length greater than 120. 
zerver/views/webhooks/airbrake.py: Fix Indentation.

zerver/views/webhooks/airbrake.py: Fix Indentation for args in function.
 2016-07-18 14:42:39 -07:00
Taranjeet Singh 10afbc06f9 zerver/views/report.py: Fix line with length greater than 120. 
zerver/views/report.py: Capture string in temporary variables.
 2016-07-18 14:42:39 -07:00
Taranjeet Singh 24ba060421 Fix line with length greater than 120 in deliver_email.py. 2016-07-18 14:34:34 -07:00
Tim Abbott 014a13df7c cache: Fix echoing of mkdir command to console. 2016-07-18 14:25:13 -07:00
Taranjeet Singh ba3f9de9a9 zerver/lib/cache.py: Move remote_cache_prefix to var directory. 
This commit ensures the var directory exists before its needed in both
development and production environments.
 2016-07-18 14:13:02 -07:00
Steve Howell 8ac1398b0f Use topic_name() in to_log_dict(). 2016-07-18 14:10:11 -07:00
Steve Howell e341fe0102 Use topic_name() in do_edit_message_assert_error(). 2016-07-18 14:10:11 -07:00
Steve Howell 2bfa763c0b Use topic_name() in update_message_backend(). 2016-07-18 14:10:11 -07:00
Umair Khan b546391f0b Move locale to static/locale 2016-07-18 13:59:07 -07:00
Umair Khan cdf2664030 `makemessages` command should not remove plurals. 2016-07-18 13:59:07 -07:00
Steve Howell 1a9a630526 Add zerver.tests.tests.HomeTest.test_home. 2016-07-18 12:22:14 -07:00
Steve Howell 5fcbd0a178 Remove muting logic in approximate_unread_count(). 
The muting logic in approximate_unread_count() was confusing
stream/subject and only using the first of many stream/subject
pairs, so it was rarely excluding rows from the count, and when
it did exclude rows, they were the wrong rows.

This fixes part of #1300, but we may want to keep the issue open.
 2016-07-18 11:21:51 -07:00
Eklavya Sharma e10caf0b65 Gracefully give up when using email-mirror on python 3. 
Running `./manage.py email-mirror` used to fail on python 3
because twisted.mail.imap4 is not python 3 compatible.
Display a message informing the user that email-mirror is not
available on python 3 instead of failing with a traceback.

Also add tools/test-management to py3-backend.
 2016-07-18 09:57:28 -07:00
Steve Howell 6afa22f473 Add LogDictTest.test_to_log_dict(). 2016-07-17 11:25:02 -07:00
Eklavya Sharma da36947400 Change unbuffering strategy in runtornado.py. 
runtornado unbuffers its output using
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0).
This is not python 3 compatible since we can't specify
buffering on a text stream in python 3.  So use the '-u'
option of python when calling runtornado.py to make output
unbuffered.
 2016-07-17 10:31:15 -07:00
Eklavya Sharma 3ee15feeb8 Make twitter tests more readable. 2016-07-17 10:23:25 -07:00
Eklavya Sharma c389d22b5f Fix twitter mock data. 2016-07-17 10:23:25 -07:00
Eklavya Sharma fb1d00fc6c Upgrade python-twitter to latest version. 
Fixes #1145.
 2016-07-17 10:23:25 -07:00
Eklavya Sharma 4130170da8 bugdown: Fix string handling. 
Use unicode strings where needed and correctly encode/decode strings.
 2016-07-17 10:23:25 -07:00
Eklavya Sharma abe7faa2f9 bugdown: Annotate some variables. 2016-07-17 10:23:25 -07:00
Steve Howell 60bd20da61 Remove zerver/lib/query.py (dead code). 2016-07-16 14:51:15 -07:00
Rishi Gupta 09754c9861 Remove JsonableErrors from actions.do_update_message. 
There were a bunch of authorization and well-formedness checks in
zerver.lib.actions.do_update_message that I moved to
zerver.views.messages.update_message_backend.

Reason: by convention, functions in actions.py complete their actions;
error checking should be done outside the file when possible.

Fixes: #1150.
 2016-07-15 13:55:49 -07:00
Rishi Gupta d529a94e4d Add realm setting to time-limit editing of message content. 
This is controlled through the admin tab and a new field in the Realms table.
Notes:
* The admin tab setting takes a value in minutes, whereas the backend stores it
  in seconds.
* This setting is unused when allow_message_editing is false.
* There is some generosity in how the limit is enforced. For instance, if the
  user sees the hovering edit button, we ensure they have at least 5 seconds to
  click it, and if the user gets to the message edit form, we ensure they have
  at least 10 seconds to make the edit, by relaxing the limit.
* This commit also includes a countdown timer in the message edit form.

Resolves #903.
 2016-07-15 13:55:49 -07:00
Eklavya Sharma 29673411df Make backend tests (hackishly) pass on python3.4. 
Replace bot_owner=bot_owner by bot_owner_id=bot_owner.id while
creating a UserProfile object.
 2016-07-15 11:13:33 -07:00
Eklavya Sharma 8c0d7311ac zerver/decorator.py: Use BytesIO to read request. 
Use BytesIO instead of StringIO to get a file-like object on the
request's body.
 2016-07-15 11:13:33 -07:00
Umair Khan 8dabc97d9e Add tests for frontend i18n regexes. 2016-07-15 14:48:09 +05:00
Umair Khan a07a810a2e Fix regex for capturing frontned strings. 
Fixes: #1155
 2016-07-15 14:48:05 +05:00
Steve Howell 9d7716f368 Add message.topic_name() helper function. 
This little helper will facilitate adding a new Topic
table in the future.
 2016-07-14 09:54:56 -07:00
Steve Howell a5b9e3b893 Add test_patch_bot_avatar(). 
Fixes: #1002
 2016-07-13 22:53:46 -07:00
Steve Howell 21fb7693d2 Add test_helpers.client_patch_multipart(). 2016-07-13 22:53:40 -07:00
Steve Howell 538962f3ca Add test_add_bot_with_username_in_use(). 2016-07-13 22:53:40 -07:00
Steve Howell 468faf5724 Add test_add_bot_with_bad_username(). 2016-07-13 22:53:40 -07:00
Steve Howell de4cc80aa0 Add test_add_bot_with_too_many_files(). 2016-07-13 22:53:40 -07:00
Steve Howell e9ddf28b2c Add test_add_bot_with_user_avatar(). 2016-07-13 22:53:40 -07:00
Tomasz Kolek f689fbfa4d Replace old Trello integration with a webhook integration. 
Fixes: #709.

[With tweaks on the documentation by tabbott]
 2016-07-13 21:00:27 -07:00
Steve Howell 701bb7a59f Remove test_helpers.DummySession 2016-07-13 19:08:28 -07:00
Steve Howell 616e49e2e8 Remove test_helpers.DummyTornadoRequest 2016-07-13 19:08:28 -07:00
Steve Howell 0060ea7903 Remove test_helpers.DummyStream 2016-07-13 19:08:28 -07:00
Steve Howell 191ac80475 Remove callback logic from test_helpers.DummyHandler. 2016-07-13 19:08:28 -07:00
Eklavya Sharma d7ea2b8a67 zerver/tests/test_upload.py: Use byte strings. 
Some string literals are incorrectly marked as `str` strings.
Change them to byte strings.
 2016-07-13 16:00:46 -07:00
Eklavya Sharma a7d2dab28f Fix non-deterministic output from Taiga integration. 
Taiga's webhook integration would give output events in a random
order which caused test failures on python 3 (seems like python
3 is more prone to non-deterministic failures).  Fix that by
sorting the outputs obtained from events before concatenating them.
 2016-07-13 16:00:46 -07:00
Eklavya Sharma 5382aeb385 Render PagerDuty message using ujson.dumps. 
Use ujson.dumps to render raw messages sent by the PagerDuty
integration instead of using pprint.pformat.  pprint.pformat
gives different results on python 2 and 3.
 2016-07-13 16:00:46 -07:00
Eklavya Sharma 4f633bcd0b zerver/lib/notifications.py: Fix string encoding/decoding. 
Correctly encode and decode strings in convert_html_to_markdown.
It wasn't possible to use universal_newlines=True since
Popen.communicate doesn't encode/decode strings correctly on
python 2.
 2016-07-13 16:00:46 -07:00
Eklavya Sharma e6502710b6 Change exception.message to str(exception). 
The 'message' attribute in Exception has been deprecated.
It has been removed in python 3.
 2016-07-13 16:00:46 -07:00
Eklavya Sharma 993558c680 Get mogrified SQL queries as text. 
zerver.lib.test_helpers.queries_captured returns a byte string,
while we want the queries to be text.  So decode the captured
queries.
 2016-07-13 16:00:46 -07:00
Tim Abbott c0a6672471 Update path to language_options.json in production. 
The previous code didn't correctly transport language_options.json to
the production environment.
 2016-07-13 12:30:45 -07:00
Umair Khan 395e053ce3 Revert "Revert "Extract reply from email."" 
This reverts commit f1ba3ded42.
 2016-07-13 11:24:18 -07:00
Umair Khan f15dfc69fb Make code Python 3 compatible. 
Make convert_html_to_markdown function Python 3 compatible.
 2016-07-13 11:24:18 -07:00
Tim Abbott 34a251adb1 upload: Fix exception uploaded files with unknown content type. 
It turns out our detected content_type can actually be None, which
meant calling force_text on it broke uploading some files.
 2016-07-13 10:57:55 -07:00
Umair Khan 0aae0eab49 Switch to PyAPNS for sending push notifications. 
Switch to [PyAPNS](https://github.com/djacobs/PyAPNs).

Fixes #538.
 2016-07-13 10:55:07 -07:00
Umair Khan 636466ff8b Fix unreliable tests for missed messages. 
Generate random token using getrandints.

Resolves the proximal issue discussed in #1212.
 2016-07-13 10:51:21 -07:00
Steve Howell c069c8f1e7 Add test_update_api_key_for_invalid_user(). 2016-07-13 07:51:00 -07:00
Steve Howell aea79517f5 Test bot_owner logic in /json/users 
This fixes a small gap in our coverage for get_members_backend.
 2016-07-12 23:01:56 -07:00
Steve Howell 108ce82571 Add test_api_with_insufficient_permissions(). 2016-07-12 22:16:00 -07:00
Steve Howell cee47e7f82 Add test_api_with_nonexistent_user(). 2016-07-12 22:16:00 -07:00
Steve Howell f43be3bd8f Add test_updating_non_existent_user(). 2016-07-12 22:16:00 -07:00
Tim Abbott 647cead0d1 slow queries: Include full log line in slow query log. 
The extra data is useful, and I think this won't make the lines annoying long.
 2016-07-12 19:12:49 -07:00
Tim Abbott 305189956b update_message_flags: Log number of messages updated. 2016-07-12 19:12:49 -07:00
Eklavya Sharma e28b038f1d Use assert_in_response by fixing line-wrapping in templates. 2016-07-12 17:52:47 -07:00
Steve Howell 3923f94a2b Remove unused code path in avatar() endpoint. 2016-07-12 17:37:02 -07:00
Steve Howell 3995a5d8eb Add test coverage for /avatar/<email> 2016-07-12 17:37:02 -07:00
Tomasz Kolek d0066c37ff Add bitbucket2 integration. 
Bitbucket changed the format of their API.  The old format is still
useful for BitBucket enterprise, but for the main cloud verison of
Bitbucket, we need a new BitBucket integration supporting the new API.
 2016-07-12 17:34:34 -07:00
Eklavya Sharma c98c3d5f8d Re-enable some backend tests on python 3. 2016-07-12 14:06:29 -07:00
Eklavya Sharma d740a87d04 zerver/tests: Use unicode strings. 
* Use unicode strings for strings containing non-ASCII characters.
* Decode response content when text output is expected.
 2016-07-12 14:06:29 -07:00
Eklavya Sharma 161c27d0e9 Add methods to AuthedTestCase to test response content. 
Add methods assert_equals_response and assert_in_response to
AuthedTestCase.  These methods make it convenient to check if
a string equals the contents of an HttpResponse's body or if a
string is a substring of the contents of an HttpResponse's body.
 2016-07-12 14:06:29 -07:00
Eklavya Sharma 2080ff6c2a Decode response.content everywhere except in tests. 
response.content is binary data, but code usually assumes it to
be text.  Fix this by decoding response.content where required.
Don't do this in tests yet.
 2016-07-12 14:06:29 -07:00
Steve Howell 45d1eefc52 Add test_create_user_backend(). 
This tests provides full line coverage on the
create_user_backend view.
 2016-07-12 12:29:24 -07:00
Eklavya Sharma 73a3c9fa47 On python 3, skip expected failures in backend tests. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 4868cd9969 zerver/lib/test_helpers.py: Add skip_py3 decorator. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 0a9c600c8b Show skipped tests in test runner. 2016-07-12 09:27:55 -07:00
Eklavya Sharma 958335bdb3 Change all default values in models to unicode. 2016-07-11 21:30:32 -07:00
Tomasz Kolek d7c7279523 Add is_status_message method to Message model. 2016-07-11 12:09:06 -07:00
Tim Abbott e21bc11cfd Extract attachment_url_to_path_id. 2016-07-10 18:07:37 -07:00
Tim Abbott 48ae178d0b Refactor attachment_url_re to be a common value in upload.py. 2016-07-10 18:04:58 -07:00
rahuldeve c5756e4fa4 Update attachment tracking on message update. 2016-07-10 18:01:59 -07:00
Tim Abbott 3647973069 update_message_backend: Reorganize validation checks. 
The new organization more clearly does the permission check first.
 2016-07-10 18:01:59 -07:00
Tim Abbott 05632b68e1 edit: Add validation for setting empty content. 2016-07-10 18:01:59 -07:00
Tim Abbott 0d418d5695 Add some message editing tests for invalid input. 2016-07-10 18:01:59 -07:00
Tim Abbott 211a166abc message edit: Move new topic non-empty check to view. 2016-07-10 18:01:59 -07:00
Eklavya Sharma 0900ca5353 Replace assertItemsEqual by assertEqual. 
This is needed because assertItemsEqual doesn't exist in python 3.
 2016-07-10 17:33:11 -07:00
Rishi Gupta 43c2f35776 Add realm setting to disable message editing. 
This is controlled through the admin tab and a new field in the Realms
table.  This mirrors the behavior of the old hardcoded setting
feature_flags.disable_message_editing.  Partially resolves #903.
 2016-07-10 11:57:24 -07:00
Rishi Gupta 07e7230ae1 models.py: Add post_save to Message. 
Flushes message from cache after a message.save(). Needed for tests
where we directly manipulate Message objects in the database.
 2016-07-10 11:42:18 -07:00
Eklavya Sharma f1ba3ded42 Revert "Extract reply from email." 
This reverts commit f1f48f305e.

The use of sklearn unfortunately caused a substantial slowdown to the
Zulip provisioning process, which didn't seem worth it for a
relatively minor feature.
 2016-07-10 11:30:30 -07:00
Eklavya Sharma 9161ddaee0 zerver/middleware.py: Handle binary data in errors. 
In write_log_line, error_content can be binary_type and
error_content_iter can be a Sequence of binary_type.  Handle
this this in a python 3 compatible way.  Also change annotations
to reflect this fact.
 2016-07-10 11:30:13 -07:00
Steve Howell c671881713 decorators: Extract is_local_addr(). 2016-07-09 17:58:48 -07:00
Steve Howell 8e528569a7 Clean up rate_limit() for deployments that opt out. 
If settings.RATE_LIMITING is False, short circuit rate
limiting earlier in rate_limit().  This change particularly
avoids inspect request.user and possibly spamming the error
log for sites that don't care about rate limiting.
 2016-07-09 17:58:20 -07:00
Eklavya Sharma 801bcdd956 zerver/tests/tests.py: Fix non-deterministic failure. 
Replace occurences of list(d.keys()) by sorted(d.keys()).
 2016-07-09 17:54:54 -07:00
Eklavya Sharma 1f3ce7cf38 zerver/worker/queue_processors.py: Open file in binary. 
Open a file in binary mode instead of text mode.
 2016-07-09 17:54:54 -07:00
Eklavya Sharma 2f2e543a0e zerver/tests/tests.py: Replace assertItemsEqual. 
Replace assertItemsEqual(a, b) by assertEqual(sorted(a), sorted(b))
because assertItemsEqual has been removed in python 3.
 2016-07-09 17:54:54 -07:00
Steve Howell 89105e41d7 Added TestInternalNotifyView. 2016-07-09 07:47:12 -07:00
Steve Howell ce14a3551d Added test_get_client_name(). 2016-07-09 07:37:13 -07:00
Steve Howell 69f18c26fc Improve test_api_key_only_webhook_view(). 
This test now covers all lines of api_key_only_webhook_view(),
including the error logic and rate limiting logic.
 2016-07-09 07:37:13 -07:00
Steve Howell 3d095beb63 Extracted client_is_exempt_from_rate_limiting(). 2016-07-09 07:37:13 -07:00
Steve Howell 49543b9ec4 Added RateLimitTestCase tests. 2016-07-09 07:37:13 -07:00
Steve Howell c680c6a981 Removed unused to_non_negative_float() function. 2016-07-09 07:37:13 -07:00
Umair Khan b7ec66fc96 Remove templates, styles and js through storage. 2016-07-09 07:33:35 -07:00
Umair Khan 043ae8ad65 Upgrade to Django-Pipeline==1.6.8. 2016-07-09 07:09:55 -07:00
Umair Khan 2d243c0703 get_all_templates should not return __init__.py. 2016-07-09 07:09:55 -07:00
Tim Abbott 1e2d38e790 Move stringify_message_dict into to_dict_uncached. 2016-07-08 17:58:56 -07:00
Tim Abbott 72e948d19a Remove now-unused message_cache_key message cache. 
Originally this cache was used to transmit data from Django to Tornado
(and also for general message caching purposes), but now nothing
actually reads from this cache, so we can eliminate it.
 2016-07-08 17:58:56 -07:00
Tim Abbott 8d5ec14b31 cache_helpers: Fill to_dict cache instead of old message cache. 
Apparently, the message cache we were filling was completely useless
and unused, and furthermore, the cache we were filling as part of
restarting the server was also totally useless, since it didn't have
the messages users would be requesting.
 2016-07-08 17:58:56 -07:00
Taranjeet a8a4caf2c0 zerver: Fix lines with length greater than 120. 2016-07-08 11:41:43 -07:00
Umair Khan f1f48f305e Extract reply from email. 2016-07-08 10:58:25 -07:00
Umair Khan 75bd3541ea Add tests for new email policy. 
- Update test_extra_context_in_missed_stream_messages
- Add test_extra_context_in_personal_missed_stream_messages
- Add test_extra_context_in_huddle_missed_stream_messages
 2016-07-08 10:58:25 -07:00
Umair Khan 8538ba8ea8 Remove do_send_missedmessage_events function. 2016-07-08 10:58:25 -07:00
Umair Khan 06355105f5 Missed message emails should come from user email. 
Fixes #448
FIxes #612
 2016-07-08 10:58:25 -07:00
Eklavya Sharma 3e9349df4f zerver/decorator.py: Use force_bytes instead of encode. 
The value type of request.META is str, not text type.
So use force_bytes on the data instead of encode('utf-8').
 2016-07-08 01:24:30 +05:30
Eklavya Sharma 4cf7641ab1 zerver/tests/test_i18n.py: Ignore due to incomplete stubs. 
In python 3, http.cookies has incomplete stubs.
 2016-07-07 12:42:51 -07:00
Eklavya Sharma 6c3f1bb967 beanstalk.py: Encode and decode strings correctly. 2016-07-07 12:42:51 -07:00
Eklavya Sharma 26b8e7357a zerver/views/messages.py: Operate on bytes in highlight_string. 2016-07-07 12:42:51 -07:00
Eklavya Sharma 83640ed0cd runtornado.py: Ignore due to incorrect stubs. 2016-07-07 10:09:35 -07:00
Eklavya Sharma 6fd8906358 rename_stream.py: Fix broken code. 
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
 2016-07-07 10:08:29 -07:00
Eklavya Sharma c679c180f5 enqueue_file.py: Add type hint for mypy. 2016-07-07 10:07:51 -07:00
Eklavya Sharma 896c18a57b email-mirror.py: Make it pass on mypy in python 3. 
* Replace filter by list comprehension.

* Add '# type: ignore' to statements which use attributes from
  modeule `posix`, since stubs for posix are missing on python 3.
 2016-07-07 10:07:28 -07:00
Eklavya Sharma 17cb6e00bd create_stream.py: Fix broken code. 
* get_realm returns None if no matching realm is present, but
  create_stream.py assumed it raises Realm.DoesNotExist.
* encoded/decode strings properly.
 2016-07-07 10:06:39 -07:00
Eklavya Sharma 7956fcbf0d zerver/lib/str_utils.py: Allow specifying encoding. 2016-07-07 10:06:22 -07:00
Eklavya Sharma 63d55bdd86 zerver/views/__init__.py: decode b64encoded ccache. 
Convert b64encoded ccache to `str` before passing to
subprocess.check_call.
 2016-07-07 10:02:08 -07:00
Eklavya Sharma 628e45defc zerver/lib/actions.py: Use text_type in truncate_ functions. 
The functions truncate_content, truncate_body and truncate_topic
are only meant to be used on text strings.  So change its
parameter types from AnyStr to text_type.
 2016-07-07 10:02:08 -07:00
Eklavya Sharma 976858f536 tornado_ioloop_logging.py: Ignore because of missing stub. 
There is no stub for select.epoll on python 3.  So ignore the
statement which uses it.
 2016-07-07 10:02:08 -07:00
Eklavya Sharma 9d2a539aaa zerver/lib/notifications.py: Add hints for mypy. 
Add type hints for mypy using isinstance in assert and if.
 2016-07-07 10:02:08 -07:00
Tim Abbott 98db1d996f email_mirror: Fix some indentation issues. 2016-07-07 10:02:08 -07:00
Eklavya Sharma 5e81a4d93f zerver/lib/email_mirror.py: Improve annotation in python 3. 
Add asserts and if statements to help mypy.
 2016-07-07 10:01:30 -07:00
Eklavya Sharma 4f221c21a0 zerver/lib/email_mirror.py: Improve subject extraction. 
Improve subject extraction in process_message.
 2016-07-07 09:55:23 -07:00
Eklavya Sharma 06a7a6caee zerver/lib/debug.py: Remove from mypy's exclude_py3. 
traceback.print_stack doesn't have a stub yet.  So ignore the
statement which uses it.
 2016-07-07 09:52:24 -07:00
Eklavya Sharma 6bb266d262 bugdown's __init__.py: Add python 3 compatibility. 
* Use Response.text instead of Response.content.
* Make unescaping work on python 3.
 2016-07-07 09:52:24 -07:00
Eklavya Sharma 05046d9288 bugdown's codehilite.py: Fix annotations in python 3 mode. 
Many stubs in xml.etree.ElementTree use Union[str, bytes] as
return type.  Mypy wants us to correctly handle each case.  This
is correct, but not useful for us since we know that we'll always
get str.  So force the return value to text_type, to supress mypy
errors.
 2016-07-07 09:52:24 -07:00
Eklavya Sharma 610f19c791 zerver/lib/camo.py: Type ignore statement with hex encode. 2016-07-07 09:52:24 -07:00
Eklavya Sharma 6a63870136 zerver/decorator.py: Correctly encode/decode strings. 2016-07-07 09:52:24 -07:00
Eklavya Sharma 4e698ab1f6 Remove zerver/lib/parallel.py from mypy's exclude_py3. 2016-07-07 09:52:24 -07:00
Eklavya Sharma ce6ddd574a zerver/lib/test_helpers.py: Replace os.path.walk by os.walk. 
os.path.walk has been removed in python 3.
 2016-07-06 11:49:55 -07:00
Eklavya Sharma 6505cbf2bf zerver/lib/timeout.py: Replace isAlive by is_alive. 
In threading.Thread, isAlive has been removed in python 3.
We should use is_alive instead.
 2016-07-06 11:49:48 -07:00
Eklavya Sharma d8de54abe5 zerver/migrations: Squash an AlterField from 0002 into 0001. 
Squash the AlterField on UserProfile.groups in 0002 into the
AddField in 0001.  This is done to avoid a probable bug in Django,
where running migrations in python 3 sometimes led to a KeyError.
 2016-07-06 11:40:12 -07:00
Umair Khan 47fb293f8a i18n: Add English locale to language list if missing. 
The English locale data is only present if the user has run `manage.py
makemigrations` on the system.
 2016-07-06 11:34:28 -07:00
Eklavya Sharma 2d4dcd1698 Replace bytes by strings in zerver migrations 2, 13, 18. 2016-07-04 12:08:37 -07:00
Eklavya Sharma 6553e16b89 Replace bytes by strings in zerver/migrations/0001. 2016-07-04 12:08:37 -07:00
Umair Khan 035fceb814 Add dynamically loaded language dropdown. 2016-07-04 11:56:02 -07:00
Conrad Dean bbf7a9c801 Annotate zerver/lib/event_queue.py. 2016-07-04 03:01:33 +05:30
Conrad Dean 9812e676f0 Annotate zerver/lib/queue.py. 2016-07-04 03:01:32 +05:30
Eklavya Sharma 192663edcf zerver/lib/narrow.py: Fix annotations. 2016-07-04 03:01:32 +05:30
Eklavya Sharma 95b6e668a7 zerver/lib/actions.py: Fix annotations. 
Change queue_ids from text_type to str.
 2016-07-04 03:01:32 +05:30
Eklavya Sharma 9772a512cb Use abstract types in annotations. 2016-07-04 03:01:32 +05:30
Eklavya Sharma fdf4d67cde zerver/tornado.py: Convert queue_id to str. 2016-07-04 02:58:57 +05:30
Eklavya Sharma de19b24f74 zerver/lib/handlers.py: Fix an annotation. 2016-07-04 02:20:25 +05:30
Eklavya Sharma 4761cc27dd zerver/middleware.py: Fix annotations. 
* Use abstract types where relevant.
* Fix string types.
* Fix annotation of args and kwargs.
 2016-07-04 02:14:42 +05:30
Eklavya Sharma 6d8ba90db9 zerver/models.py: Modify imports for mypy in py3 mode. 
Change 'from zerver.lib import bugdown' to
'import zerver.lib.bugdown as bugdown' to make zerver/models.py
pass mypy check in python 3 mode.
 2016-07-02 10:38:48 -07:00
Eklavya Sharma b902c1ae45 bugdown: Replace raw unicode literal with non-raw. 
Raw unicode literals are disallowed in python 3.
 2016-07-02 10:38:48 -07:00
Kartik Maji f8bb7503e6 Add ability to pin streams to top of the streams sidebar list. 
Based on work by Lauren Long, with some tweaks by tabbott.
 2016-06-30 22:26:09 -07:00
Kartik Maji a32167d921 test_events: Sort subscribers list. 
The subscribers list is appended to in `peer_add` events with not
regard for preserving the ordering, and ordering isn't really
important here, so it seems best to just sort it in these checks.
 2016-06-30 16:52:51 -07:00
Vishnu Ks 1cbd39b768 Unique link generator for realm creation. 2016-06-30 15:52:41 -07:00
Alex Wilson 8c62cff1b7 bugdown: Clean up paths to emojis. 2016-06-30 15:48:02 -07:00
Alex Wilson b040839c76 Add unicode emoji to frontend markdown parser. 
Fixes 2nd half of #1011.
 2016-06-30 15:48:02 -07:00
Alex Wilson 5ec29101eb Add unicode emoji support to bugdown. 
Fixes half of #1011.
 2016-06-30 15:48:02 -07:00
Tim Abbott 0397855fdd test_hooks: Use send_json_payload for private message tests. 2016-06-30 15:12:03 -07:00
Rishi Gupta 17b6d136d5 Add Semaphore webhook integration. 2016-06-29 16:36:19 -07:00
Eklavya Sharma def8cd8e78 Annotate zerver/lib/timeout.py. 2016-06-29 10:41:01 -07:00
Eklavya Sharma 21f789eb05 zerver/lib/session_user.py: Replace Dict by Mapping. 2016-06-29 10:36:50 -07:00
Eklavya Sharma b76dc9bf4e zerver/lib/upload.py: Fix string types. 2016-06-29 20:43:28 +05:30
Tim Abbott abc2c03b0f Fix missing 'current_url' value in registration flow. 
This fixes some tracebacks I got while testing the Zulip htpasswd SSO
functionality.

I think that this stopped working as a result of the Jinja2 migration.
 2016-06-28 23:05:38 -07:00
Tim Abbott 56e7a2f6f3 Annotate zerver.migrations. 2016-06-28 16:38:33 -07:00
Eklavya Sharma 2cba2caa7f zerver/lib/test_helpers.py: Fix string annotations. 2016-06-29 01:54:17 +05:30
Eklavya Sharma d9eb711e5e zerver/tests/test_helpers.py: Use text_type for api_keys. 
Also encode/decode strings appropriately when using api_keys to generate
basic auth header.
Also fix clashing annotations in zerver/tests/test_external.py.
 2016-06-29 01:50:38 +05:30