Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
12,483 Commits 28 Branches 143 Tags 605 MiB
Commit Graph

2965 Commits

Author SHA1 Message Date
Tim Abbott 4a46b879ee import_uploads_s3: Fix setting of content-type. 2016-08-13 11:26:53 -07:00
Tim Abbott 8edc5110cd export: Set re_map_foreign_keys verbose default to False. 
Otherwise, this is super spammy when doing a large import.
 2016-08-13 11:26:16 -07:00
Tim Abbott 8b170665e4 export: Add assertion that ./manage.py exists in current directory. 
Otherwise we'll fail starting the UserMessage export, later on.
 2016-08-13 10:13:11 -07:00
Tim Abbott 856ab48ec6 export: Fix stream export sanity check. 2016-08-13 10:02:34 -07:00
Steve Howell e932b03999 export: Clean up path.join/makedirs for avatars/uploads. 2016-08-13 09:31:49 -07:00
Steve Howell 12c25e7d5c export: Filter attachments by message id. 2016-08-13 09:31:49 -07:00
Steve Howell 0f493d5000 export: Return msg ids from export_partial_message_files(). 2016-08-13 09:31:49 -07:00
Tim Abbott 924d0475c1 cache_helpers: Disable filling message cache. 
The message cache filling script actually used both database and
memcached queries as part of filling the cache (all used to compute
the needed display_recipient values).  We would ideally fix this by
using bulk operations to fill the display_recipient cache, but until
we do so, this cache filler is counterproductive.

I believe this disabling fixes an issue where memcached would get
overloaded and stop handling requests during a server restart on busy
servers.
 2016-08-13 09:27:56 -07:00
Rishi Gupta befd665b91 Update static_header.txt with new license information. 
This is the header file for the minified javascript.
 2016-08-13 00:37:21 -07:00
Rishi Gupta 708b416ca1 Terms of Service: Fix corner cases around new users being created. 
This fixes a few bugs in 7910a6e134,
related to automatically created user accounts.
 2016-08-13 00:19:54 -07:00
Tim Abbott 3bae3cd54d settings: Add option to disable email deliverer. 2016-08-12 19:24:45 -07:00
Steve Howell 0c3e98fa91 export: Introduce attachment.json file. 
Now attachment data gets written to its own json file.  We are
splitting this out so that will be easier for us to cross-check
attachments against messages without holding up writing a lot
of the other realm data.  (message cross-checking is coming soon)
 2016-08-12 18:59:14 -07:00
Steve Howell ea0a7d87c8 export: Refactor how we fetch attachment data. 
This commit doesn't change any behavior; it just moves fetching
attachments out of the Config scheme and into its own method.
This prepares us to start writing attachment data to its own
file and cross-checking against message ids (coming soon).
 2016-08-12 18:59:14 -07:00
Steve Howell fba7a9ca21 export: Unify top-down export configuration. 
We now just have a single configuration get_realm_config() that
handles most of the top-down realm export tables.  (It basically
does everything not related to messages or uploads/avatars.)

Unifying the configs allows us to be more strict in our
configuration about checking for anomalies.  In the future
we may need to loosen up some of those restrictions again,
but for now we are picky and paranoid.
 2016-08-12 15:27:23 -07:00
Steve Howell 5a5353b846 export: Fetch stream data only for stream recipients. 
Fetch stream data only for stream recipients, instead of
getting streams via realm_id.

(This change is kind of moot for now, since our stream recipients
include all possible stream recipients in the realm, but this
sets us up for when we start restricting users that we export
within the realm.)
 2016-08-12 15:27:22 -07:00
Steve Howell 7a429d1e30 export: Add sanity_check_stream_data(). 2016-08-12 15:27:22 -07:00
Steve Howell ec86e475b4 export: Add Config.post_process_data 2016-08-12 15:12:01 -07:00
Steve Howell 0c2c331905 export: Flip how we fetch stream subscription data. 
We now get stream subscriptions BEFORE stream recipients.
 2016-08-12 15:12:01 -07:00
Steve Howell 70a916aae3 export: Flip how we fetch user subscription data. 
We now get user subscriptions BEFORE user recipients.
 2016-08-12 15:12:01 -07:00
Steve Howell 2a2ce6ada1 export: Remove hard-to-maintain code comment. 
Subsequent changes are gonna make the top-down/bottom-up comment
no longer valid.
 2016-08-12 15:12:01 -07:00
Steve Howell 90494cfec2 export: Note user-owned bot limitations. 2016-08-12 15:10:49 -07:00
Tim Abbott ed01842f95 notifications: Fix missed message emails with multiple senders. 
Previously, missed message emails with multiple senders would
incorrectly have a "," outside the quoted sender name part of the from
address string, resulting in confusing email output.
 2016-08-12 12:28:06 -07:00
Tim Abbott 386c6c2a31 settings: Refactor SYSTEM_ONLY_REALMS configuration. 2016-08-12 12:12:53 -07:00
Tim Abbott f0e495831e forms: Fix check for whether terms of service are enabled. 2016-08-12 11:04:45 -07:00
Steve Howell 6fdd42c08b export: Create convenient soft links. 2016-08-12 10:48:33 -07:00
Steve Howell 70b68ddcc3 export: Use a config for export_single_user(). 2016-08-12 10:37:41 -07:00
Steve Howell c69a5bdec3 export: Handle more tables via export_from_config(). 
This commit introduces the ability to do custom fetches
and to essentially use temp tables for intermediate results.

(The temp table stuff deals with recipients/subscriptions
having three different flavors--user, stream, and huddle.)
 2016-08-12 10:37:35 -07:00
Steve Howell f471a1779e export: Handle simple exports with export_from_config(). 
This handles the simple tables that don't need custom fetches.
 2016-08-12 09:54:57 -07:00
Steve Howell 682155778d export: Add export_with_config(). 
Subsequent commits will start to use this.
 2016-08-12 09:54:57 -07:00
Steve Howell b0e6d20321 export: Write stats.txt for `./manage.py export <realm>`. 2016-08-12 09:06:10 -07:00
Rishi Gupta 7910a6e134 Terms of Service: Add ability to update TOS and have users re-sign. 
Most directly useful for the migration to zulipchat.com.

Creates a new field in UserProfile to store the tos_version, as well as two
new settings TOS_VERSION and FIRST_TIME_TOS_TEMPLATE. We check for a version
mismatch between what the user has signed and the current
settings.TOS_VERSION whenever the user hits the home page, and redirect them
if needed.

Note that accounts_accept_terms.html and
zerver.views.accounts_accept_terms were unused before this commit
(they date from c327446537)
 2016-08-11 23:37:48 -07:00
Tim Abbott 11cac86a58 settings: Add support for specifying a remote redis password. 2016-08-11 16:54:22 -07:00
Steve Howell df3aa39be3 export: Extract write_data_to_file(). 2016-08-11 15:51:22 -07:00
Steve Howell f29b32bbb2 export: Clarify message exporting code. 
The function to create the message partial files has been
renamed to export_partial_message_files().  It now gets its own
list of user profile ids and recipient ids from the response,
so that we can de-clutter do_export_realm().
 2016-08-11 15:51:22 -07:00
Steve Howell 5cd915694a export: Extract launch_user_message_subprocesses(). 
This is the last in a series of commits that makes it
so that do_export_realm() mostly delegates work out
to other functions.
 2016-08-11 15:21:30 -07:00
Steve Howell b383f5ca5d export: Extract fetch_user_profile_cross_realm(). 2016-08-11 15:21:30 -07:00
Steve Howell fee2106c6f export: Extract fetch_huddle_objects(). 
This also removes the dead codepath for include_private=False.
 2016-08-11 15:21:30 -07:00
Steve Howell a6235f6a60 export: Add comments to export_single_user(). 
(This is a bit of a prefactoring to hopefully create a nice
diff in a subsequent commit.)
 2016-08-11 15:21:30 -07:00
Steve Howell 6e7fe76cf4 export: s/avatar_bucket/processing_avatars 
The name avatar_bucket was confusing for a boolean, and
in some places it was used for non-S3 paths.

I considered the more concise 'is_avatar', but that
was still confusing when you are processing multiple
files, because you think it's a calculated property
on one file instead of an overall codepath switch.

I also considered splitting up some functions, but
there is a lot of common logic between handling
file uploads and avatars that's not trivial to extract
into helpers, especially on the S3 side.
 2016-08-11 15:21:30 -07:00
Steve Howell 3dab366733 export: Clean up names of upload/avatar export functions. 
I did some minor moving around of code that made us have
one fewer function without any additional conditional
logic. The names are more explicit about saying
"from_local" and "from_s3".  Also, there is less clutter
now in do_export_realm(), which is evolving into more of
a dispatcher and less of a worker.
 2016-08-11 15:21:30 -07:00
Steve Howell 9582a83d10 export: Fix usage in export_single_user command. 2016-08-11 15:21:30 -07:00
Steve Howell d62a351107 export: Add sanity_check_output(). 2016-08-11 15:21:30 -07:00
Tim Abbott 95aee366c8 Enable local browser error reporting if ZILENCER_ENABLED. 2016-08-11 13:49:18 -07:00
Umair Khan af9f308f0e Send double columned language list in page_params. 2016-08-10 16:19:28 -07:00
Umair Khan 73e5289736 Add default_language_name in page_params. 2016-08-10 16:19:28 -07:00
Umair Khan ddaf1a2b67 Return unformatted list from get_language_list. 2016-08-10 16:19:28 -07:00
Steve Howell 06b0df5efc export: Remove spurious select_related() call for Client. 2016-08-10 14:16:17 -07:00
Steve Howell cb59a11f0a export: Extract get_primary_ids(). 2016-08-10 14:16:17 -07:00
Steve Howell 90e9083b81 export: Extract filter_by_realm(). 2016-08-10 14:16:17 -07:00
Steve Howell 4b6b1b8ad4 export: Extract filter_by_users(). 2016-08-10 14:16:17 -07:00
Steve Howell db9edfce34 export: Use DATE_FIELDS in fix_datetime_fields(). 
Now we only call this once per table and use DATE_FIELDS to
look up the data fields.
 2016-08-10 14:16:17 -07:00
Steve Howell 35c59fc4d7 export: Clean up export_messages(). 
This is pretty minor cleanup, but it makes it a little more
explicit what we're writing to the shard file, and it allows
us to use a more specific mypy type when calling
floatify_datetime_fields.
 2016-08-10 14:16:17 -07:00
Steve Howell 1d1f36c0b8 export: Always use subprocesses to export UserMessage. 
We no longer have an in-process code path to export
UserMessage rows.  We want to only maintain the
subprocess code, which we'll always use in production,
and which will work fine in dev.
 2016-08-10 14:16:17 -07:00
Steve Howell 78bbefbf94 export: Create import_attachments. 2016-08-10 14:16:17 -07:00
Rag Sagar 2fef36f15a Add realm-level default language setting. 
Adds a new field default language in the zerver_realm model.
This realm level default language will be used as default language
for newly created users. Realm level default language can be
changed from the administration page.

Fixes #1372.
 2016-08-09 17:38:29 -07:00
Steve Howell 7ec6a394fe export: Filter Attachment objects by realm. 2016-08-09 16:47:14 -07:00
Steve Howell cecfaa7761 export: Extract import_message_data(). 2016-08-09 16:47:14 -07:00
Steve Howell 5386ed280e export: Extract update_id_map(). 
We also use a vanilla dictionary instead of a defaultdict, so
that we explicitly initialize what tables are being re-mapped.
 2016-08-09 16:47:14 -07:00
Steve Howell 217ef8a4d2 export: Split fix_foreign_keys() into two functions. 
We now have convert_to_id_fields for the simple case, and
re_map_foreign_keys for the more complex case. I also
renamed some parameters and variables.
 2016-08-09 16:47:14 -07:00
Umair Khan e1090118c5 Percent translated should be an integer. 2016-08-09 16:19:07 -07:00
Rag Sagar 5771e2eb25 Move i18n functions from zerver/views/__init__.py to zerver/lib/i18n.py 2016-08-09 16:10:24 -07:00
Rag Sagar 28f0079432 Use a constant to set UserProfile default_language field max_length attribute. 2016-08-09 16:10:24 -07:00
Steve Howell dd88ffccfd export: Extract make_raw() in lib/export.py. 2016-08-09 15:58:27 -07:00
Steve Howell 09fa343bdd export: Use DATE_FIELDS in floatify_datetime_fields. 
This avoids a little bit of code duplication, plus it should
make it a little easier to add new date fields in the future.
 2016-08-09 15:58:27 -07:00
Steve Howell c14ab3c91f export: Add annotations to zerver/lib/export.py. 
I also fixed some small things like removing unnecessary return
statements, and adding a TODO.

In some cases I explicitly cast stuff at run-time to set() or
str() to appease mypy, as well as make it clear to somebody
reading the code that the callee might not respect ordering
or tolerate unicode.
 2016-08-09 15:58:27 -07:00
Steve Howell f18cc4ae3a export: Added export_avatars_local_helper(). 2016-08-09 15:58:27 -07:00
Umair Khan 86920a297c Mock out loggers in push notification tests. 
Fixes: #1575
 2016-08-09 22:16:09 +05:00
Umair Khan 7a84f12a38 Verify mock calls without strict order. 
In push notification tests, call assert_has_calls with `any_order=True`
so that the order of the calls does not matter.
 2016-08-09 22:15:37 +05:00
Tim Abbott 42b63f5caa test_templates: Fix failure to report a user-facing error. 2016-08-09 09:54:44 -07:00
Tim Abbott f3365f4089 test_templates: Add integrations_dict to context. 
This fixes the fact that this test was (silently) failing.
 2016-08-09 09:54:36 -07:00
Umair Khan fe812a89bf mypy: Type of remote_username should be text_type. 2016-08-09 09:19:24 -07:00
Umair Khan 8c249f859e Push Notification: Add tests for GCM. 2016-08-09 16:13:02 +05:00
Umair Khan acd76eb604 Annotate push_notifications.py. 2016-08-08 15:17:02 -07:00
Umair Khan 9c13e9a072 Annotate test_push_notifications. 2016-08-08 15:17:02 -07:00
Umair Khan 35dd1ffb13 Pass password as string to authenticate. 2016-08-08 15:17:01 -07:00
Umair Khan 266c791e64 Email should be a unicode. 2016-08-08 15:17:01 -07:00
Umair Khan 503ad774f5 Tokens should be unicode. 2016-08-08 15:17:01 -07:00
Umair Khan 0dd7d0dda1 push_notifications: Change name to 'identifier'. 2016-08-08 15:15:29 -07:00
Umair Khan fdbd132ba4 Annotate zerver/context_processors.py. 2016-08-08 15:15:29 -07:00
Umair Khan 00eab4e526 Annotate zerver/views/__init__.py. 2016-08-08 15:15:29 -07:00
Umair Khan 25dd1aeb5c Annotate zerver/lib/actions.py. 2016-08-08 15:13:29 -07:00
Umair Khan ef62506fb1 Annotate zerver/models.py. 2016-08-08 15:13:29 -07:00
Umair Khan e34b7a0691 Fix type annotation in message_cache_items. 2016-08-08 15:13:29 -07:00
Umair Khan ec910d96fc Annotate zerver/storage.py. 2016-08-08 15:12:12 -07:00
Umair Khan 0dfc757447 Annotate test_auth_backends.py. 2016-08-08 15:12:12 -07:00
Tim Abbott 6264ff7039 Add new Zulip realm import/export tool. 
The previous export tool would only work properly for small realms,
and was missing a number of important features:
* Export of avatars and uploads from S3
* Export of presence data, activity data, etc.
* Faithful export/import of timestamps
* Parallel export of messages
* Not OOM killing for large realms

The new tool runs as a pair of documented management commands, and
solves all of those problems.

Also we add a new management command for exporting the data of an
individual user.
 2016-08-08 14:58:18 -07:00
Tim Abbott e7adaf8db1 utils: Add mkdir_p implementation. 2016-08-08 14:36:50 -07:00
Tim Abbott 54a6bdc3ad views: Fix mypy annotations in get_invitee_emails_set. 
Apparently this buggy annotation slipped through into master.
 2016-08-05 15:04:16 -07:00
Tomasz Kolek 64b5d0ce64 Add support for inviting emails in "name" <email> format. 
Often, users will copy email addresses with a name (rather than pure
email addresses) into the Zulip "invite users" UI.  Previously, that
would throw an error.

This change also adds a get_invitee_emails_set function for parsing
emails content and a test suite for this new feature.

Fixes: #1419.
 2016-08-05 14:48:04 -07:00
Umair Khan d33fc0046f Add APNS feedback server tests. 2016-08-05 10:36:57 -07:00
Umair Khan 33450d1a0b Add APNS push notification tests. 2016-08-05 10:36:57 -07:00
Umair Khan 40054ce26c Add APNS response listener test case. 2016-08-05 10:36:57 -07:00
Umair Khan df8fc30b75 Create MockRedis class. 
Redis is not available on Travis and creating a mock Redis server looks
easier than using `mock` module.
 2016-08-05 10:36:57 -07:00
Umair Khan 2ab796aeb5 Add tests for APNS message. 2016-08-05 10:36:57 -07:00
Umair Khan 882bb5558b Push to APNS in a loop. 2016-08-05 10:36:57 -07:00
Umair Khan 4e8ca0a326 Make sure apns response code is an integer. 2016-08-05 10:36:57 -07:00
Umair Khan 5c41eae7ec Don't send connection to apns response listener. 2016-08-05 10:36:57 -07:00
Umair Khan ffaf7cb2ba Create get_connection function for APNS. 2016-08-05 10:36:57 -07:00
Umair Khan 1d2bdfdfb1 Save user id instead of user profile. 2016-08-05 10:36:57 -07:00
Umair Khan 0cac1aa135 For APNS frame identifier generate random int. 2016-08-05 10:36:57 -07:00
Umair Khan 0b5dc56b8d Redis key expiry should be an integer. 2016-08-05 10:36:57 -07:00
Umair Khan cbd53fbac8 Don't import OrderedDict. 2016-08-05 10:36:57 -07:00
Umair Khan 0b88deb640 APNS_SANDBOX should be a bool. 
Fixes: #1480
 2016-08-05 10:36:57 -07:00
Tim Abbott dcb92d41b8 Annotate test_messages.py. 2016-08-04 15:57:04 -07:00
Tim Abbott 39f1024740 test_messages: Fix remaining mypy errors. 2016-08-04 15:57:03 -07:00
Tim Abbott 88550edd9e Annotate queue_processors.py. 2016-08-04 15:57:03 -07:00
Tim Abbott 4e8054b84a Refactor queue worker tests to match actual API. 2016-08-04 15:57:03 -07:00
Tim Abbott 1e54897ca7 process_queue: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 4ddc567c50 views/__init__: Add missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 641aa08721 compilemessages: Add mypy annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a787dee48b makemessages: Add missing annotations and fix string types. 2016-08-04 15:57:03 -07:00
Tim Abbott 2da6b6bfd8 views.messages: Add some more annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 2f13b0b18a actions: Fix various string annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 4f782bc186 test_events: Fix annotation gaps. 2016-08-04 15:57:03 -07:00
Tim Abbott 2eb5baa024 zendesk: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott d7d8d6b4d2 parallel: Add missing annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott d00719f204 Annotate IFTTT tests. 2016-08-04 15:57:03 -07:00
Tim Abbott d449f6ba72 Annotate bitbucket2 tests. 2016-08-04 15:57:03 -07:00
Tim Abbott 5253c044e7 test_helpers: Add some missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a7086d3d8a trello: Fix accidentally commented type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 4840e493b2 decorator: Add missing annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott ef5b5bb45b views/streams: Add missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 92062b9526 process_queue: Add missing annotation. 2016-08-04 15:57:02 -07:00
Tim Abbott cc11229377 test_auth_backends: Add missing type annotations. 2016-08-04 15:57:02 -07:00
Tim Abbott 0689485666 Add lint check for malformed type annotations missing :. 
Also fix the 2 annotations that weren't being checked because of this.
 2016-08-04 15:53:23 -07:00
Tim Abbott af823f7a76 Annotate circleci integration. 2016-08-04 15:53:23 -07:00
Tim Abbott 55611cd21a Annotate test_i18n.py. 2016-08-04 15:53:23 -07:00
Tim Abbott 028efdcb78 integrations: Add detailed module docstrings. 2016-08-04 12:13:25 -07:00
Tomasz Kolek 09d081d9bd Replace hardcoded integration-lozenges to generating ones. 2016-08-04 11:41:18 -07:00
Tomasz Kolek 0e33b8bd4d Add integrations library. 
Define Integration and WebhookIntegration classes.
Change webhook part of integration's guide.
Replace hardcoded webhook urls to generating
based on WEBHOOKS list.
 2016-08-04 11:39:20 -07:00
Umair Khan b38ca0c690 Add tests for GitHub team and organization auth. 2016-08-03 12:01:15 -07:00
Umair Khan fc1d8e217f Don't pass `return_data` to mock functions. 
GitHubAuthBackend already passes the `return_data` keyword argument
to the `do_auth` function.
 2016-08-03 12:01:15 -07:00
Tim Abbott 5f275c9868 Re-enable mobile app advertisements in /apps.html. 2016-08-02 16:45:11 -07:00
Rishi Gupta 948ea7663c ModelReprMixin: Fix handling of missing __unicode__ function. 
The old behavior was to raise an exception, but Django was catching
the exception and doing unexpected things. For instance, in the
manage.py shell, printing out a ModelReprMixin object (with
__unicode__ not implemented) would result in nothing being printed,
rather than it raising a error or otherwise alerting the programmer as
to what was going on.
 2016-08-02 11:46:01 -07:00
Umair Khan 2f6293027d Add test for empty email gateway pattern. 2016-07-31 20:39:09 -07:00
Umair Khan 2c07f1b19a Use NOREPLY_EMAIL_ADDRESS if email gateway not enabled. 
This fixes a regression where missed message emails would not be sent
at all in the event that EMAIL_GATEWAY_PATTERN was unset.

The overall experience still isn't great, but it's better than crashing.

Fixes: #1411

[commit message expanded by tabbott]
 2016-07-31 20:38:18 -07:00
Rag Sagar b9f2397073 Add validation and tests for default language setting. 2016-07-31 17:44:10 -07:00
Christie Koehler 31efecf03d Add HelloWorld webhook integration + Walkthough. 
Improves webhook integration docs and Hello World webhook.  Includes
many suggested improvements from @timabbott and @tomaszkolek.
 2016-07-29 21:38:40 -07:00
Steve Howell 2a37dafcbb Write untested_url_report.txt. 2016-07-29 21:26:51 -07:00
Steve Howell 8b13530712 Add tools/analyze-url-coverage 2016-07-29 21:23:33 -07:00
Steve Howell 21f83afe3a Add --url-coverage option to ./tools/test-backend. 2016-07-29 21:23:33 -07:00
Tim Abbott 297566510c test_activity: Increase expected query count by 1. 
This is required because the Zulip UserProfile caching logic has
changed with 8e5e6a06f2, and sometimes
the current user will not be in the cache.
 2016-07-29 21:15:10 -07:00
Tim Abbott f7083b4079 test_templates: Fix Python 3 compatibility. 2016-07-29 21:11:27 -07:00
Tim Abbott 51672f9ddc Remove now-unused zulip_com template context variable. 2016-07-29 20:47:42 -07:00
acrefoot b7ccf64c79 Move Zulip's TOS to TERMS_OF_SERVICE markdown file. 
This reimplements zulip.com/terms using the new markdown system.
 2016-07-29 20:47:42 -07:00
acrefoot e568dbc76f Add TERMS_OF_SERVICE setting. 
This new setting allows the server administrator to add a custom
Terms of Service page by supplying the path to a markdown file.

Also adds a test.
 2016-07-29 20:47:31 -07:00
Tim Abbott 700e803840 settings: Add new DEBUG_RATE_LIMITING setting. 
This exists primarily in order to allow us to mock settings.DEBUG for
the purposes of rate limiting, without actually mocking
settings.DEBUG, which I suspect Django never intended one to do, and
thus caused some very strange test failures (see
https://github.com/zulip/zulip/pull/776 for details).
 2016-07-29 19:32:02 -07:00
Eklavya Sharma 5691ca61b0 Add tests for email_mirror management command. 2016-07-29 19:13:52 -07:00
Steve Howell 9fb071947d Remove expected_run_time from @slow test decorator. 2016-07-29 16:41:21 -07:00
Steve Howell 438a118ea5 Simplify enforce_timely_test_completion(). 
This will lead to minor differences in the warnings that
people see when they run tests that are slow.  We call out
the slowness a little more clearly from a visual standpoint,
and we simplify the calculation of the slowness threshold.

We still allow more time for tests with the `@slow` decorator
to run, but we don't use their expected_run_time.
 2016-07-29 16:41:21 -07:00
Steve Howell f67b56702a Remove @slow decorators for tests that are no longer slow. 
These tests aren't exactly zippy, but they don't even make
the top 100 slow tests.
 2016-07-29 16:41:21 -07:00
Steve Howell 4556bf528f Add --report-slow-tests option to tools/test-backend. 2016-07-29 16:41:20 -07:00
Tim Abbott 6b2b7ab3ff Rename GitHubBackend to GitHubAuthBackend for consistency. 2016-07-29 12:49:10 -07:00
Umair Khan d43b031a32 Add tests to verify GitHub backend. 2016-07-29 12:49:10 -07:00
Umair Khan 80d62de40a Add GitHub authentication. 
Fixes: #1042
 2016-07-29 12:49:10 -07:00
Umair Khan 7531d9679b Sort language dropdown list. 2016-07-29 11:58:41 -07:00
Umair Khan 2edb79776e Calculate translation percentages. 
Fixes: #1397
 2016-07-29 11:58:41 -07:00
Umair Khan 0aa72cb347 Point registration to correct url. 
While logging through GitHub if the realm of the user doesn't
exist then we are redirected to registration page but the action
points to the complete url of the GitHub oAuth overflow.
 2016-07-28 15:27:22 -07:00
Eklavya Sharma 3b3b5c7c16 Rename management command email-mirror to email_mirror. 
All other zulip management command names have underscores, so
rename email-mirror to email_mirror.

This will also make it possible to import this module, which will
help in writing tests for it.
 2016-07-28 14:52:09 -07:00
Eklavya Sharma b98b049377 Fix string handling in email_mirror queue_processor. 2016-07-28 14:52:09 -07:00
Eklavya Sharma 6e9bc44123 Re-implement email-mirror using imaplib. 
Switch from twisted to imaplib to gain python 3 compatibility and
make code easier to understand.
 2016-07-28 14:52:09 -07:00
Tim Abbott 832951d2dd Fix IFTTT integration documention on 'content' field. 2016-07-28 14:05:45 -07:00
Steve Howell ffe364c719 Don't pass UserProfile object to client_put(). 
This was in AdminCreateUserTest.test_create_user_backend().
For end to end tests we are logged in, and we need to verify
that our decorators add UserProfile to the parameters of
the view on our behalf, so that we don't get false positives.

In an upcoming commit, we will want to be able to serialize
the parameters for client_put to produce url coverage reports,
so that is another reason not to pass in the UserProfile
object.  (That was how this was discovered.)
 2016-07-28 13:57:28 -07:00
Umair Khan 8e5e6a06f2 Delete cache entry for user profile. 
Our flush functions update user profile cache entries which can cause
confusing race conditions (see e.g. #1257).  To resolve this, we move
all the user_profile flush functions to delete the entry instead of
updating it -- it will then be fetched as part of the next request
that needs to access the user object.

There are still races here, and there is perhaps an argument that a
better fix for this would be to re-fetch the object and then put it
into the cache, but this resolves the main cache correctness problem
we had with the previous implementation.

Fixes: #1322.
 2016-07-28 13:43:14 -07:00
Steve Howell 713797a65c Add lint checks for self.client.{get,patch,put,post,delete}. 2016-07-27 20:50:54 -07:00
Steve Howell 985de51903 Use self.client_delete() wrapper in more places. 2016-07-27 20:49:33 -07:00
Steve Howell 6937f63fd5 Add client_get() test helper. 2016-07-27 20:49:33 -07:00
Steve Howell 38f2a2f475 Add client_post() test helper. 
This makes us more consistent, since we have other wrappers
like client_patch, client_put, and client_delete.

Wrapping also will facilitate instrumentation of our posting code.
 2016-07-27 20:49:32 -07:00
Steve Howell 51bae8abc4 Remove stub() test helper. 2016-07-27 14:12:10 -07:00
rahuldeve ec6c9ba436 Add test for removing an uploaded file via message editing. 
Fixes #1373.
 2016-07-27 14:10:15 -07:00
rahuldeve 4267dcbb68 Change the way referred attachments are removed on message update. 
This fixes incorrect behavior when two or more attachments are removed
in the same message.
 2016-07-27 14:08:59 -07:00
Tim Abbott df525ad1c5 Remove old MitUser model and related code. 
The MitUser model caused a constant series of little problems for
users with mit.edu email addresses trying to sign up for different
Zulip servers.

The new implementation just uses conditionals on the realm object when
selecting the confirmation template to use.
 2016-07-26 20:30:12 -07:00
Tim Abbott ed6c134cf4 Clean up and document clean_email code path. 2016-07-26 20:30:12 -07:00
Tim Abbott 375551aaa6 Clean up most hardcoding of mit.edu domain checks. 
This moves all this code to be gated on a few virtual realm settings.
 2016-07-26 20:30:12 -07:00
Tim Abbott 70543e059a bugdown: Remove hardcoding of mit.edu for zephyr_mirror realm config. 2016-07-26 20:30:12 -07:00
Tim Abbott c17676b00c Cleanup MIT hardcoding for disabling presence. 2016-07-26 20:30:12 -07:00
Tim Abbott 884f50cdd7 validate_user_access: Assert user_profile is not None. 
This function is only called in cases where user_profile isn't None,
and the code reads better if we just check that first rather than
checking it on every line that accesses user_profile.
 2016-07-26 20:30:12 -07:00
Tim Abbott d1adbd798b get_subscribers_to_streams: Pass through requesting_user. 2016-07-26 20:30:12 -07:00
Tim Abbott 691ad7fcfc maybe_get_subscriber_emails: Pass user_profile through. 2016-07-26 20:30:12 -07:00
Tim Abbott f790a04102 Remove unused get_subscriber_ids. 2016-07-26 20:30:12 -07:00
Tim Abbott 1771b8275c Remove unused get_other_subscriber_ids. 2016-07-26 20:30:12 -07:00
Steve Howell eb11e02309 Add MirroredMessageUsersTest. 2016-07-26 20:01:36 -07:00
Steve Howell 3fa745ebe3 Remove untested code in is_public_stream(). 
We were calling valid_stream_name(), but more appropriate
validation checks for stream names happen elsewhere in the codepath.
 2016-07-26 18:10:00 -07:00
Steve Howell 81cd9d33e4 Add test_non_string_narrow_operand_in_dict(). 2016-07-26 18:10:00 -07:00
Tim Abbott b2d1c18268 views: Move push token views to new push_notification.py. 2016-07-26 14:44:24 -07:00
Tim Abbott 6dc5681171 views: Move presence views to presence.py. 2016-07-26 14:29:32 -07:00
Tim Abbott 62affa53c9 views: Move update_realm to new realm.py views file. 2016-07-26 14:29:24 -07:00
Umair Khan 7c138c6e33 Import is_inactive from proper file. 2016-07-26 12:26:55 -07:00
Eklavya Sharma e0dbaf1031 Use universal_newlines=True in subprocess.check_output. 
In python 3, subprocess uses bytes for input and output if
universal_newlines=False (the default).  It uses str for input and
output if universal_newlines=True.

Since we mostly deal with strings, add universal_newlines=True to
subprocess.check_output.
 2016-07-26 12:06:41 -07:00
Eklavya Sharma 5ce80eb4ba Update types of images accepted as avatars. 
* Reject SVG in frontend because backend doesn't accept it.
* Add JPEG format in backend tests for avatar upload.
 2016-07-26 16:36:47 +05:30
Eklavya Sharma a2a8b54f6e Upgrade pillow. 
Update test images because pillow's resize algorithm has changed.
 2016-07-26 16:36:07 +05:30
rahuldeve 5ed2b01bc7 upload: Fix single attachments not being claimed on message edit. 
Fixed an attachments not being claimed if just one new attachment is
added (aka the common case).
 2016-07-25 17:56:27 -07:00
rahuldeve e06d5fbec9 Pass Message object instead of message dict to do_claim_attachments. 2016-07-25 17:54:36 -07:00
rahuldeve c61d149837 Pass Message object instead of message dict to claim_attachment. 2016-07-25 17:51:17 -07:00
Steve Howell 57c436b32d Add TODOs for #1379 (bugdown mutual dependency) 2016-07-25 14:59:32 -07:00
Ashish Kumar 8398dac025 Fix 'Invalid stream' error message in get_subscription_or_die(). 2016-07-25 14:48:48 -07:00
Ashish Kumar 56048919f5 Increase test coverage of streams API by covering JSON errors. 2016-07-25 14:48:48 -07:00
Ashish Kumar 84566c750f Add test_successful_subscriptions_add_with_announce(). 2016-07-25 14:45:29 -07:00
Ashish Kumar a143677ea2 Add test_json_get_subscribers(). 2016-07-25 14:45:28 -07:00
Ashish Kumar 9104468926 Add test_existing_subscriptions_autosubscription(). 2016-07-25 14:43:27 -07:00
Steve Howell 2846e23cfc Extracted test helper: get_and_check_messages(). 
This new helper combines two old helpers, one of which was misnamed
and the other of which was always called after the first, so it
made sense to just combine the helpers.

Fixes: #1386
 2016-07-24 19:43:56 -07:00
Steve Howell 1da9325476 Require non-empty full_name or password in json_change_settings 2016-07-23 14:53:15 -07:00
Steve Howell 14717f414c Add default for full_name in json_change_settings. 2016-07-23 14:53:15 -07:00
Steve Howell f27ad0d800 Add test_changing_nothing_still_returns_success(). 
(in ChangeSettingsTest)
 2016-07-23 14:53:15 -07:00
Steve Howell 728ac09df9 Remove ChangeSettingsTest.post_with_params(). 
The individual tests now explicitly show all the parameters
they are passing to the endpoint, which makes them easier
to troubleshoot.
 2016-07-23 14:53:15 -07:00
Steve Howell f01b10f97a Improve test coverage for use_first_unread_anchor parameter. 
This commit adds these two tests:

test_use_first_unread_anchor_with_some_unread_messages
test_use_first_unread_anchor_with_no_unread_messages

The new tests add coverage to the conditional logic in
get_old_messages_backend() that looks at first_unread_result
when use_first_unread_anchor is set to True.
 2016-07-23 14:39:58 -07:00
Steve Howell 08f6ae7c14 Improved and renamed test_use_first_unread_anchor(). 
The test is now called test_use_first_unread_anchor_with_muted_topics().
Before this commit, the test exercised setting
use_first_unread_anchor to True, but it didn't inspect the
most relevant query affected by the flag.  Now it does.

This test is still kind of hard to read, and it's far from ideal,
but I'm reluctant to remove it from the test suite.
 2016-07-23 14:39:58 -07:00
Steve Howell ee2007136d Test "empty list" logic in exclude_muting_conditions(). 2016-07-23 14:39:58 -07:00
Eklavya Sharma 196e021a7f Add automated tests for zerver/lib/type_debug.py. 2016-07-23 14:38:28 -07:00
Eklavya Sharma 5b06edc402 zerver/lib/type_debug.py: Handle container types better. 2016-07-23 14:35:39 -07:00
Eklavya Sharma a632c71cce Add zerver/lib/type_debug.py. 
This file has a decorator print_types, which makes the decorated
function print the types of its arguments whenever it is called.
 2016-07-23 14:35:39 -07:00
Taranjeet Singh 9bea1f46c7 zerver/views/webhooks/bitbucket2.py: Fix line with length greater than 120. 2016-07-23 14:33:11 -07:00
Tim Abbott eba2d470dc Fix removing attachments when editing a message. 
Fixes #1373.
 2016-07-22 14:48:47 -07:00
Tim Abbott 4de203a23f Fix MIT users joining a Zulip realm. 2016-07-22 14:27:36 -07:00
Eklavya Sharma 09e4e5aea7 Add links after `type: ignore`s. 2016-07-22 13:59:31 -07:00
Eklavya Sharma 674f6999e1 Improve annotations of decorators. 2016-07-22 11:14:33 -07:00
Eklavya Sharma 4b7d94564a zerver/views/streams.py: Annotate variables to help mypy. 2016-07-22 11:14:33 -07:00
Eklavya Sharma 07e36d87a2 Remove return where no return value is expected. 
zerver.lib.notifications.send_future_email has return type None.
So replace `return send_future_email(...` by `send_future_email(...`.
 2016-07-22 11:14:33 -07:00
Krit Karan a61ef3bc0e Reformat the content variable in ``do_refer_friend()``. 
Fixes #1162.
 2016-07-22 10:36:35 -07:00
Eklavya Sharma c59740385d zerver/lib/str_utils.py: Replace ValueError by TypeError. 
When a parameter of a wrong type is passed to a `force_*` method
in str_utils.py, raise a TypeError.
 2016-07-21 23:14:00 +05:30
Umair Khan dce48d58b6 Cache translations. 
Fixes: #1158
 2016-07-21 10:23:31 -07:00
hackerkid b6620cfa57 Replace UserProfile with HttpRequest in logged_in_and_active. 2016-07-20 20:08:00 -07:00
Tim Abbott c791037166 narrow: Document BuildNarrowFilterTest. 2016-07-20 14:18:40 -07:00
acrefoot 0dfb76fc5e Change HTTP verb for create_user_backend to PUT (#1326) 2016-07-20 12:57:47 -07:00
Steve Howell 2484d870b4 Add test_build_narrow_filter(). 2016-07-20 11:04:00 -07:00
Tim Abbott 19b860ceec Rename local_settings.py symlink to prod_settings.py. 2016-07-19 20:59:59 -07:00
Tim Abbott e876f12b83 settings: Extract settings.WELCOME_EMAIL_SENDER. 2016-07-19 15:28:41 -07:00
Tim Abbott 7725c62892 Use VERBOSE_SUPPORT_OFFERS in day1 emails. 2016-07-19 15:28:41 -07:00
Tim Abbott 3ea4cbb5c3 settings: Extract settings.VERBOSE_SUPPORT_OFFERS. 2016-07-19 15:28:41 -07:00
Tim Abbott ff1faffecd settings: Extract EMAIL_GATEWAY_EXTRA_PATTERN_HACK. 2016-07-19 15:28:41 -07:00
Tim Abbott 82f78621dd settings: Extract CUSTOM_LOGO_URL. 2016-07-19 15:28:41 -07:00
Tim Abbott de679a23c9 settings: Extract settings.REGISTER_LINK_DISABLED. 2016-07-19 15:28:41 -07:00
Tim Abbott c25b077224 settings: Extract settings.SHOW_OSS_ANNOUNCEMENT. 2016-07-19 15:28:41 -07:00
Tim Abbott fb386da552 settings: Extract settings.SAVE_FRONTEND_STACKTRACES. 2016-07-19 15:28:41 -07:00
Tim Abbott 522ccf8eb2 settings: Extract settings.STAGING_ERROR_NOTIFICATIONS. 2016-07-19 15:28:41 -07:00
Tim Abbott 1f4c9eefe4 settings: Extract settings.ZILENCER_ENABLED. 2016-07-19 15:28:41 -07:00
Tim Abbott 0aaa55fb8f settings: Extract settings.SHARE_THE_LOVE. 2016-07-19 15:28:41 -07:00
Tim Abbott afaac85dc6 Move get_sqlalchemy_connection to its own file. 2016-07-19 15:28:41 -07:00
Steve Howell 83679a7775 Test search queries and highlight_string(). 
This increases test coverage by exercising highlight_string().
It also gives deeper test coverage to NarrowBuilder.by_search(),
which had test coverage before, but only in terms of inspecting
the SQL that was generated.  This test actually runs the SQL
under the hood.

This partly fixes #1006.
 2016-07-18 16:39:19 -07:00
Tim Abbott 50c3e42f0e Fix test failure due to recent merge interaction. 
599b15cb84 broke master because it had
previously only been tested before HomeTest was created.
 2016-07-18 16:37:25 -07:00
Kartik Maji 599b15cb84 Allow fetching subscribers for streams the user has never subscribed to. 
This allows the frontend to fetch data on the subscribers list (etc.)
for streams where the user has never been subscribed, making it
possible to implement UI showing details like subscribe counts on the
subscriptions page.

This is likely a performance regression for very large teams with
large numbers of streams; we'll want to do some testing to determine
the impact (and thus whether we should make this feature only fully
enabled for larger realms).
 2016-07-18 16:24:19 -07:00
Taranjeet Singh 5462341cb4 zerver/views/webhooks/github.py: Fix lines with length greater than 120. 2016-07-18 15:10:41 -07:00
Taranjeet Singh 3031214718 zerver/views/messages.py: Fix line with length greater than 120. 2016-07-18 15:05:52 -07:00
Tim Abbott 3c591aa724 compilemessages: Don't try to include zh-CN and zh_CN in language options. 2016-07-18 15:02:43 -07:00
Tim Abbott 55a98a41d8 compilemessages: Improve error handling for unknown locale. 2016-07-18 15:02:43 -07:00
Taranjeet Singh 84660a5087 zerver/views/__init__.py: Fix lines with length greater than 120. 2016-07-18 14:51:06 -07:00
Taranjeet Singh 37ea785b8f zerver/forms.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh 9f3f82d06d zerver/lib/socket.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh 5499136bfd zerver/views/webhooks/teamcity.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00
Taranjeet Singh 7199ee8f08 zerver/views/webhooks/taiga.py: Fix line with length greater than 120. 2016-07-18 14:42:39 -07:00