Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
12,109 Commits 28 Branches 143 Tags 624 MiB
Commit Graph

2795 Commits

Author SHA1 Message Date
Steve Howell 90494cfec2 export: Note user-owned bot limitations. 2016-08-12 15:10:49 -07:00
Tim Abbott ed01842f95 notifications: Fix missed message emails with multiple senders. 
Previously, missed message emails with multiple senders would
incorrectly have a "," outside the quoted sender name part of the from
address string, resulting in confusing email output.
 2016-08-12 12:28:06 -07:00
Tim Abbott 386c6c2a31 settings: Refactor SYSTEM_ONLY_REALMS configuration. 2016-08-12 12:12:53 -07:00
Tim Abbott f0e495831e forms: Fix check for whether terms of service are enabled. 2016-08-12 11:04:45 -07:00
Steve Howell 6fdd42c08b export: Create convenient soft links. 2016-08-12 10:48:33 -07:00
Steve Howell 70b68ddcc3 export: Use a config for export_single_user(). 2016-08-12 10:37:41 -07:00
Steve Howell c69a5bdec3 export: Handle more tables via export_from_config(). 
This commit introduces the ability to do custom fetches
and to essentially use temp tables for intermediate results.

(The temp table stuff deals with recipients/subscriptions
having three different flavors--user, stream, and huddle.)
 2016-08-12 10:37:35 -07:00
Steve Howell f471a1779e export: Handle simple exports with export_from_config(). 
This handles the simple tables that don't need custom fetches.
 2016-08-12 09:54:57 -07:00
Steve Howell 682155778d export: Add export_with_config(). 
Subsequent commits will start to use this.
 2016-08-12 09:54:57 -07:00
Steve Howell b0e6d20321 export: Write stats.txt for `./manage.py export <realm>`. 2016-08-12 09:06:10 -07:00
Rishi Gupta 7910a6e134 Terms of Service: Add ability to update TOS and have users re-sign. 
Most directly useful for the migration to zulipchat.com.

Creates a new field in UserProfile to store the tos_version, as well as two
new settings TOS_VERSION and FIRST_TIME_TOS_TEMPLATE. We check for a version
mismatch between what the user has signed and the current
settings.TOS_VERSION whenever the user hits the home page, and redirect them
if needed.

Note that accounts_accept_terms.html and
zerver.views.accounts_accept_terms were unused before this commit
(they date from c327446537)
 2016-08-11 23:37:48 -07:00
Tim Abbott 11cac86a58 settings: Add support for specifying a remote redis password. 2016-08-11 16:54:22 -07:00
Steve Howell df3aa39be3 export: Extract write_data_to_file(). 2016-08-11 15:51:22 -07:00
Steve Howell f29b32bbb2 export: Clarify message exporting code. 
The function to create the message partial files has been
renamed to export_partial_message_files().  It now gets its own
list of user profile ids and recipient ids from the response,
so that we can de-clutter do_export_realm().
 2016-08-11 15:51:22 -07:00
Steve Howell 5cd915694a export: Extract launch_user_message_subprocesses(). 
This is the last in a series of commits that makes it
so that do_export_realm() mostly delegates work out
to other functions.
 2016-08-11 15:21:30 -07:00
Steve Howell b383f5ca5d export: Extract fetch_user_profile_cross_realm(). 2016-08-11 15:21:30 -07:00
Steve Howell fee2106c6f export: Extract fetch_huddle_objects(). 
This also removes the dead codepath for include_private=False.
 2016-08-11 15:21:30 -07:00
Steve Howell a6235f6a60 export: Add comments to export_single_user(). 
(This is a bit of a prefactoring to hopefully create a nice
diff in a subsequent commit.)
 2016-08-11 15:21:30 -07:00
Steve Howell 6e7fe76cf4 export: s/avatar_bucket/processing_avatars 
The name avatar_bucket was confusing for a boolean, and
in some places it was used for non-S3 paths.

I considered the more concise 'is_avatar', but that
was still confusing when you are processing multiple
files, because you think it's a calculated property
on one file instead of an overall codepath switch.

I also considered splitting up some functions, but
there is a lot of common logic between handling
file uploads and avatars that's not trivial to extract
into helpers, especially on the S3 side.
 2016-08-11 15:21:30 -07:00
Steve Howell 3dab366733 export: Clean up names of upload/avatar export functions. 
I did some minor moving around of code that made us have
one fewer function without any additional conditional
logic. The names are more explicit about saying
"from_local" and "from_s3".  Also, there is less clutter
now in do_export_realm(), which is evolving into more of
a dispatcher and less of a worker.
 2016-08-11 15:21:30 -07:00
Steve Howell 9582a83d10 export: Fix usage in export_single_user command. 2016-08-11 15:21:30 -07:00
Steve Howell d62a351107 export: Add sanity_check_output(). 2016-08-11 15:21:30 -07:00
Tim Abbott 95aee366c8 Enable local browser error reporting if ZILENCER_ENABLED. 2016-08-11 13:49:18 -07:00
Umair Khan af9f308f0e Send double columned language list in page_params. 2016-08-10 16:19:28 -07:00
Umair Khan 73e5289736 Add default_language_name in page_params. 2016-08-10 16:19:28 -07:00
Umair Khan ddaf1a2b67 Return unformatted list from get_language_list. 2016-08-10 16:19:28 -07:00
Steve Howell 06b0df5efc export: Remove spurious select_related() call for Client. 2016-08-10 14:16:17 -07:00
Steve Howell cb59a11f0a export: Extract get_primary_ids(). 2016-08-10 14:16:17 -07:00
Steve Howell 90e9083b81 export: Extract filter_by_realm(). 2016-08-10 14:16:17 -07:00
Steve Howell 4b6b1b8ad4 export: Extract filter_by_users(). 2016-08-10 14:16:17 -07:00
Steve Howell db9edfce34 export: Use DATE_FIELDS in fix_datetime_fields(). 
Now we only call this once per table and use DATE_FIELDS to
look up the data fields.
 2016-08-10 14:16:17 -07:00
Steve Howell 35c59fc4d7 export: Clean up export_messages(). 
This is pretty minor cleanup, but it makes it a little more
explicit what we're writing to the shard file, and it allows
us to use a more specific mypy type when calling
floatify_datetime_fields.
 2016-08-10 14:16:17 -07:00
Steve Howell 1d1f36c0b8 export: Always use subprocesses to export UserMessage. 
We no longer have an in-process code path to export
UserMessage rows.  We want to only maintain the
subprocess code, which we'll always use in production,
and which will work fine in dev.
 2016-08-10 14:16:17 -07:00
Steve Howell 78bbefbf94 export: Create import_attachments. 2016-08-10 14:16:17 -07:00
Rag Sagar 2fef36f15a Add realm-level default language setting. 
Adds a new field default language in the zerver_realm model.
This realm level default language will be used as default language
for newly created users. Realm level default language can be
changed from the administration page.

Fixes #1372.
 2016-08-09 17:38:29 -07:00
Steve Howell 7ec6a394fe export: Filter Attachment objects by realm. 2016-08-09 16:47:14 -07:00
Steve Howell cecfaa7761 export: Extract import_message_data(). 2016-08-09 16:47:14 -07:00
Steve Howell 5386ed280e export: Extract update_id_map(). 
We also use a vanilla dictionary instead of a defaultdict, so
that we explicitly initialize what tables are being re-mapped.
 2016-08-09 16:47:14 -07:00
Steve Howell 217ef8a4d2 export: Split fix_foreign_keys() into two functions. 
We now have convert_to_id_fields for the simple case, and
re_map_foreign_keys for the more complex case. I also
renamed some parameters and variables.
 2016-08-09 16:47:14 -07:00
Umair Khan e1090118c5 Percent translated should be an integer. 2016-08-09 16:19:07 -07:00
Rag Sagar 5771e2eb25 Move i18n functions from zerver/views/__init__.py to zerver/lib/i18n.py 2016-08-09 16:10:24 -07:00
Rag Sagar 28f0079432 Use a constant to set UserProfile default_language field max_length attribute. 2016-08-09 16:10:24 -07:00
Steve Howell dd88ffccfd export: Extract make_raw() in lib/export.py. 2016-08-09 15:58:27 -07:00
Steve Howell 09fa343bdd export: Use DATE_FIELDS in floatify_datetime_fields. 
This avoids a little bit of code duplication, plus it should
make it a little easier to add new date fields in the future.
 2016-08-09 15:58:27 -07:00
Steve Howell c14ab3c91f export: Add annotations to zerver/lib/export.py. 
I also fixed some small things like removing unnecessary return
statements, and adding a TODO.

In some cases I explicitly cast stuff at run-time to set() or
str() to appease mypy, as well as make it clear to somebody
reading the code that the callee might not respect ordering
or tolerate unicode.
 2016-08-09 15:58:27 -07:00
Steve Howell f18cc4ae3a export: Added export_avatars_local_helper(). 2016-08-09 15:58:27 -07:00
Umair Khan 86920a297c Mock out loggers in push notification tests. 
Fixes: #1575
 2016-08-09 22:16:09 +05:00
Umair Khan 7a84f12a38 Verify mock calls without strict order. 
In push notification tests, call assert_has_calls with `any_order=True`
so that the order of the calls does not matter.
 2016-08-09 22:15:37 +05:00
Tim Abbott 42b63f5caa test_templates: Fix failure to report a user-facing error. 2016-08-09 09:54:44 -07:00
Tim Abbott f3365f4089 test_templates: Add integrations_dict to context. 
This fixes the fact that this test was (silently) failing.
 2016-08-09 09:54:36 -07:00
Umair Khan fe812a89bf mypy: Type of remote_username should be text_type. 2016-08-09 09:19:24 -07:00
Umair Khan 8c249f859e Push Notification: Add tests for GCM. 2016-08-09 16:13:02 +05:00
Umair Khan acd76eb604 Annotate push_notifications.py. 2016-08-08 15:17:02 -07:00
Umair Khan 9c13e9a072 Annotate test_push_notifications. 2016-08-08 15:17:02 -07:00
Umair Khan 35dd1ffb13 Pass password as string to authenticate. 2016-08-08 15:17:01 -07:00
Umair Khan 266c791e64 Email should be a unicode. 2016-08-08 15:17:01 -07:00
Umair Khan 503ad774f5 Tokens should be unicode. 2016-08-08 15:17:01 -07:00
Umair Khan 0dd7d0dda1 push_notifications: Change name to 'identifier'. 2016-08-08 15:15:29 -07:00
Umair Khan fdbd132ba4 Annotate zerver/context_processors.py. 2016-08-08 15:15:29 -07:00
Umair Khan 00eab4e526 Annotate zerver/views/__init__.py. 2016-08-08 15:15:29 -07:00
Umair Khan 25dd1aeb5c Annotate zerver/lib/actions.py. 2016-08-08 15:13:29 -07:00
Umair Khan ef62506fb1 Annotate zerver/models.py. 2016-08-08 15:13:29 -07:00
Umair Khan e34b7a0691 Fix type annotation in message_cache_items. 2016-08-08 15:13:29 -07:00
Umair Khan ec910d96fc Annotate zerver/storage.py. 2016-08-08 15:12:12 -07:00
Umair Khan 0dfc757447 Annotate test_auth_backends.py. 2016-08-08 15:12:12 -07:00
Tim Abbott 6264ff7039 Add new Zulip realm import/export tool. 
The previous export tool would only work properly for small realms,
and was missing a number of important features:
* Export of avatars and uploads from S3
* Export of presence data, activity data, etc.
* Faithful export/import of timestamps
* Parallel export of messages
* Not OOM killing for large realms

The new tool runs as a pair of documented management commands, and
solves all of those problems.

Also we add a new management command for exporting the data of an
individual user.
 2016-08-08 14:58:18 -07:00
Tim Abbott e7adaf8db1 utils: Add mkdir_p implementation. 2016-08-08 14:36:50 -07:00
Tim Abbott 54a6bdc3ad views: Fix mypy annotations in get_invitee_emails_set. 
Apparently this buggy annotation slipped through into master.
 2016-08-05 15:04:16 -07:00
Tomasz Kolek 64b5d0ce64 Add support for inviting emails in "name" <email> format. 
Often, users will copy email addresses with a name (rather than pure
email addresses) into the Zulip "invite users" UI.  Previously, that
would throw an error.

This change also adds a get_invitee_emails_set function for parsing
emails content and a test suite for this new feature.

Fixes: #1419.
 2016-08-05 14:48:04 -07:00
Umair Khan d33fc0046f Add APNS feedback server tests. 2016-08-05 10:36:57 -07:00
Umair Khan 33450d1a0b Add APNS push notification tests. 2016-08-05 10:36:57 -07:00
Umair Khan 40054ce26c Add APNS response listener test case. 2016-08-05 10:36:57 -07:00
Umair Khan df8fc30b75 Create MockRedis class. 
Redis is not available on Travis and creating a mock Redis server looks
easier than using `mock` module.
 2016-08-05 10:36:57 -07:00
Umair Khan 2ab796aeb5 Add tests for APNS message. 2016-08-05 10:36:57 -07:00
Umair Khan 882bb5558b Push to APNS in a loop. 2016-08-05 10:36:57 -07:00
Umair Khan 4e8ca0a326 Make sure apns response code is an integer. 2016-08-05 10:36:57 -07:00
Umair Khan 5c41eae7ec Don't send connection to apns response listener. 2016-08-05 10:36:57 -07:00
Umair Khan ffaf7cb2ba Create get_connection function for APNS. 2016-08-05 10:36:57 -07:00
Umair Khan 1d2bdfdfb1 Save user id instead of user profile. 2016-08-05 10:36:57 -07:00
Umair Khan 0cac1aa135 For APNS frame identifier generate random int. 2016-08-05 10:36:57 -07:00
Umair Khan 0b5dc56b8d Redis key expiry should be an integer. 2016-08-05 10:36:57 -07:00
Umair Khan cbd53fbac8 Don't import OrderedDict. 2016-08-05 10:36:57 -07:00
Umair Khan 0b88deb640 APNS_SANDBOX should be a bool. 
Fixes: #1480
 2016-08-05 10:36:57 -07:00
Tim Abbott dcb92d41b8 Annotate test_messages.py. 2016-08-04 15:57:04 -07:00
Tim Abbott 39f1024740 test_messages: Fix remaining mypy errors. 2016-08-04 15:57:03 -07:00
Tim Abbott 88550edd9e Annotate queue_processors.py. 2016-08-04 15:57:03 -07:00
Tim Abbott 4e8054b84a Refactor queue worker tests to match actual API. 2016-08-04 15:57:03 -07:00
Tim Abbott 1e54897ca7 process_queue: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 4ddc567c50 views/__init__: Add missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 641aa08721 compilemessages: Add mypy annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a787dee48b makemessages: Add missing annotations and fix string types. 2016-08-04 15:57:03 -07:00
Tim Abbott 2da6b6bfd8 views.messages: Add some more annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 2f13b0b18a actions: Fix various string annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 4f782bc186 test_events: Fix annotation gaps. 2016-08-04 15:57:03 -07:00
Tim Abbott 2eb5baa024 zendesk: Add missing type annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott d7d8d6b4d2 parallel: Add missing annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott d00719f204 Annotate IFTTT tests. 2016-08-04 15:57:03 -07:00
Tim Abbott d449f6ba72 Annotate bitbucket2 tests. 2016-08-04 15:57:03 -07:00
Tim Abbott 5253c044e7 test_helpers: Add some missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott a7086d3d8a trello: Fix accidentally commented type annotation. 2016-08-04 15:57:03 -07:00