Commit Graph

1024 Commits

Author SHA1 Message Date
Steve Howell 217ef8a4d2 export: Split fix_foreign_keys() into two functions.
We now have convert_to_id_fields for the simple case, and
re_map_foreign_keys for the more complex case. I also
renamed some parameters and variables.
2016-08-09 16:47:14 -07:00
Rag Sagar 5771e2eb25 Move i18n functions from zerver/views/__init__.py to zerver/lib/i18n.py 2016-08-09 16:10:24 -07:00
Steve Howell dd88ffccfd export: Extract make_raw() in lib/export.py. 2016-08-09 15:58:27 -07:00
Steve Howell 09fa343bdd export: Use DATE_FIELDS in floatify_datetime_fields.
This avoids a little bit of code duplication, plus it should
make it a little easier to add new date fields in the future.
2016-08-09 15:58:27 -07:00
Steve Howell c14ab3c91f export: Add annotations to zerver/lib/export.py.
I also fixed some small things like removing unnecessary return
statements, and adding a TODO.

In some cases I explicitly cast stuff at run-time to set() or
str() to appease mypy, as well as make it clear to somebody
reading the code that the callee might not respect ordering
or tolerate unicode.
2016-08-09 15:58:27 -07:00
Steve Howell f18cc4ae3a export: Added export_avatars_local_helper(). 2016-08-09 15:58:27 -07:00
Umair Khan acd76eb604 Annotate push_notifications.py. 2016-08-08 15:17:02 -07:00
Umair Khan 0dd7d0dda1 push_notifications: Change name to 'identifier'. 2016-08-08 15:15:29 -07:00
Umair Khan 25dd1aeb5c Annotate zerver/lib/actions.py. 2016-08-08 15:13:29 -07:00
Umair Khan e34b7a0691 Fix type annotation in message_cache_items. 2016-08-08 15:13:29 -07:00
Tim Abbott 6264ff7039 Add new Zulip realm import/export tool.
The previous export tool would only work properly for small realms,
and was missing a number of important features:
* Export of avatars and uploads from S3
* Export of presence data, activity data, etc.
* Faithful export/import of timestamps
* Parallel export of messages
* Not OOM killing for large realms

The new tool runs as a pair of documented management commands, and
solves all of those problems.

Also we add a new management command for exporting the data of an
individual user.
2016-08-08 14:58:18 -07:00
Tim Abbott e7adaf8db1 utils: Add mkdir_p implementation. 2016-08-08 14:36:50 -07:00
Umair Khan d33fc0046f Add APNS feedback server tests. 2016-08-05 10:36:57 -07:00
Umair Khan 40054ce26c Add APNS response listener test case. 2016-08-05 10:36:57 -07:00
Umair Khan 882bb5558b Push to APNS in a loop. 2016-08-05 10:36:57 -07:00
Umair Khan 4e8ca0a326 Make sure apns response code is an integer. 2016-08-05 10:36:57 -07:00
Umair Khan 5c41eae7ec Don't send connection to apns response listener. 2016-08-05 10:36:57 -07:00
Umair Khan ffaf7cb2ba Create get_connection function for APNS. 2016-08-05 10:36:57 -07:00
Umair Khan 1d2bdfdfb1 Save user id instead of user profile. 2016-08-05 10:36:57 -07:00
Umair Khan 0cac1aa135 For APNS frame identifier generate random int. 2016-08-05 10:36:57 -07:00
Umair Khan 0b5dc56b8d Redis key expiry should be an integer. 2016-08-05 10:36:57 -07:00
Umair Khan cbd53fbac8 Don't import OrderedDict. 2016-08-05 10:36:57 -07:00
Umair Khan 0b88deb640 APNS_SANDBOX should be a bool.
Fixes: #1480
2016-08-05 10:36:57 -07:00
Tim Abbott 2f13b0b18a actions: Fix various string annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott d7d8d6b4d2 parallel: Add missing annotation. 2016-08-04 15:57:03 -07:00
Tim Abbott 5253c044e7 test_helpers: Add some missing annotations. 2016-08-04 15:57:03 -07:00
Tim Abbott 0689485666 Add lint check for malformed type annotations missing :.
Also fix the 2 annotations that weren't being checked because of this.
2016-08-04 15:53:23 -07:00
Tim Abbott 028efdcb78 integrations: Add detailed module docstrings. 2016-08-04 12:13:25 -07:00
Tomasz Kolek 0e33b8bd4d Add integrations library.
Define Integration and WebhookIntegration classes.
Change webhook part of integration's guide.
Replace hardcoded webhook urls to generating
based on WEBHOOKS list.
2016-08-04 11:39:20 -07:00
Rishi Gupta 948ea7663c ModelReprMixin: Fix handling of missing __unicode__ function.
The old behavior was to raise an exception, but Django was catching
the exception and doing unexpected things. For instance, in the
manage.py shell, printing out a ModelReprMixin object (with
__unicode__ not implemented) would result in nothing being printed,
rather than it raising a error or otherwise alerting the programmer as
to what was going on.
2016-08-02 11:46:01 -07:00
Umair Khan 2c07f1b19a Use NOREPLY_EMAIL_ADDRESS if email gateway not enabled.
This fixes a regression where missed message emails would not be sent
at all in the event that EMAIL_GATEWAY_PATTERN was unset.

The overall experience still isn't great, but it's better than crashing.

Fixes: #1411

[commit message expanded by tabbott]
2016-07-31 20:38:18 -07:00
Steve Howell 2a37dafcbb Write untested_url_report.txt. 2016-07-29 21:26:51 -07:00
Steve Howell 8b13530712 Add tools/analyze-url-coverage 2016-07-29 21:23:33 -07:00
Steve Howell 21f83afe3a Add --url-coverage option to ./tools/test-backend. 2016-07-29 21:23:33 -07:00
Steve Howell 9fb071947d Remove expected_run_time from @slow test decorator. 2016-07-29 16:41:21 -07:00
Steve Howell 438a118ea5 Simplify enforce_timely_test_completion().
This will lead to minor differences in the warnings that
people see when they run tests that are slow.  We call out
the slowness a little more clearly from a visual standpoint,
and we simplify the calculation of the slowness threshold.

We still allow more time for tests with the `@slow` decorator
to run, but we don't use their expected_run_time.
2016-07-29 16:41:21 -07:00
Steve Howell 4556bf528f Add --report-slow-tests option to tools/test-backend. 2016-07-29 16:41:20 -07:00
Umair Khan 8e5e6a06f2 Delete cache entry for user profile.
Our flush functions update user profile cache entries which can cause
confusing race conditions (see e.g. #1257).  To resolve this, we move
all the user_profile flush functions to delete the entry instead of
updating it -- it will then be fetched as part of the next request
that needs to access the user object.

There are still races here, and there is perhaps an argument that a
better fix for this would be to re-fetch the object and then put it
into the cache, but this resolves the main cache correctness problem
we had with the previous implementation.

Fixes: #1322.
2016-07-28 13:43:14 -07:00
Steve Howell 713797a65c Add lint checks for self.client.{get,patch,put,post,delete}. 2016-07-27 20:50:54 -07:00
Steve Howell 6937f63fd5 Add client_get() test helper. 2016-07-27 20:49:33 -07:00
Steve Howell 38f2a2f475 Add client_post() test helper.
This makes us more consistent, since we have other wrappers
like client_patch, client_put, and client_delete.

Wrapping also will facilitate instrumentation of our posting code.
2016-07-27 20:49:32 -07:00
Steve Howell 51bae8abc4 Remove stub() test helper. 2016-07-27 14:12:10 -07:00
rahuldeve 4267dcbb68 Change the way referred attachments are removed on message update.
This fixes incorrect behavior when two or more attachments are removed
in the same message.
2016-07-27 14:08:59 -07:00
Tim Abbott 375551aaa6 Clean up most hardcoding of mit.edu domain checks.
This moves all this code to be gated on a few virtual realm settings.
2016-07-26 20:30:12 -07:00
Tim Abbott 70543e059a bugdown: Remove hardcoding of mit.edu for zephyr_mirror realm config. 2016-07-26 20:30:12 -07:00
Tim Abbott c17676b00c Cleanup MIT hardcoding for disabling presence. 2016-07-26 20:30:12 -07:00
Tim Abbott 884f50cdd7 validate_user_access: Assert user_profile is not None.
This function is only called in cases where user_profile isn't None,
and the code reads better if we just check that first rather than
checking it on every line that accesses user_profile.
2016-07-26 20:30:12 -07:00
Tim Abbott d1adbd798b get_subscribers_to_streams: Pass through requesting_user. 2016-07-26 20:30:12 -07:00
Tim Abbott 691ad7fcfc maybe_get_subscriber_emails: Pass user_profile through. 2016-07-26 20:30:12 -07:00
Tim Abbott f790a04102 Remove unused get_subscriber_ids. 2016-07-26 20:30:12 -07:00