Commit Graph

1390 Commits

Author SHA1 Message Date
Rishi Gupta 84bb73ab65 bulk_create: Clean up ORM calls in bulk_create_streams. 2016-12-15 13:22:17 -08:00
Rishi Gupta fbf48f7ec6 bulk_create: Refactor bulk_create_streams to not take domain dictionary.
First step in cleaning up populate_db.create_streams and
bulk_create.bulk_create_streams. Part of a series of commits to remove
Realm.domain from populate_db.
2016-12-15 13:22:17 -08:00
Umair Khan c24fbd28b2 Shift to python-gcm for Android push notifications.
Fixes #2650
2016-12-15 12:17:07 -08:00
Sampriti Panda 91780180f6 integrations: Add webhook code, API endpoint, and tests for AppFollow 2016-12-15 11:13:27 -08:00
Umair Khan 770a899239 Django 1.10: Use single cache prefix for casper tests.
There is a change in Django 1.10 due to which whenever the password
of the user is changed the session hash changes. This change affects
us because we cache user profile objects and these cached objects need
to be refreshed. However, the signal sent by Django in which objects are
refreshed fails to refresh the cache for Tornado because it uses a
different cache prefix.

Note: Backend tests are not affected because they don't rely on Tornado.
2016-12-14 22:40:33 -08:00
Tim Abbott 0b33be50f3 lint: fix some whitespace issues in new reactions code. 2016-12-14 20:37:13 -08:00
Kracekumar R 61d2297c17 Add reactions in the /json/messages endpoint. 2016-12-14 19:21:04 -08:00
Harshit Bansal dc4e56abae Show offending stream name incase of a non existent stream.
Fixes: #2171.
2016-12-14 12:17:20 -08:00
Rishi Gupta 92a72f7039 mypy: Use typing.Text in actions.py. 2016-12-14 11:40:34 -08:00
Bojidar Marinov d28f1ddfb4 zerver: Handle `update_global_notifications` in `apply_events`
Fixes #2358
2016-12-13 20:20:26 -08:00
reallyly 94b28e0afb pep8: Fix E128 violations.
With some line-wrapping tweaks by tabbott.
2016-12-13 10:50:14 -08:00
Igor Tokarev fae59502ab URL preview: Improve test coverage. 2016-12-13 10:43:02 -08:00
Joy Chen bfb6ac5fdb streams: Add default stream description tests and functions.
This includes making the default stream description setting into a
dict.  That is an API change; we'll discuss it in the changelog but it
seems small enough to be OK.

With some small tweaks by tabbott to remove unnecessary backwards
compatibility code for the settings.

Fixes #2427.
2016-12-07 21:12:44 -08:00
Joy Chen dd95a5e03f streams: Add default stream description population data. 2016-12-07 21:02:17 -08:00
Igor Tokarev c93f1d4eda Add oembed/Open Graph/Meta tags data retrieval from inline links.
This change adds support for displaying inline open graph previews for
links posted into Zulip.

It is designed to interact correctly with message editing.

This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.

By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.

Eventually, we may want to make this manageable via a (set of?)
per-realm settings.  E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
2016-12-07 17:40:18 -08:00
Tim Abbott b68fef8933 actions: Extract update_to_dict_cache. 2016-12-07 17:26:16 -08:00
Sampriti Panda 7c3aff92d9 integrations: Add webhook payloads, webhook code, API endpoint, and tests for heroku 2016-12-06 11:05:00 -08:00
Bickio 84765e48a9 integrations: Add webhook code, API endpoint, and tests for papertrail 2016-12-05 21:48:11 -08:00
Mikebarson 690d72d35f mypy: Convert more zerver/lib files to typing.Text.
This commit touches files in zerver/lib/bugdown/ and
zerver/lib/webhooks.
2016-12-05 21:17:24 -08:00
Vamshi Balanaga f89e732d9e integrations: Add webhook code, API endpoint, and tests for mention. 2016-12-05 16:46:32 -08:00
sidhant bhavnani 4634c3d656 Change call signature for users_subscribed_to_stream()
The function users_subscribed_to_stream() now takes realm now, instead of domain.
2016-12-05 15:46:19 -08:00
AZtheAsian 441743cb89 integrations: Add webhook code, API endpoint, and tests for stripe.
This integration still needs documentation.
2016-12-03 20:42:43 -08:00
Reid Barton ecfa397567 Fix several type annotations for mypy 0.4.6 compatibility.
A few functions had arguments removed without having their type
annotations updated accordingly. As a result mypy version 0.4.6
thinks that the first type in the annotation is supposed to be
the type of `self`, a new mypy feature which we are not intending
to use here.
2016-12-03 13:45:52 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
bulat22101 adebc75740 pep8: Fix E502 violations 2016-12-03 10:56:36 -08:00
Arpith Siromoney 4491ea8d6b reactions: Add support for removing emoji reactions.
This commit adds support for removing reactions via DELETE requests to
the /reactions endpoint with parameters emoji_name and message_id.

The reaction is deleted from the database and a reaction event is sent
out with 'op' set to 'remove'.

Tests are added to check:
1. Removing a reaction that does not exist fails
2. When removing a reaction, the event payload and users are correct
2016-12-02 16:39:53 -08:00
Sidhant Bhavnani 8c0c12c1d9 pep8: Fix E303 violations. 2016-12-02 15:34:11 -08:00
Alex Huang 007b693cc7 pep8: Fix E131. 2016-12-01 23:16:47 -08:00
Alex Huang c8ddea16c3 pep8: Fix E122. 2016-12-01 23:16:35 -08:00
AZtheAsian 1ba150fa85 pep8: Fix E203 violations 2016-12-01 20:37:57 -08:00
AZtheAsian c23f9e0df7 pep8: Fix E111 violations 2016-12-01 14:18:11 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Tim Abbott f8f017c221 test_runner: Fix template rendering test.
The previous logic would allow the same template to be added to both
the shallow_tested list and the normal list.
2016-12-01 10:12:38 -08:00
Tim Abbott 5e0d2c4e1b tests: Exclude markdown files from shallow template testing.
The markdown files under templates/zerver/help/ are technically not
templates in the standard sense, and thus should not be being
checked with this code path.

(We probably do want to add a test to make sure they all render fine,
but that can be its own project.)
2016-12-01 10:12:38 -08:00
Vamshi Balanaga 5f60258a85 pep8: Fix E211 violations. 2016-11-30 20:13:10 -08:00
Bickio 6b0df43463 pep8: Fix E125. 2016-11-30 20:03:29 -08:00
Bickio e009383460 pep8: Fix E231. 2016-11-30 19:59:25 -08:00
Tommy Ip e4091c6413 pep8: Fix E222 violations. 2016-11-30 21:49:02 +00:00
Tim Abbott e6ebffefa0 test_helpers: Cleanup output a bit. 2016-11-29 16:40:05 -08:00
Rafid Aslam 41bd88d5ed pep8: Fix E301 pep8 violations.
Fix "E301: expected (1 or 2) blank line" pep8 violations.
2016-11-29 08:51:44 -08:00
Rafid Aslam 7a2282986a pep8: Fix E225 pep8 violations. 2016-11-28 15:21:15 -08:00
sylvan1 443cf92640 mypy: Change Generator[None, None, None] to Iterator[None].
Fixes #1648.
2016-11-27 10:42:16 -08:00
Tim Abbott 3d1bcb05e1 tornado: Move event_queue.py to zerver/tornado/.
Fixes #729.
2016-11-26 22:29:28 -08:00
Tim Abbott 169d404579 tornado: Move zerver.lib.handlers into zerver.tornado.handlers.
This cleans up the confusingly duplicated file names, while also
moving more of the Tornado-specific code under zerver/tornado/.
2016-11-26 22:29:28 -08:00
Tim Abbott d75f49b119 tornado: Move descriptor dict management code to handlers.py. 2016-11-26 22:29:28 -08:00
Tim Abbott 282f74609c tornado: Move socket code to zerver/tornado/. 2016-11-26 22:29:27 -08:00
Tim Abbott 8739f4d9f1 tornado: Move tornado_ioloop_logging to new zerver/tornado tree. 2016-11-26 21:24:05 -08:00
Arpith Siromoney 001847ac5b Add backend support for emoji reactions.
This commit adds the following:

1. A reaction model that consists of a user, a message and an emoji that
are unique together (a user cannot react to a particular message more
than once with the same emoji)
2. A reaction event that looks like:
    {
        'type': 'reaction',
	'op': 'add',
	'message_id': 3,
	'emoji_name': 'doge',
	'user': {
	    'user_id': 1,
            'email': 'hamlet@zulip.com',
            'full_name': 'King Hamlet'
	}
    }
3. A new API endpoint, /reactions, that accepts POST requests to add a
reaction to a message
4. A migration to add the new model to the database
5. Tests that check that
   (a) Invalid requests cannot be made
   (b) The reaction event body contains all the info
   (c) The reaction event is sent to the appropriate users
   (d) Reacting more than once fails

It is still missing important features like removing emoji and
fetching them alongside messages.
2016-11-26 18:10:21 -08:00
Calvin Lee 4e5c4c5ffb streams: Add support for setting stream description during creation.
Add new box for stream description in the stream creation form,
modify backend so its contents end up as the stream description.

Fixes: #2283.
2016-11-26 16:55:35 -08:00
Calvin Lee b8d7f8008a streams: move the invite_only field into the stream_dict argument.
Refactor list_to_streams and create_streams_if_needed.  The
stream_dict now has a "invite_only" key, which is mapped to
a boolean.
2016-11-26 16:52:40 -08:00
Calvin Lee 8461cc411e streams: refactor stream creation code path.
Refactor list_to_streams and create_streams_if_needed to take a list
of dictionaries, instead of a list of stream names.  This is
preparation for being able to pass additional arguments into the
stream creation process.

An important note: This removes a set of validation code from the
start of add_subscriptions_backend; doing so is correct because
list_to_streams has that same validation code already.

[with some tweaks by tabbott for clarity]
2016-11-26 16:48:59 -08:00
Tomasz Kolek a79acf854f docs: Automate creation of Hubot documentation lozenges.
This removes a bunch of semi-duplicated code.
2016-11-26 15:27:54 -08:00
Rishi Gupta 4b183cd526 domain migration: Remove several instances of get_realm.
Remove the easy to remove instances of get_realm.
2016-11-26 15:19:56 -08:00
Umair Khan 0536aeba4d Django 1.10: Use same cache prefix for JS tests.
Previously, the key prefix was based on the process id due to which
the JS tests couldn't properly flush user profiles from the cache as
our application spans over multiple processes. This problem becomes
apparent when in json_change_settings view after changing the user_profile
the tornado views continue to get the cached user profile corresponding
to their process id.
2016-11-26 15:10:50 -08:00
Calvin Lee 7a3ef2b0eb tests: Add assert_in_success_response in ZulipTestCase.
Clean up the instances of self.assertIn("string", result.content.decode("utf-8")),
and replace them with self.assert_in_response("string").

Fixes: #2313
2016-11-26 14:52:44 -08:00
Steve Howell 6ed1dc9341 Create api docs from url coverage data from tests.
I also retire tools/analyze-url-coverage in this commit,
since the API docs cover most of the functionality.
2016-11-26 14:49:23 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Anders Kaseorg 573ec14955 Remove shebang line from non-scripts
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 13:20:22 -08:00
Steve Howell 92030204bb Look for untested urls more exhaustively. 2016-11-26 09:52:37 -08:00
Steve Howell 1527823b9f code cleanup: Remove some unused variables in test classes. 2016-11-26 09:52:37 -08:00
Rishi Gupta 17f71befb4 mypy: Change six.text_type to typing.Text for a few files.
Preparation for GCI.
2016-11-25 13:55:38 -08:00
Steve Howell 5f5e6b6d83 tests: Enforce 100% URL coverage.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.

If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.

With some tweaks to the output from tabbott.

Fixes #1441.
2016-11-18 18:10:29 -08:00
Vladislav Manchev d7e1e4a2c0 Add initial implementation of custom realm filters.
This PR was abandoned by Vladislav and then substantially modified by
Igor Tokarev and Tim Abbott to complete it and fix a number of bugs.

Fixes #544.
2016-11-17 17:11:25 -08:00
hackerkid cdb716aa63 bugdown: Add explanation for unicode-emoji JS regex. 2016-11-17 10:44:07 -08:00
Steve Howell 3a8282ff44 Add test coverage for /notify_tornado.
We now have test coverage for all of our endpoints.
2016-11-17 10:11:46 -08:00
Tomasz Kolek 074571dddc Fix English grammar in push/remove tag message. 2016-11-15 10:48:12 -08:00
Tomasz Kolek 1feedbe346 Add github webhook integration. 2016-11-15 10:48:12 -08:00
Umair Khan 2314adc4fc Django 1.10: Compare CSRF tokens after unsalting them.
For reference: https://docs.djangoproject.com/en/1.10/ref/csrf/#how-it-works
2016-11-15 09:34:08 -08:00
Umair Khan 29254142f6 Django 1.10: Run maybe_update_realm_filter when it's needed. 2016-11-15 09:34:08 -08:00
hackerkid 67120d7a9d Support backend unicode-emoji rendering without whitespace. 2016-11-14 21:02:18 -08:00
Tim Abbott 8a5ae2893e send_pm_if_empty_stream: Clean up unnecessary nesting.
This has no functional changes.
2016-11-14 20:40:05 -08:00
Tim Abbott b09c2369c1 send_pm_if_empty_stream: Use realm object to check cross-realm.
The previous check didn't work correctly in the event that the target
stream did not exist.
2016-11-14 20:36:46 -08:00
Tim Abbott af675998d5 Fix send_pm_if_empty_stream with nonexistent streams. 2016-11-14 20:32:29 -08:00
hackerkid 8c83b34ad5 Support backend emoji rendering without whitespace. 2016-11-11 16:18:57 -08:00
Tomasz Kolek 4659abae65 test_fixtures: handle a case when given database does not exist.
Sometimes i.e. when you force close (ctrl-c) during ./tools/test-backend
a next execution might raise an error becuase DATBASE_NAME does not exist.
2016-11-11 15:40:10 -08:00
Tomasz Kolek ed1d7f7997 Add is_truncated param to git integrations push commits event.
Add is_truncated param for cases when we know that there were more commits
but we dont know actual number and want just inform about existing of the rest.
2016-11-11 15:36:45 -08:00
Tomasz Kolek 6be27a1396 Add get_push_tag_event_message function to git integrations common (git.py). 2016-11-11 15:36:45 -08:00
Rishi Gupta 5681ee90fa models.py: Rename split_email_to_domain to email_to_domain.
In order to better match models.email_to_username.
2016-11-11 15:26:51 -08:00
Rishi Gupta c1713c9659 Prevent code from using email domain to determine realm when subdomains.
Also removes the intermediate step of going through Realm.domain in the
non-subdomains case. Part of a larger project to remove Realm.domain
entirely.
2016-11-11 15:26:51 -08:00
Rishi Gupta 411a7b08e0 bulk_create_users: No longer de-alias user email addresses.
bulk_create.bulk_create_users is only called by initialize_voyager_db.py and
populate_db.py, both of which only have realms with single domains.

Part of a larger project to remove the Realm.domain field.
2016-11-11 15:26:51 -08:00
Tim Abbott 3b145b2370 add_new_user_history: Fix ordering of created UserMessage rows.
This has no functional effect because user_message IDs aren't used,
but will make the database cleaner.
2016-11-10 20:39:34 -08:00
Tim Abbott 08ca209aed add_new_user_history: Fix race with new messages arriving during signup.
Previously, if a new message arrived between when a user is subscribed
to the default streams and when the user's initial messages are
queried, we would try to create two UserMessage rows for the same
Message, resulting in an IntegrityError crash.  We fix this and add a
test for that race condition.
2016-11-10 20:38:27 -08:00
Tim Abbott 80c2df616e signup: Extract add_new_user_history. 2016-11-10 20:37:03 -08:00
sinwar 4582a98c09 tests: Split out ZulipTestCase and WebhookTestCase to a separate file.
Fixes #1671.
2016-11-10 19:29:43 -08:00
Igor Tokarev 852bc6b491 bugdown: Re-enable support for italics in bugdown.
Fixes: #1103.
2016-11-10 19:14:40 -08:00
Umair Khan 010ab0637d Django 1.10: Upgrade integration urls. 2016-11-10 16:20:03 +05:00
Umair Khan cfded8b5af Django 1.10: Resolve QuerySet returned by model_to_dict.
Django 1.10 resolves ManyToManyField into a QuerySet in model_to_dict.
This commit further resolves the QuerySet to the primary keys.
2016-11-09 15:29:58 -08:00
paxapy 6d93b3b60c lint: Fix E703 pep8 violations. 2016-11-09 15:18:35 -08:00
Steve Howell b5ef679360 Add get_topic_history_for_stream(). 2016-11-09 13:56:27 -08:00
Steve Howell a03a7d4c95 peer_remove: Send user_id, not email, for unsubscribe events. 2016-11-08 15:36:29 -08:00
paxapy ff1e97603d context_processors: use a common context for emails.
Fixes #1611.
2016-11-08 11:37:16 -08:00
Tim Abbott 56552007bf create_user: Disable stream desktop notifications by default.
This feature intended to be disabled by default months ago, but due to
the override code deleted in this commit, was actually still on by default.
2016-11-08 07:48:49 -08:00
Arpith Siromoney fbe8171813 Typing indicators backend: Pass list of user ids to send_event
send_event() expects a list of user ids (ints) except for the special case
of messages. This commit:
1. Fixes this in the call to send_event() in do_send_typing_notification()
2. Renames the variables in do_send_typing_notification() to better reflect
their content (for example, recipient_ids instead of recipients).
3. Renames the id field in the dicts sent in the typing event body (sender,
recipients) to user_id.
4. Adds assertions to the tests to verify that the tornado event user ids
are the same as the recipients in the event body.
5. Adds assertions to the tests to verify that the tornado event user
ids and the recipient user ids (in the event body) are the same as the
expected user ids (obtained from the emails using
get_user_profile_by_email)
6. Changes all assertTrues to assertEquals in the tests

This fixes #2151.
2016-11-07 22:28:09 -08:00
Meena Rajan 30acb23314 bugdown: Add support for strikethrough in markdown processor.
[Tweaked to move tests to bugdown_data.json, add additional tests, and
add frontend processor support by tabbott]
2016-11-07 22:26:38 -08:00
Rishi Gupta 9e6e1a1e69 Remove several instances of datetime.datetime.utcnow().
Change to the timezone-aware django.utils.timezone.now() where django is
available.
2016-11-07 20:13:53 -08:00
Rishi Gupta 9e5ec2fd29 notifications.send_future_email: Use timezone aware times.
Previous behavior raises an AmbiguousTimeError during daylight savings hour!
2016-11-07 20:13:53 -08:00
Rishi Gupta a416a9aeae test_runner.py: Fix "Import unexpectedly succeeded" messages in tests.
Previously we were getting this message whether or not the import would have
succeeded. The regression was introduced in ea050d5f.
2016-11-07 20:13:17 -08:00
Kevin Chen 6107c877e8 bugdown: Add option to support "file:///" as hyperlink.
This contains contributions from Tim Abbott and Igor Tokarev.

Fixes #380.
2016-11-06 22:30:05 -08:00
umkay f4c621ffe3 admin: Enable admins to toggle supported auth methods via UI.
Add a table to the administration page that will allow realm admins to
activate and deactivate the supported authentication methods for that
realm.
2016-11-06 16:29:35 -08:00