Commit Graph

7422 Commits

Author SHA1 Message Date
Tim Abbott 34e165c100 webhooks: Fix passing client string to authenticated webhook API views.
This fixes a regression in 93678e89cd
and a4979410f9, where the webhooks using
authenticated_rest_api_view were migrated to a new model that didn't
include setting a custom Client string for the webhook.

When restoring these webhooks' client strings, we also fix places
where the client string was not capitalized the same was as the
product's name.
2018-03-16 15:43:19 -07:00
Eeshan Garg a4979410f9 webhooks: Migrate most integrations to use check_send_webhook_message.
This commit migrates all of our webhooks to use
check_send_webhook_message, except the following:

beeminder: Rishi wanted to wait on this one.
teamcity: This one is slightly more work.

yo: This one is PM-only. I am still trying to decide whether we
    should have a force_private argument or something in
    check_send_webhook_message.

facebook: No point in migrating this, will be removed as part of
          #8433.

slack: Slightly more work too with the `channel_to_topics` feature.
       Warrants a longer discussion.
2018-03-16 19:23:50 -02:30
Eeshan Garg a1e3c91213 webhooks: Delete duplicate statuspage.io fixtures from GCI.
These fixtures were submitted by a GCI student but then another
student wrote the webhook from scratch anyway.
2018-03-16 19:12:09 -02:30
Shubham Dhama 610f2cbacf notification email: Send followup_day2 email two days later.
This changes the followup_day2 emails delay from one day later to two days
later if it is getting delivered on any working days(i.e. Mon - Fri).
For Thursday it is compromised to next day as it would be too late to
postponed to Monday and for Friday it should be Monday.
At last actually, emails should send one hour before the above calculated so
that user can catch them when they are dealing with these kinds of stuff.
Fixes: #7078.
2018-03-16 13:35:57 -07:00
neiljp (Neil Pilgrim) 5f7f2d6e76 mypy: Remove type: ignore from check_url test, as is now true Validator. 2018-03-16 13:30:32 -07:00
neiljp (Neil Pilgrim) 5726d26d50 mypy: Use centralized Validator in request.pyi & validator.py.
These changes are in one commit, since the previous typing of check_url
does not match the centralized strict definition (object/Any vs Text),
actually already used elsewhere in validator.py, and also had a different
API.

check_url is updated here to match the API of the other check_* functions,
ie. val is an object (not Text) & returns Optional[str]. It also now checks
the value is text explicitly at run-time, which was only type-checked
previously. Tests are updated accordingly.
2018-03-16 13:30:32 -07:00
YJDave 0281079a39 stream settings: Send all private streams subs to realm admins on load.
Tweaked by tabbott to simplify the conditionals in actions.py.

Fixes #8695.
2018-03-16 12:26:56 -07:00
YJDave 72a440a86d stream settings: Fix error in real time sync in subs add/remove event.
Currently, when other private stream subscriber add realm admin to
stream, new copy private stream is created in realm admin's streams.
Which resulted in error, cause there are two similar stream element
in stream settings.

If new subscriber is added to private stream, we first send them
stream `create` event, cause private stream are not visible until
user don't get subscribed at least once. But realm admins can now
always access private stream, so when realm admin is subscribed to
stream, realm admin get stream `create` event even if stream already
exist in on realm admin client side.

Fix this by extracting realm admins from stream `create` event on
`add` subscription operation and sending private stream `create`
event to all realm admins on stream creation operation.

Fixes #8695
2018-03-16 12:22:06 -07:00
Eeshan Garg 93678e89cd webhooks: Migrate 14 webhooks to use check_send_webhook_message.
These are the straightforward ones.

Note that there is a line in zerver.lib.test_classes.build_webhook_url
that lost test coverage. That's because most of our tests test using
stream messages so the webhook URLs being tested always have a query
parameter. So the line that accounts for there being no query
parameters never gets called, which is fine, but we should still
keep it.
2018-03-16 11:34:20 -07:00
Eeshan Garg af56df7723 webhooks: Enable custom topics and default PM notifications.
This commit adds a generic function called check_send_webhook_message
that does the following:
* If a stream is specified in the webhook URL, it sends a stream
  message, otherwise sends a PM to the owner of the bot.
* In the case of a stream message, if a custom topic is specified
  in the webhook URL, it uses that topic as the subject of the
  stream message.

Also, note that we need not test this anywhere except for the
helloworld webhook. Since helloworld is our default example for
webhooks, it is here to stay and it made sense that tests for a
generic function such as check_send_webhook_message be tested
with an actual generic webhook!

Fixes #8607.
2018-03-16 11:34:20 -07:00
Tim Abbott 707af5ab56 cache: Remove a now-unnecessary TODO.
We solved the problem the TODO raised by using a different type
annotation syntax, and I'm not sure whether that refactor would
actually improve the code.
2018-03-16 11:32:14 -07:00
neiljp (Neil Pilgrim) 966ca7015f mypy: Finalize migration of cache.py to python3 function annotation.
- Use forward declarations of some types from models.py to avoid cycles.
- Remove cache.py from linter rule exclude list to ensure it stays that way.
2018-03-16 11:29:12 -07:00
neiljp (Neil Pilgrim) 005cb6bd03 mypy: Improve [get_]cache_with_key typing & use py3 annotation. 2018-03-16 11:29:12 -07:00
neiljp (Neil Pilgrim) bf4dce2a7b mypy: Use centralized Validator definition from types.py in models.py. 2018-03-16 11:16:38 -07:00
neiljp (Neil Pilgrim) b79c3840da mypy: Introduce strict definition of Validator into types.py. 2018-03-16 11:16:38 -07:00
Tim Abbott 99bae7fa5d slack import: Improve algorithm and formatting for attachments.
The previous system would crash with some files (because for some
reason the comment count was 1 but there was no "initial comment") and
also the file comment and file name were sorta redundant.
2018-03-16 11:12:58 -07:00
Rhea Parekh 06071166c7 slack importer: Handle case where attachment download link is not from slack. 2018-03-16 11:12:58 -07:00
Rhea Parekh 6f3c87006b slack importer: Move output folder being extracted from /tmp to var/. 2018-03-16 11:12:58 -07:00
Rhea Parekh b7d6608ba6 slack importer: Clean 'create_converted_data_files' function.
The 'make_new_dir' bool value was used to create a new directory
every time True is passed. Now that avatars and uploads directory
are being created seperately, we don't need this anymore.
2018-03-16 11:12:58 -07:00
Rhea Parekh 4b66a2d0dc slack importer: Add function to fetch and save uploads. 2018-03-16 11:12:58 -07:00
Rhea Parekh e62945eb86 slack importer: Implement changes in script due to zerver_attachment. 2018-03-16 11:12:58 -07:00
Rhea Parekh 8e2d930644 slack importer: Implement changes in script due to user upload object. 2018-03-16 11:12:58 -07:00
Rhea Parekh b0851eb20b slack importer: Add helper functions to build attachment object. 2018-03-16 11:12:58 -07:00
Rhea Parekh 68af6e4b7a slack importer: Add helper functions to build user uploads object. 2018-03-16 11:12:58 -07:00
Rhea Parekh 6d85f8c9ef slack importer: Randomize stream colors. 2018-03-15 23:50:32 -07:00
Rhea Parekh 90a3ffc5c0 slack importer: Include only slack's purpose field in description. 2018-03-15 23:50:32 -07:00
Rhea Parekh 8a4f307c43 slack importer: Change topic for imported content. 2018-03-15 23:50:32 -07:00
Greg Price 4139e6c763 reactions: Fix migration to correctly handle corner case.
If an emoji that was deleted was the only realm emoji, or more
generally if all realm emoji were deleted, then we would just leave
the reaction unchanged, with an `emoji_code` that is now corrupt.

Instead, treat this case the same as if only this emoji was deleted
while others remain.
2018-03-15 18:53:51 -07:00
Tim Abbott ec21997bfe slack import: Fix importing of bot users from Slack. 2018-03-15 18:35:40 -07:00
Rhea Parekh a5b0957e5d slack importer: Set domain name in 'do_convert_data'.
The domain name is being set in the helper function
'slack_workspace_to_realm', but it should be set in the main function
'do_convert_data', as we need it in other child functions of
'do_convert_data'.
2018-03-15 18:34:51 -07:00
Tim Abbott 14e8ac5675 migrate: Remove dead migration code.
This code was originally written when we were using the old South
system, and hasn't been used in a few years.  It probably doesn't
work, and thus only serves to clutter the codebase.
2018-03-15 17:56:32 -07:00
Tim Abbott 68f816bba1 forms: Fix missing translation tag for disposable emails. 2018-03-15 14:43:40 -07:00
Vishnu Ks b13150a438 models: Do the check for disposable email in email_allowed_for_realm. 2018-03-15 14:35:24 -07:00
Vishnu Ks 951b88dd30 models: Make email_allowed_for_realm raise exception. 2018-03-15 14:35:24 -07:00
Vishnu Ks 1df2ed4c68 models: Remove unused GetRealmByDomainException,. 2018-03-15 14:35:24 -07:00
neiljp (Neil Pilgrim) 88046f815a mypy: Rewrite zulip_login_required annotations in terms of ViewFuncT. 2018-03-15 14:33:56 -07:00
neiljp (Neil Pilgrim) 8edb47e212 mypy: Fully use ViewFuncT in decorators.py; remove WrappedViewFuncT.
Many declarations were previously annotated with
Callable[..., HttpResponse]; this is equivalent to ViewFuncT, so here we
switch to it.

To enable this migration, the WrappedViewFuncT alias is removed; this is
equivalent to the simple & legible Callable[[ViewFuncT], ViewFuncT], so
for relatively no space change, a clearer return type is possible.
2018-03-15 14:33:56 -07:00
neiljp (Neil Pilgrim) a8f7a49e7b mypy: Apply type: ignore to api decorator tests. 2018-03-15 14:33:56 -07:00
neiljp (Neil Pilgrim) e30ba19b79 mypy: Apply type: ignore to api_github_webhook_dispatch calls. 2018-03-15 14:33:56 -07:00
neiljp (Neil Pilgrim) 31dce7c87c minor: Request parameter should be of type HttpRequest. 2018-03-15 14:33:53 -07:00
neiljp (Neil Pilgrim) 80a7c16baf minor: Remove unused WrapperT from zerver/decorator.py. 2018-03-15 14:33:28 -07:00
neiljp (Neil Pilgrim) 3dbe772b50 mypy: Migrate some Callable[..., HttpResponse] to ViewFuncT in decorator.py. 2018-03-15 14:33:27 -07:00
neiljp (Neil Pilgrim) 17937175ac mypy: Centralize ViewFuncT definition into new file zerver/lib/types.py.
Originally was going to centralize this in zerver/lib/request.pyi, but this
file is not visible at run-time, being only a stub. The matching request.py
file seemed inappropriate, as it doesn't actually use ViewFuncT.
2018-03-15 14:16:40 -07:00
neiljp (Neil Pilgrim) a3d8cc1f86 Mypy: Migrate tornado/descriptors.py to python3 function annotations.
Use quoted form of ClientDescriptor to break presumed circular dependency,
and modify import statement to shorten quotes.
2018-03-15 12:54:43 -07:00
neiljp (Neil Pilgrim) a7bfb09067 Mypy: Use models.py QuerySet annotation approach in stream_subscription.py.
Namely, annotate as best as possible, and add notes to indicate preference,
if QuerySet develops generic typing.

Note that the return values of functions with annotations changed in this
commit are used elsewhere as QuerySets, so the Sequence[T] approach used
for some functions in models.py is not applicable.
2018-03-15 12:54:43 -07:00
neiljp (Neil Pilgrim) ce4ac0d2cf Mypy: Ensure consistency of QuerySet return types in models.py.
Other functions took the form of returning Sequence[T] when the QuerySet
functionality is unused beyond the function, with T being the objects
filtered for in the function body; this commit follows that practice for the
one remaining python2 comment-annotated function, completing the transition
of models.py to py3.5 function annotations.

A note is also added to another function regarding a need to return a
QuerySet, and ideally a QuerySet[T] in line with the other functions, as and
when QuerySet becomes annotated as a generic.
2018-03-15 12:54:43 -07:00
Steve Howell a4a8527ec5 search: Return info flags in payload.
We now return these:
    anchor
    found_anchor
    found_oldest
    found_newest

Fixes #8639
2018-03-15 12:36:06 -07:00
Steve Howell c6839e07c0 search: Fix num_after/num_before semantics precisely.
We now post-process query results so that you never get
more than `num_after` rows with id < `anchor`, and likewise
for `num_before`.
2018-03-15 12:36:06 -07:00
Steve Howell 6f3ebf6c4c Add post_process_limited_query(). 2018-03-15 12:36:06 -07:00
Steve Howell bd95b37d67 search: Make `num_after`/`num_after` more consistent.
We now consistently set our query limits so that we get at
least `num_after` rows such that id > anchor.  (Obviously, the
caveat is that if there aren't enough rows that fulfill the
query, we'll return the full set of rows, but that may be less
than `num_after`.)  Likewise for `num_before`.

Before this change, we would sometimes return one too few rows
for narrow queries.

Now, we're still a bit broken, but in a more consistent way.  If
we have a query that does not match the anchor row (which could
be true even for a non-narrow query), but which does match lots
of rows after the anchor, we'll return `num_after + 1` rows
on the right hand side, whether or not the query has narrow
parameters.

The off-by-one semantics here have probably been moot all along,
since our windows are approximate to begin with.  If we set
num_after to 100, its just a rough performance optimization to
begin with, so it doesn't matter whether we return 99 or 101 rows,
as long as we set the anchor correctly on the subsequent query.

We will make the results more rigorous in a follow up commit.
2018-03-15 12:36:06 -07:00