Commit Graph

2571 Commits

Author SHA1 Message Date
Tim Abbott f2e3e779eb mypy: Properly annotate generic_bulk_cached_fetch.
Along with fixing some minor bugs, this requires extracting out the
default functions so that we can do type: ignores on them properly.

While we're at it, we switch to the Python 3 syntax.
2017-10-28 10:07:15 -07:00
Tim Abbott 73c27e1277 cache: Fix type aliasing of cached_objects.
Previously, it was converted from a CompressedItemT to an ItemT
without changing the variable name.
2017-10-28 10:01:44 -07:00
Tim Abbott 94c1da7025 cache: Move generic_bulk_cached_fetch typevars up a bit. 2017-10-28 10:00:43 -07:00
Tim Abbott 6c242379f4 bugdown: Remove unused caching imports. 2017-10-28 09:02:36 -07:00
neiljp (Neil Pilgrim) c063ba72a2 mypy: Improve typing of cache_with_key and cache decorators.
Fixes #1348.
2017-10-28 08:57:49 -07:00
Tim Abbott fa55d7ed33 retry_event: Remove requirement of failed_tries being already there.
This fixes a bug where retries in the signups queue threw an exception.
2017-10-27 18:15:53 -07:00
Tim Abbott bfa148ab39 requirements: Update mypy to latest version.
This involves fixing the mypy annotations in one file
2017-10-27 17:05:52 -07:00
Tim Abbott e5df05fd35 tests: Suppress logging spam in email mirror tests. 2017-10-27 16:06:03 -07:00
Tim Abbott 073c81bbe8 slow_tests: Increase limit for TOO slow notices. 2017-10-27 15:38:17 -07:00
Greg Price 6d403ff255 tests: Simplify set_http_host to dedupe its logic.
This will make it easier to change this logic.
2017-10-27 14:42:24 -07:00
Greg Price e4b4f67b44 subdomains: Tighten search for EXTERNAL_HOST in get_subdomain.
If a Zulip install at example.org got a request at an HTTP `Host`
like foo.example.org.evil.com (or even foo.example.orgevil.com),
we would accept it as subdomain foo.  This isn't likely to happen
in practice because it shouldn't pass ALLOWED_HOSTS, and it's not
obvious to me that anything untoward could be done with it even
if ALLOWED_HOSTS were set wide open, but if nothing else it
multiplies the cases in analyzing this logic.

The reason we had a loose match like this, I assume, is to allow
the user to come from arbitrary ports -- especially in development.
So tighten the pattern to allow just that, and add some tests for
that behavior and a comment explaining why this complication is
needed.
2017-10-27 14:42:24 -07:00
Greg Price 318682fd52 auth: Use URL rather than cookie to pass signed data cross-domain.
The cookie mechanism only works when passing the login token to a
subdomain.  URLs work across domains, which is why they're the
standard transport for SSO on the web.  Switch to URLs.

Tweaked by tabbott to add a test for an expired token.
2017-10-27 14:42:04 -07:00
Henrik Pettersson 09cd47c6ad Add UI for viewing and cancelling open Zulip invitations.
Lets administrators view a list of open(unconfirmed) invitations and
resend or revoke a chosen invitation.

There are a few changes that we can expect for the future:

  * It is currently possible to invite an email that you have already
    invited, it might make sense to change this behavior.

  * Resend currently sends an invite reminder instead of resending the
    original invite, this is because 'custom_body' was not stored when
    the first invite was sent.

Tweaked in various minor ways, primarily in the backend, by tabbott,
mostly for style consistency with the rest of the codebase.

Fixes: #1180.
2017-10-27 13:07:43 -07:00
Vishnu Ks 8c68a167fe backend: Allow Administrators to invite new users as admins.
Tweaked by tabbott to have the field before the invitation is
completed be called invite_as_admins, not invited_as_admins, for
readability.

Fixes #6834.
2017-10-27 11:19:38 -07:00
Steve Howell 4b78f69a87 tests: Introduce send_huddle_message().
The tighter interface here makes for shorter lines (or
fewer multi-line calls) and it leads to stricter type
checking.
2017-10-27 11:00:23 -07:00
Steve Howell 9707767b21 tests: Introduce send_personal_message().
The tighter interface prevents the need to specify
Recipient.PERSONAL (which can often be inaccurate in the
huddle case, anyway), and it prevents tests from confusingly
specifying a "subject" field for PMs.
2017-10-27 11:00:23 -07:00
Steve Howell 25a6fbd05d tests: Introduce send_stream_message().
Having send_stream_message() avoids the need to supply
Recipient.STREAM as a parameter, and it also uses the more
modern name of `topic_name` for topics.  Under the hood, it
avoids some annoying steps for re-formatting the recipients,
since we just have a single stream name.
2017-10-27 11:00:23 -07:00
Steve Howell 65b6676844 minor: Remove unused import (internal_send_message). 2017-10-27 10:48:11 -07:00
Steve Howell e5c5d10375 Rename subject vars in do_update_message(). 2017-10-27 10:48:11 -07:00
Steve Howell fbe5f93141 Rename subject -> topic_name in internal_send_message. 2017-10-27 10:48:11 -07:00
Steve Howell 973ee97612 Rename subject -> topic_name in internal_prep_message(). 2017-10-27 10:48:11 -07:00
Steve Howell f04550d143 Rename subject -> topic_name as var in check_message().
There were two variables, subject and subject_name, that
really had no clear distinction, so now they're just
topic_name.
2017-10-27 10:48:11 -07:00
Steve Howell 70b05998b1 Rename subject_name -> topic_name in check_send_message(). 2017-10-27 10:48:11 -07:00
derAnfaenger 64025d2ce7 integrations: Add Errbot documentation.
Fixes zulip/python-zulip-api#49.
2017-10-27 09:22:51 -07:00
Tim Abbott cab39dc6eb get_service_bot_events: Add some future-proofing.
The comment is the most important part here, but the new code
structure should help, too.
2017-10-26 22:16:47 -07:00
Tim Abbott b897c763f8 get_recipient_info: Simplify active_user_ids construction. 2017-10-26 22:16:47 -07:00
Steve Howell a70ede6c75 Allow "default" bots to see mentions on all streams.
This change allows normal bots to get UserMessage rows when
they are mentioned on a stream, even if they are not actually
subscribed to the stream.

Fixes #7140.
2017-10-26 22:16:47 -07:00
Steve Howell 53dbcbe2a4 refactor: Handle service bot mentions earlier in the code.
We now find all (possibly) relevant service bots for a message
in the call to get_recipient_info.  This allows us to eliminate
some code that would patch them after we rendered.

The get_service_bot_events() function will ignore any service
bots that weren't actually mentioned in the message (due to
backticks) or part of the active user ids.
2017-10-26 22:16:47 -07:00
Steve Howell 87b48f722d Add MentionData.get_user_ids().
This prepares us to take advantage of the user_ids
in subsequent commits related to `do_send_messages()`.
2017-10-26 22:16:47 -07:00
Steve Howell 8ac26dfb9b refactor: Introduce bugdown.MentionData class.
We now have a MentionData class that encapsulates
the users who are possibly mentioned in a message.

Not that the rendering code may not keep all the mentions,
since things like backticks will suppress the mention.

We populate this now in do_send_messages, so that we can use
the info earlier in the message-sending process.  This info
now gets passed down the call stack as an optional parameter.

Note that bugdown.convert() still populates the data when its
callers decline to pass in a MentionData object.

This is mostly a preparatory commit, as we don't take advantage
of the data yet in do_send_messages.
2017-10-26 22:16:47 -07:00
Tim Abbott 8e2cdedf9a lint: Fix lines in Python codebase longer than 120 characters. 2017-10-26 17:47:30 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Steve Howell 635675fe48 Reduce queries needed for sending messages.
In do_send_messages, we only produce one dictionary for
the event queues, instead of different flavors for text
vs. html.  This prevents two unnecessary queries to the
database.

It also means we only put one dictionary on the "message"
event queue instead of two, albeit a wider one that has
some values that won't be sent to the actual clients.

This wider dictionary from MessageDict.wide_dict is also
used for the `feedback_messages` queue and service bot
queues.  Since the extra fields are possibly useful down
the road, and they'll just be ignored for now, we don't
bother to remove them.  Also, those queue processors won't
have access to `content_type`, which they shouldn't need.

Fixes #6947
2017-10-26 16:35:28 -07:00
Steve Howell 9b6a4d0b16 refactor: Extract MessageDict.finalize_payload(). 2017-10-26 16:35:28 -07:00
Steve Howell df93a99b50 Cache only one row per message.
Before this change, we populated two cache entries for each
message that we sent.  The entries were largely redundant,
with the only difference being whether we sent the content
as raw markdown or as the rendered HTML.

This commit makes it so we only have one cache entry per
message, and it includes both content and rendered_content.

One legacy source on confusion here is that `content`
changes meaning when you're on the front end.  Here is the
situation going forward:

    database:
        content = raw
        rendered_contented = rendered

    cache entry:
        content = raw
        rendered_contented = rendered

    payload for the frontend:
        content = raw (for apply_markdown=False)
        content = rendered (for apply_markdown=True)
2017-10-26 16:35:28 -07:00
derAnfaenger 1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
derAnfaenger 18e5bcbbb1 tests: Enable call_consume_in_tests for email mirror queue. 2017-10-26 14:53:27 -07:00
derAnfaenger 17949adf11 embedded bots: Add StateHandler remove() function. 2017-10-26 14:49:50 -07:00
Tim Abbott b8658c6901 default stream groups: Use cleaner system for error handling.
Wherever possible, we always want to move checking for error
conditions to the views code, so that we don't need to worry about
handling failures with (in this case) a user that's half-created
because a DefaultStreamGroup doesn't exist.
2017-10-26 11:38:27 -07:00
Vishnu Ks eecdc5bb61 registration: Add option to choose default stream groups.
This effectively implements the feature of default stream groups,
except for a UI, nice styling, etc.

Note that we're careful to not have this do anything in an
organization that doesn't have any default stream groups.
2017-10-26 11:34:06 -07:00
Greg Price 66ebe9720f subdomains: Refactor get_subdomain a bit.
The helper _extract_subdomain doesn't have a super meaningful
interface, and this is its one callsite.  So just inline it.
2017-10-26 10:29:17 -07:00
Greg Price cc32634439 subdomains: Simplify is_subdomain_root_or_alias.
This logic is equivalent, though it takes a couple of readings
to convince oneself of that.  This version should then be
easier to reason about.
2017-10-26 10:29:17 -07:00
Greg Price 093bae4bc5 subdomains: Fix some implicit uses of "" for the root subdomain.
These are just instances that jumped out at me while working on the
subdomains code, mostly while grepping for get_subdomain call sites.
I haven't attempted a comprehensive search, and there are likely
still others left.
2017-10-26 10:29:17 -07:00
Greg Price be309bc8b1 subdomains: Complete the refactor to user_matches_subdomain.
Now that the old `check_subdomain` has no callers except in
implementing the new, improved interface `user_matches_subdomain`,
inline it into that.  Also simplify the Boolean logic a bit.
2017-10-26 10:29:17 -07:00
Greg Price c9457d4af0 subdomains: Refactor check_subdomain to a clearer interface.
Now that every call site of check_subdomain produces its second
argument in exactly the same way, push that shared bit of logic
into a new wrapper for check_subdomain.

Also give that new function a name that says more specifically what
it's checking -- which I think is easier to articulate for this
interface than for that of check_subdomain.
2017-10-26 10:29:17 -07:00
derAnfaenger 8680541912 embedded bots: Add message flow tests. 2017-10-25 16:13:37 -07:00
derAnfaenger ce4ba9c178 bot services: Use call_consume_in_tests. 2017-10-25 15:56:12 -07:00
derAnfaenger 0473692abd embedded bots: Properly reply to PMs and streams
This fixes an exception occurring when engaging an embedded
bot in a PM, makes it respond as itself instead of the sender,
and makes it respond to the PM conversation it is engaded in.
2017-10-25 15:56:12 -07:00
derAnfaenger e041abd300 tests: Add create_test_bot() helper function. 2017-10-25 15:46:07 -07:00
derAnfaenger 3cf88d8c09 embedded bots: Add virtual_fs bot.
This will be our main bot for testing the StateHandler code.
2017-10-25 15:39:28 -07:00