Commit Graph

125 Commits

Author SHA1 Message Date
rht 926f0a7b45 zerver/tornado: Remove unused imports (F401). 2017-11-07 16:36:57 -08:00
rht fa7016644e refactor: Remove six.moves.https_cookies import. 2017-11-07 10:51:44 -08:00
rht 9c7d5812ce refactor: Remove six.moves.urllib.parse import. 2017-11-07 10:51:44 -08:00
rht 19bd335cbb Change urllib import to be Python 3-specific. 2017-11-07 10:46:42 -08:00
Steve Howell 2bbfda041a Support client_gravatar field for event registration.
This commit allows clients to register client_gravatar=True, and
then we recognize that flag for message events.  If the flag is
True, we will not calculate gravatar URLs and let the clients do
it themselves.  (Clients can calculate gravatar URLs based on
emails with just a little bit of code.)
2017-11-07 10:42:17 -08:00
Steve Howell 88e1e284bb Restructure send-message code for gravatars.
This refactoring doesn't change behavior, but it sets us up
to more easily handle a register setting for `client_gravatar`,
which will allow clients to tell us they're going to compute
their own gravatar URLs.

The `client_gravatar` flag already exists in our code, but it
is only used for Django views (users/messages) but not for
Zulip events.

The main change is to move the call to `set_sender_avatar` into
`finalize_payload`, which adds the boolean `client_gravatar`
parameter to that function.  And then we update various callers
to supply that flag.

One small performance benefit of this change is that we now
lazily compute the client message payloads in
`event_queue.process_message_event` now, so this will improve
performance if all interested clients have the same value of
`apply_markdown`.  But the change here is really preparing us
for the additional boolean parameter, which will cause us to
have four variations of the payload.
2017-11-07 10:36:02 -08:00
Steve Howell 2a6c6518fc refactor: Don't shadow message_dict in process_message_event(). 2017-11-07 10:36:02 -08:00
rht 2ceb0b45b3 zerver/tornado: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
neiljp (Neil Pilgrim) 54cbb808e6 mypy: Add explicit Any parameters to get_user_events return type. 2017-11-04 19:47:44 -07:00
Greg Price 68b0a419ec decorator: Cut a bunch of dead imports of two view decorators.
Saw these when grepping for these two decorators; they're actually
more numerous than the surviving use sites are.  Cut out the noise.
2017-11-04 19:27:00 -07:00
Tim Abbott c3db9513f2 tornado: Delete fake_message_sender.
This test helper method duplicated a bunch of logic in
`zerver/worker/queue_processors.py` in a specialized fashion for the
tests.  Now that we're using `call_consume_in_tests` in this code
path, we don't need it.
2017-11-03 14:11:23 -07:00
Tim Abbott eade4d0052 tornado: Use call_consume_in_tests for message_sender queue.
This increases test coverage of queue_processors.py significantly,
with no real cost, achieving valuable progress torwards #6542.
2017-11-03 14:09:48 -07:00
neiljp (Neil Pilgrim) 7288451329 mypy: Slightly clarify Callable parameter to WebsocketClient.__init__. 2017-10-31 00:03:35 -07:00
neiljp (Neil Pilgrim) 452802bf75 mypy: Add Callable parameters/returns to AsyncDjangoHandler in handlers.py, 2017-10-31 00:03:35 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
rht 1047733486 zerver/tornado: Use python 3 syntax for typing. 2017-10-26 21:58:22 -07:00
Steve Howell 635675fe48 Reduce queries needed for sending messages.
In do_send_messages, we only produce one dictionary for
the event queues, instead of different flavors for text
vs. html.  This prevents two unnecessary queries to the
database.

It also means we only put one dictionary on the "message"
event queue instead of two, albeit a wider one that has
some values that won't be sent to the actual clients.

This wider dictionary from MessageDict.wide_dict is also
used for the `feedback_messages` queue and service bot
queues.  Since the extra fields are possibly useful down
the road, and they'll just be ignored for now, we don't
bother to remove them.  Also, those queue processors won't
have access to `content_type`, which they shouldn't need.

Fixes #6947
2017-10-26 16:35:28 -07:00
Steve Howell 0cef7c9fd5 Refactor: Extract get_client_info_for_message_event().
This removes some clutter from process_message_event, and it
makes process_message_event() a bit easier to test.
2017-10-26 16:35:28 -07:00
Tim Abbott e98ca0714b notifications: Simplify how triggers are passed around.
This removes the utterly unnecessary `triggers` dict (which always was
a dict with exactly one value True) in favor of a single field,
'trigger'.

Inspired by Kunal Gupta's work in #6659.
2017-10-18 21:42:05 -07:00
Tim Abbott 513b6d624f notifications: Fix double-sending in missedmessage_hook.
While the missedmessage_hook logic originally did a reasonably good
job of avoiding double-sending notifications, there was a corner case
it didn't handle, namely a user who had been presence-idle when a
message was sent and became also event-queue-idle as well within the
next 10 minutes.  For those users, they got a notification at message
send time, and the missedmessage_hook would deliver it a second time.

We fix this by just checking the conveniently available push_notified
and email_notified variables that indicate whether the message already
had a notification triggered.

Fixes #7031.
2017-10-17 21:54:03 -07:00
derAnfaenger af699500b7 tests: Add option to call queue processor consumer.
This makes tests of queue processors more realistic,
by adding a parameter to `queue_json_publish` that
calls a queue's consumer function if accessed in a test.

Fixes part of #6542.
2017-10-16 23:20:13 -07:00
Steve Howell f0194f1821 Reset client descriptors at the start of event tests.
Fixes #6958
2017-10-12 10:27:16 -07:00
Tim Abbott 539ff20ec2 socket: Add logging for CSRF issue in logging code.
This should help make it possible to debug #6961.
2017-10-11 22:39:14 -07:00
Steve Howell 9ecf41980c Remove message.is_mentioned in message events.
This field would get overwritten with an improper value when
we looped over multiple clients, due to not making full copies
of the message dictionary.  This failure would be somewhat
random depending on how clients were ordered in the loop.

The only consumers of this field were the mobile app and the
apply-events-to-unread-counts logic.  Both of these will now
use `flags` instead.
2017-10-11 16:55:34 -07:00
Steve Howell 10a30bece1 Rename presence_idle_userids -> presence_idle_user_ids. 2017-10-07 12:16:45 -07:00
Tim Abbott f11832536c event_queue: Fix a now-incorrect mypy annotation.
It's not clear tagging things like this as an Iterable is a good idea,
when it's in fact definitely a list.
2017-10-05 23:11:16 -07:00
Tim Abbott fbdeb63bc9 tornado: Add debug logging for time consumed processing an event.
I've occasionally found this logging helpful.
2017-10-05 22:08:29 -07:00
Steve Howell aae0b2a826 Notify offline users about edited stream messages.
We now do push notifications and missed message emails
for offline users who are subscribed to the stream for
a message that has been edited, but we short circuit
the offline-notification logic for any user who presumably
would have already received a notification on the original
message.

This effectively boils down to sending notifications to newly
mentioned users.  The motivating use case here is that you
forget to mention somebody in a message, and then you edit
the message to mention the person.  If they are offline, they
will now get pushed notifications and missed message emails,
with some minor caveats.

We try to mostly use the same techniques here as the
send-message code path, and we share common code with the
send-message path once we get to the Tornado layer and call
maybe_enqueue_notifications.

The major places where we differ are in a function called
maybe_enqueue_notifications_for_message_update, and the top
of that function short circuits a bunch of cases where we
can mostly assume that the original message had an offline
notification.

We can expect a couple changes in the future:

    * Requirements may change here, and it might make sense
      to send offline notifications on the update side even
      in circumstances where the original message had a
      notification.

    * We may track more notifications in a DB model, which
      may simplify our short-circuit logic.

In the view/action layer, we already had two separate codepaths
for send-message and update-message, but this mostly echoes
what the send-message path does in terms of collecting data
about recipients.
2017-10-03 15:57:06 -07:00
Tim Abbott 3a286756ff tornado: Document missedmessage_hook in more detail. 2017-10-03 10:55:38 -07:00
Steve Howell e660428c21 Rename missed_message_userids to presence_idle_userids. 2017-10-02 15:19:28 -07:00
rht 426548097e zerver/tornado: Remove from __future__ import *. 2017-09-28 10:14:03 -07:00
rht 241e318eba zerver/tornado: remove `import six`. 2017-09-27 19:10:28 -07:00
rht b8aa92194c zerver/tornado: Remove print_function. 2017-09-27 18:05:45 -07:00
rht a7803cbf1b zerver/tornado: remove `import six`.
Tweaked by tabbott to remove the changes in
zerver/tornado/handlers.py, which are hard to test.
2017-09-27 17:07:08 -07:00
Tim Abbott 06e6b3abe1 event_queue: Fixed stream_push_notify code path in missedmessage_hook.
This finishes implementing the stream_push_notify feature, which
previously didn't work if you were not idle when the message was sent.
2017-09-27 15:57:48 -07:00
Tim Abbott 2b1ea1555c event_queue: Simplify API of maybe_enqueue_notifications. 2017-09-27 15:57:47 -07:00
Tim Abbott fa461c6a4f missedmessage_hook: Fix name of ClientDescription argument.
This was previously incorrectly named to suggest it was the EventQueue
object.
2017-09-27 15:57:47 -07:00
Tim Abbott d6732d0781 event_queue: Fix handling of private messages in missedmessage_hook.
Previously, we didn't correctly trigger email/push notifications for
PMs in these cases.
2017-09-27 15:57:47 -07:00
Tim Abbott 71fc8363b4 tests: Add a test suite for maybe_enqueue_notifications.
This ensures that as we expand the logic for under what circumstances
email and push notifications should be sent, we can be confident about
this code path always doing the right thing.
2017-09-27 15:55:29 -07:00
Tim Abbott 830d3e8d07 event_queue: Use maybe_enqueue_notifications in missedmessage_hook.
This fixes a problem introduced in the recent refactoring where
`triggers` would not be set correctly when a push or email
notification was triggered by missedmessage_hook.

Fixes #6612.
2017-09-27 15:55:29 -07:00
Tim Abbott 9749826e9c event_queue: Refactor code for checking if a message was mentioned.
Now, the two code paths do the same thing for this check.

It seems like there may be more work to do here, in that
wildcard_mentioned messages seem to not be eligible for sending
email/push notifications.  We probably want to add some logic there
for the user doing the mention to control whether or not it does.
2017-09-27 15:55:29 -07:00
Tim Abbott f68089fbe1 event_queue: Extract maybe_enqueue_notifications.
Conceptually, this code path should be shared between the
process_message handler and the garbage-collection hook.
2017-09-27 15:55:29 -07:00
rht 74fd3d9f31 zerver/tornado: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott d06cb4e4fd event_queue: Make path forward for push/email notifications clearer.
This is a nonfunctional refactor, designed primarily to make it
simpler to extend this code path when we later add support for
controlling whether email notifications go out on stream messages.
2017-09-15 01:01:11 -07:00
Tim Abbott d0e8163f13 event_queue: Remove some unnecessary parenthesis. 2017-09-15 01:01:11 -07:00
Tim Abbott e085af3324 Fix stream_push_notify feature to not send emails.
Previously, due to a logic bug, this feature would also send email
notifications for all messages on the stream, which is definitely not
the intent.  The recent refactoring we just did makes the logic more
obvious.
2017-09-15 01:01:11 -07:00
Tim Abbott 22ea2a5858 event_queue: Separate email and push notification loops. 2017-09-15 01:01:11 -07:00
Tim Abbott 7fa0325fb5 event_queue: Refactor notified logic. 2017-09-15 01:01:11 -07:00
Greg Price c4b506998f tornado: Disable routine logging in dev.
This creates a lot of logging noise, and also causes confusion
for new contributors when something isn't working as they expect
and they aren't sure if this message is normal or an error.
2017-09-14 12:38:57 -07:00
Tim Abbott 5722237f59 push: Rename received_pm to private_message.
This is a clearer name for this now more broadly used interface.
2017-09-14 05:41:37 -07:00