Commit Graph

537 Commits

Author SHA1 Message Date
Steve Howell 57030f53cc Use cache_set_many() in the flush_realm() code path.
The function update_user_profile_caches now operates on a list
of user_profiles, so callers like flush_realm() can benefit from
having a single cache_set_many() call.  This slightly complicates
the call from flush_user_profile().

(imported from commit e064871d849b873c6ca388f00d4f7afaba1bf222)
2014-01-28 17:40:01 -05:00
Steve Howell bbafd1dda8 Have flush_realm() make O(1) cache_delete() calls in a couple places.
For the realm-wide caches of active user dicts and alert words, just
make a single call to cache_delete() when you are deactivating a
realm.  Before this change, we were doing O(N) cache_deletes as
part of the code path through flush_user_profile().  Now we just
call update_user_profile_caches() directly to clear the user_profile
caches.

This change also sets us up to turn flush_realm() into a post-save hook.

(imported from commit 699b4ea226ae15fc8c402cb4bc64ff6bdc041fc2)
2014-01-28 17:40:01 -05:00
Steve Howell 0eed5d20d2 Create cache.flush_realm() and call it from do_deactivate_realm().
This is a slight behavior change, as we now flush user_profile
caches for bots as well as humans.

(imported from commit 24c72c44d851ee4c66a67a4728cd6c548faeedcd)
2014-01-28 17:40:00 -05:00
Steve Howell dbc1172a20 Extract update_user_profile_caches().
This function updates all the user_profile-related caches
that are keyed on a per-user basis.

(This had some test coverage already.)

(imported from commit 37979400514a7b46a6dcb7e36665b0fee2f3c525)
2014-01-28 17:40:00 -05:00
Steve Howell 3776d11c3a Rename update_user_profile_cache() to flush_user_profile().
(imported from commit 1fddb8b8dfe7943ace70c51d762eeae2ee1203a0)
2014-01-28 17:40:00 -05:00
Jason Michalski 05e00575bb Only send stream update events to clients that need them
Stream name and descriptions updates were being sent to all of the
active users on a realm. They are now only send to users who would have
information about that stream.

(imported from commit 2621ee8029f7356bf44ec493d7b5361bd546a8f5)
2014-01-28 17:12:15 -05:00
Steve Howell 1dd674c7ae Make import statements take less vertical space.
(imported from commit 4d5901491d6c8c21fe045ac9751bc80613f94a59)
2014-01-28 17:04:51 -05:00
Steve Howell 80af0b8134 Move AuthedTestCase to test_helpers.py
(imported from commit bdaf63c5ed4e8d7c19e9c19f79151e8e7885c53e)
2014-01-28 17:04:51 -05:00
Steve Howell a8f2ecf48c Move context managers to zerver/lib/test_helpers.py
(imported from commit f57678fcb68aecdbb70ad86cd02cc5441695aaf5)
2014-01-28 17:04:50 -05:00
Steve Howell 8aaf1cf8c2 Add API for removing default streams from a realm.
(imported from commit 06433126e31b372dff8a23eb8bbea14515138c85)
2014-01-28 14:30:37 -05:00
Steve Howell af9a1e967f Add API for adding default streams to a realm.
(imported from commit 0f48f722dd618c0d054aafdf3d931ef014a5c3ee)
2014-01-28 14:30:02 -05:00
Steve Howell 4cf677eb6d Add do_remove_default_stream().
(imported from commit 559dcd93ca43f4ffba58119fe429df3f6843fd63)
2014-01-28 14:28:51 -05:00
Steve Howell 4ed661ca71 Add do_add_default_stream().
(imported from commit 34c96bdc3f5c2f7f119d2efe1606316878d34d03)
2014-01-28 14:28:51 -05:00
Steve Howell af00f4609a Extract get_default_streams_for_realm().
(imported from commit 6ef41615eb660fd161bfd53de09fea56266ddd2d)
2014-01-28 14:28:50 -05:00
Tim Abbott 3fddc11cc2 Process message events more similarly to how other events are handled.
(imported from commit 5ebcd7a10fe54a8ecafb9550348a1e0418d4cbda)
2014-01-28 13:42:50 -05:00
Tim Abbott 1cc3a6a903 Switch update_message to a generic event type.
(imported from commit db3c4d2326727ca05d25ab43ebc061d3e5cd0630)
2014-01-28 11:01:30 -05:00
Tim Abbott 167b883c38 Clean up the update_pointer tornado notification code path.
(imported from commit 94997dea54e6cc8e5b5a314a95d6114773462608)
2014-01-28 11:01:30 -05:00
Tim Abbott 0507b1bb06 Split out do_update_pointer from update_pointer_backend.
(imported from commit 25da1b324a9170677b0f6fc2e29841d6f2af9dec)
2014-01-28 11:01:29 -05:00
Tim Abbott 5ccb8b1bea Add tornado_callbacks.send_event with clean API for sending events.
(imported from commit 07209a1744ac9372273858a3e69e8a2fd9e06e06)
2014-01-28 11:01:29 -05:00
Tim Abbott 3bcba79e77 Send message data to Tornado via rabbitmq instead of memcached.
This is a lot simpler and eliminates a possible failure mode in the
data transfer path.

(imported from commit 19308d2715bbd12dc9385234f1d9156f91bdfae0)
2014-01-28 11:01:29 -05:00
Leo Franchi 0917a27b79 Document realm filter regexp oddities
(imported from commit 987d8ea679f40c31f2fe2280b6d585f82b20e606)
2014-01-27 13:43:55 -05:00
Leo Franchi 883157893c Use a UserMessage flag to indicate /me messages
(imported from commit ea503b0d3eb6e90230c0859be96ede60faf5c2bd)
2014-01-27 12:07:48 -05:00
Steve Howell 6fff91493c Fix bug in apply_events() for removing subscribers.
To mutate the state for removing subscribers, the previous
code was essentially adding in event['subscriptions'] to
state['unsubscribed'], but that was a naive approach, since
the event object only has the name of the subscriber, and not
the full subscription info.

We instead effectively copy records over from state['subscriptions']
to state['unsubscribed'], and we also do surgery on the subscribers
that made me need to add the user_profile parameter to apply_events().

With the code apparently working now, I was able to remove the
match_except() test helper and use a more thorough matcher in
the test on do_remove_subscription().

Part of fixing the "remove" case was cleaning up the "add" case,
since they aren't quite symmetric opposites of each other, although
under this refactoring they now share the new name() helper.

(imported from commit 0deab67d0c7b08b3ad962493efae3762a835fd29)
2014-01-27 10:41:29 -05:00
Tim Abbott da90d63046 Split out zerver/lib/notifications.py from actions.py.
(imported from commit 784b82834ee4fcb4431e77f8fb1c526f8eec82ad)
2014-01-24 17:33:56 -05:00
Steve Howell 1d95812770 Add is_admin to page_params.
Because full_name and is_admin changes go through many similar,
generic codepaths, it is almost more work at this point to keep
is_admin out of page_params as it is to just put it in.  So
I put it in.  This should pave the path for showing admins in
the GUI.

This commit actually starting by my adding a test
that calling apply_events() with the notification you get
when calling do_change_is_admin() updates
state['realm_users'] to be similar to what you would get
out of fetch_initial_state_data().  We didn't have test coverage
there before.  Making that test pass forced my hand to
either add is_admin to page_params or to special-case
apply_events() not to update page_params with is_admin.  I
chose the former approach.

(imported from commit 1e49d59c66540014284529c29d5007224be6a0c6)
2014-01-24 16:34:12 -05:00
Steve Howell 4ad8ad2b5a Extract get_realm_user_dicts() from fetch_initial_state_data().
(imported from commit 2f0f52d90eba22a4b72bd6173e60ab463482b7d6)
2014-01-24 16:34:12 -05:00
Jason Michalski 1f98c4f4ce Add support for updating the stream description
The stream description can now be updated, but the UI is not yet updated
on success.

(imported from commit f45e0a9d5138d828ae98d6d49645ab3ddc966704)
2014-01-24 14:47:44 -05:00
Jason Michalski 4104f00229 [schema] Add description to streams and display it
A description was added to the streams and it is now displayed on the
subscriptions page. It can not be set in the UI yet.

(imported from commit 81d08b65eee42dba87cd99dd5bd30106c4eb6c6a)
2014-01-24 14:47:44 -05:00
Luke Faraone 1c67d05e5f Handle duplicate GCM registrations by dropping the old ones.
While we're at it, lets comment up the function so I know what this is
doing next time :)

(imported from commit e745be75fcd6dbce9997e1d73464619fc8b73996)
2014-01-23 19:21:28 -05:00
Jessica McKellar ee7a1f0953 tutorial: add helpful in-app links and be consistent with realm welcome message.
(imported from commit 2057dbe2606d32721192021ed2046ed9f412d8a7)
2014-01-23 17:06:29 -05:00
Jessica McKellar 086eab8bb5 bugdown: support modal in-app links in messages.
(imported from commit 0e3c0ff61cf9a84bae81e37b0a6d79831dea9224)
2014-01-23 17:06:29 -05:00
Leo Franchi 56d9446c41 [socket] Use socket req ids that contain queue ids
(imported from commit 781eafe60bf117d79ce3a30deea02ec1e875060a)
2014-01-23 16:28:58 -05:00
Leo Franchi 7743aa191c Pass through local_id and sender_queue_id in non-rabbitmq case
(imported from commit 8989225a0943b39e45bd5debe405acb9768acc8c)
2014-01-23 16:28:58 -05:00
Steve Howell 2283aa52cb Use "stream" as our type for updates to Stream.name.
(imported from commit 2eedbff5fac77b9e654ba88900167048573e4056)
2014-01-23 13:27:11 -05:00
Steve Howell bc6da3717f Fix obscure bug with page_params.is_bot.
If a name change event arrived during the call to
fetch_initial_state_data(), we would call apply_events() to
update the data structure that eventually becomes page_params.
Our update code, instead of surgically updating the fields, was
just overwriting the record, which led to is_bot being taken
out of the record when only full_name was in the event.

Apart from fixing the "update" case to do the right thing,
this commit also does a bit of cleanup on the code handling
"realm_user" events to make it more generic in how it does
add/remove/update.  If we could standardize our events a bit
more, this could eventually lead to DRYing up some of the
apply_events() code.

(imported from commit 772e2fcd6a5605ccb6e8d1bc499b5f336934cf3c)
2014-01-23 13:01:04 -05:00
Steve Howell 3b12e63d8d Send notifications for is_admin changes.
(The client doesn't do anything with them yet.)

(imported from commit df2f406644fe271af53e32a6b3521752b6a1ac41)
2014-01-22 14:43:28 -05:00
Leo Franchi c0fdbf81cf Use the feedback connection for getting APNS feedback
(imported from commit d371188e0437606b3ff435ee9c005cee2849f6ae)
2014-01-22 13:50:15 -05:00
Leo Franchi 4dac0e7d37 Print out APNS tokens in b64 format for ease of use
(imported from commit d38eb1629a599681c42f1f8d817c90229b547516)
2014-01-22 11:59:02 -05:00
Leo Franchi 6903d87809 Retry on APNS failure / retry request
(imported from commit b9d1c414731de7a4ad5d3bc93c3dce1286cbdb38)
2014-01-22 11:59:02 -05:00
Leo Franchi c0d3b2fd89 Remove APNS tokens from our database if we get an invalid token notification
(imported from commit 5ec3c053684f9574a31103e7821309de883baa82)
2014-01-22 11:59:02 -05:00
Leo Franchi 226d894257 Prefix apple-specific APNS logs with APNS:
(imported from commit 2fa866b3a626c3e42f340bc68e8bc27aef6fd521)
2014-01-22 11:59:02 -05:00
Jason Michalski c30a411c10 Add custom markdown tag to render a stream subscribe button
When new streams are created we now send a message with a custom
markdown tag that renders a subscribe button.

(imported from commit 9dfba280b3b4ff4f32f6431ef9227867c8bf4b40)
2014-01-22 11:28:31 -05:00
Leo Franchi 669951b631 Send push notifications regardless of idle status for robinhood.io, zulip.com
(imported from commit 49d2c954f6537d28ef3f8ccad669c56430afdbde)
2014-01-21 14:48:57 -05:00
Jason Michalski 4c90392261 [schema] Add an per-user option to disable notification on new streams
Added a default_desktop_notifications boolean to userprofile with a UI
in Zulip Labs. This flag is used to default the notification flag on new
subscriptions.

(imported from commit a25223cc5ecf09980cf877991e25034bb3fd4046)
2014-01-16 20:13:34 -05:00
Jessica McKellar a1d128b6cb Flush memcached on a stream privacy change.
Otherwise uninvited users can join a stream recently turned
invite-only.

(imported from commit ab43bf3b856a0e3b71e6e06cee1519214b7763cd)
2014-01-15 17:13:56 -05:00
Waseem Daher 9eed2c50b9 email-mirror: Support '.' as an alternate for '+' (and bugfix).
Google Groups won't let you add an email address that has a '+' in it
as a member of a group, so we allow '.' as well.

This commit also fixes a current issue with email mirroring, where it
doesn't work if you had a + in the stream name.

This fixes Trac #2102.

(imported from commit 9a7a5f5d16087f6f74fb5308e170a6f04387599e)
2014-01-15 16:07:19 -05:00
Jason Michalski 60f878b45e Limit cross-realm private messages to zulip.com users
Cross-realm private messages are only used to respond to support
requests. So now cross-realm private messages are only allowed if
exactly two realms are in the private message and one of them is
zulip.com.

(imported from commit f01a2824e214682acb22a6995714a9d1b0d0c66f)
2014-01-15 16:07:19 -05:00
Luke Faraone 0b5036e0fb Let message recipient in process_message be passed as an argument.
(imported from commit cfcfb9edf748a35d279235ee11a05b1a00c934b5)
2014-01-15 16:07:18 -05:00
Zev Benjamin db23674749 Do query time tracking at the psycopg2 level instead of the Django level
This allows us to track the query time of SQLAlchemy and raw queries.

(imported from commit 818a4ee41786ffc57b80d7ed1cfba075f29b6ee5)
2014-01-14 11:47:12 -05:00
Tim Abbott e38ac2f9ac Index the presence data sent from Django to Tornado by ID.
(imported from commit 8ffb70562b49ba93c5b0927b34dc57a743d49564)
2014-01-13 16:54:06 -05:00
Tim Abbott e890f06c1c Move missedmessage hook checks on UserProfile settings to workers.
This does result in a few more rabbitmq events to be processed (though
a negligible number compared to what we already do), but it saves a
database query from inside Tornado whenever we occasionally have a
cache miss looking up the UserProfile, which is far more important.

(imported from commit a553a00a3004ba27bfb54ffbc3e9c9b170ebae4d)
2014-01-13 16:54:05 -05:00
Zev Benjamin 22ae828685 Support Tornado 3
The only thing that needed changing was our instrumented IOLoop.

(imported from commit 17833500f03949f45a546050a1f99675e9f6b1b5)
2014-01-10 21:39:06 -05:00
Steve Howell e832e937c5 Make emojis be 20px tall in missed message emails.
(imported from commit 687b099f19d69f41876571c08a0627f061067f86)
2014-01-10 21:39:05 -05:00
Steve Howell 1a679795e1 Flatten the code for build_message_payload.
(imported from commit b58c91468b36d33c163d4994b501eac796d10e8f)
2014-01-10 21:39:05 -05:00
Waseem Daher fe748f7633 Make Twitter preview look like other inline image previews.
(imported from commit ca4ff9146679a5348bf53d06e992972e9a4b628a)
2014-01-10 21:39:04 -05:00
Steve Howell 3e48b4fd95 Update mentioned flags after message edits.
(imported from commit 67886db571193a0f53cf72aa00af9e5e75729dc0)
2014-01-10 21:39:03 -05:00
Steve Howell 8d79606136 Update has_alert_word flag after message edits.
(imported from commit cad1b5646742506c1e84a42366fd58aaaf6dc17d)
2014-01-10 21:39:02 -05:00
Steve Howell 0326fa92b4 Send UserMessage flags for update_message events.
When we edit a message, send out UserMessage flags to the recipients.
This sets the stage for making sure that changes related to
user-specific alert words or mentions get sent out to users.

(imported from commit bce1de19acef44b5e106352f261203352ece02b9)
2014-01-10 21:39:02 -05:00
Jason Michalski 9e822403bb Linkify twitter media links
Image and video links in the twitter API are media and need to be
handed on separately. We also include a preview image if the media link
is a to a picture.

(imported from commit 2bd00d267e51b29ad0ba681195b2bfea9b991d8c)
2014-01-10 21:39:02 -05:00
Jason Michalski 1352dccea5 Add support to linkify links and mentions on twitter
This converts links in tweets to a tags. We also convert the displayed
text to the target of the twitter short URL. Mentions are linked to the
users twitter page.

(imported from commit 192d5546a7eea82759f9ae30d82c102aed15ff71)
2014-01-10 21:39:01 -05:00
Tim Abbott 1470cb8bf4 do_send_messages: Only pass active recipient users to Tornado.
Previously, we were processing in Tornado mirroring dummy users (and
deactived users) as recipients -- resulting in bugs where the missed
message hooks would fire for these nonexistent users.

Our Tornado real-time delivery system only needs the list of active
users both for delivery and for presence information updates.

(imported from commit b81143f106a4d0eefa4b838e7c074b2963259746)
2014-01-10 21:39:00 -05:00
Tim Abbott 4bf3ace444 [manual] Allow signups for emails held by non-MIT mirror dummy accounts.
Before this is deployed to prod, we need to manually frob our database
to set the is_mirror_dummy=True bit for all existing mirror users.

(imported from commit 39f1938cef091cf1d7d97307f76b137fe1d92b6c)
2014-01-10 21:38:59 -05:00
Jessica McKellar d3f6c4ef36 Log the actual sending of digest emails in addition to queuing possible ones.
(imported from commit e43ac59e71620fe715db462347db228d5a950aed)
2014-01-10 21:38:59 -05:00
Tim Abbott 47500d8352 [schema] Add a bit for whether inactive users are mirror dummies.
(imported from commit bb21bb2c62ac09742484d7a4ca8907e7d864b982)
2014-01-09 11:08:35 -05:00
Jessica McKellar 8e3911269e Strip whitespace from emails before looking up DB entries in a few places.
(imported from commit eafdce98450d16c0ca51c4cb17a139811a6124bc)
2014-01-07 20:24:21 -05:00
Jessica McKellar 511e1a12e3 Add a management command to deactivate old realms.
(imported from commit f6b1d0cb5b10f91683afcffea88b23b1a75a9ad2)
2014-01-07 20:24:20 -05:00
Leo Franchi c7836626ea Accept a pass-through local_id option in send_message
(imported from commit 3f7b8e862a92d8a11b68da9ff23f711b6b25b5e2)
2014-01-07 17:33:34 -05:00
Leo Franchi 76892c52aa Send realm filters to clients
(imported from commit a3ba31701baa4d515ea3bf36c39d7255f1dd20d6)
2014-01-07 17:33:33 -05:00
Luke Faraone df4b2d9f85 Break out non-Twisted email mirror code into separate library
(imported from commit 45c3e70ba81cdb44f1e9db4f162e5f2d2fe5ead6)
2014-01-06 13:36:53 -05:00
Steve Howell 654ce3e0c7 Add make_stream_private endpoint.
(imported from commit b504c24fd651ecf8814e64725a9f303dc085b14e)
2014-01-06 11:59:47 -05:00
Steve Howell e0c62c5eda Add make_stream_public endpoint.
(imported from commit 03d4cff5587d0aa149997f2f6ae28ec4ede95d7a)
2014-01-06 11:59:47 -05:00
Zev Benjamin fb602da502 Don't log exceptions that we know aren't bugs when adding events to event queues
(imported from commit 55074e68603119bd11cc5218453a4e1eab710feb)
2014-01-03 14:40:55 -05:00
Jessica McKellar 30367d0c0f digest: don't include automated messages in hot conversation accounting.
(imported from commit ab8db46c644b3cf66a4bc654d204d870b8fae91f)
2013-12-31 16:45:21 -05:00
Jessica McKellar 8091634814 Move sent_by_human check to a method on the Message class.
(imported from commit 2ef49051932f9633ad11130952591c5c5b147f12)
2013-12-31 16:42:38 -05:00
Jessica McKellar 61103d8349 digest: On an app server, only send digest emails for domains it manages.
(imported from commit 9aea636a19eb6fdefacdc7dc393bf075a05862c5)
2013-12-20 12:50:23 -05:00
Tim Abbott 2264a07c03 Update server-side app checks to handle the new app names.
(imported from commit da7b2c06b413add1881b2d80a89794d0db9c810f)
2013-12-19 17:21:26 -05:00
Tim Abbott a70072f5cf socket: Don't throw traceback when sessions are expired.
(imported from commit 16c876b5bbd0c6ecd5ef48369a7f71ca74d1183f)
2013-12-19 16:52:53 -05:00
Tim Abbott 2ca5f43f05 Report json format 500 errors from all json format views.
Previously, we only did this via rest_dispatch.

(imported from commit b0edfdccea294378292b64677a64d5b01f936b08)
2013-12-19 16:48:51 -05:00
Jessica McKellar 0d7ea7353f Don't show broken images in digest and missed PMs emails.
(imported from commit 92297b4c8f020e30b5c00e93bf32df6e19069dd8)
2013-12-18 14:31:04 -05:00
Jessica McKellar 96ea91056e Support displaying emoji in digest and missed PM emails.
(imported from commit b440da2501ccb6dd95dc79c3fbce1e81b17c38f4)
2013-12-18 14:31:04 -05:00
Jessica McKellar 2f59f69a1f digest: tweak sender information.
(imported from commit fafc54d4b9c9cda447c42f2bd24845aee62624c2)
2013-12-18 14:31:03 -05:00
Steve Howell be1857ceb3 Add blank line to check_list (cosmetic).
(imported from commit 20a9eed98846e976c7fa2cfdb43e25e1c43c850d)
2013-12-18 12:47:32 -05:00
Steve Howell 6f573feaeb Add "length" option to check_list validator.
(imported from commit 4f8e203f964d1c936fe548b2f77a2e4aae745ae9)
2013-12-18 12:45:29 -05:00
Tim Abbott 3b7bf691e7 Add tool to query our usage stats as of a given date.
This contains the various fixes that needed to be made in order to get
accurate statistics.

Most notably, the active_users_between function in the previous
version of zerver/lib/statistics.py was broken for end dates in the
past, because it used the UserActivity table to get its data -- so in
fact it really was querying "users last active between".

This commit isn't super clean, but I figure we're probably better off
having our latest code for historical usage data in git so it doesn't
bitrot and anyone can improve on it.

(imported from commit 24ff2f24a22e5bdc004ea8043d8da12deb97ff2f)
2013-12-17 15:34:44 -05:00
Jessica McKellar 1d11e2cce0 digest: don't show new users for MIT.
(imported from commit b7453cb30e6fcf5b36a5e3e53f67faeb17e74048)
2013-12-17 10:47:16 -05:00
Tim Abbott 2276c6e524 Add support for loading a narrows-only embedded Zulip window.
Features:
* Only shows messages in the narrow
* New messages in the narrow will arrive as they are sent
* Works even for streams you're not subscribed to
* Automatically subscribes you to a stream on send
* Doesn't update your pointer
* All searches etc. automatically have the narrow added

(imported from commit 2e12b76849f6ca0f53dda5985dad477a04f7bbac)
2013-12-16 17:46:02 -05:00
Tim Abbott 7c829c4024 Extract adding clients to the various dicts to a function.
(imported from commit 667e5736710c23becdf4ebe5688a31421ac632f5)
2013-12-16 17:46:02 -05:00
Tim Abbott af347acf99 Fix narrow stream event queues with all event types.
(imported from commit 69c2f6f1be3dd864b5da85104b83235f0dd4df50)
2013-12-16 17:46:02 -05:00
Steve Howell 4184b9d56f Add validators.py (with tests).
This sets up a scheme to validate complex data structures and
give specific error messages for improperly typed parameters.

(imported from commit 33b2f070d993da4ee929119dd41503bd0128c8eb)
2013-12-13 18:19:07 -05:00
Tim Abbott 7800a34d84 Fix json_unhandled_exception handling of JsonableError.
(imported from commit 16ed6c213cbda3b60a16a01af32c56d6802b7acc)
2013-12-13 18:13:43 -05:00
Waseem Daher 14a5c6bf66 twitter: Detect link styles more robustly.
* Deal with shorter tweet IDs
  (some old tweets don't have a full 18-character ID)
* Allow trailing slash
* Deal with old-style #! syntax
* Deal with links that link to a photo

(imported from commit 008a98c806f3b8dddd9e2f18a8f002af6932766f)
2013-12-13 18:13:23 -05:00
Steve Howell 44b738ab75 Have rest_dispatch return JSON when exceptions are thrown.
(imported from commit 587a8f46d406c6358480db9e0ebd5afb69e12abf)
2013-12-13 17:38:20 -05:00
Kevin Mehall ebab4a59c9 Don't proxy https images through Camo on MIT
These images at least load now, but that's because Camo redirects
the browser to the origin server, so the only effect is an extra
round-trip time.

(imported from commit 0d6b9c888a5cdfaa9299272d74a085e872dfa434)
2013-12-13 17:27:01 -05:00
Jessica McKellar 685830b66a Fix broken image links for missed PM and digest emails.
In plaintext e-mails these will be simple links.

In HTML e-mails these become a <link> and <img>, which some web mail
clients may inline.

(imported from commit b1242dfd917008a019981eb2224c1c7f5f84739f)
2013-12-13 16:38:11 -05:00
Jessica McKellar 70b8d09391 digest: Fix erroneous early return giving us only 1 hot conversation.
(imported from commit db1588bf98778ef838edd6de88fb1ba750251eae)
2013-12-13 16:38:10 -05:00
Jessica McKellar e85566cbb2 digest: Always pad out to 4 conversations if possible.
(imported from commit 1b31a472db5a22a017e461ba846524226641ade1)
2013-12-13 16:38:10 -05:00
Jessica McKellar 8651e205be digest: Make it impossible to report new streams for MIT users.
We may never enqueue MIT users, but just in case we do, never report
new streams.

(imported from commit 01885f339d7bbb3c8ee665fd576831206d493b88)
2013-12-13 16:38:10 -05:00
Jessica McKellar 040cdc2f91 Don't mention new invite-only streams in digest e-mails.
(imported from commit 10ca5dd77736a085f0cc771888ed0aa1ba68a37d)
2013-12-13 16:38:09 -05:00
Jessica McKellar 67088b15a4 Sort messages before building a message list for emails.
Otherwise they sometimes appear out of order!

(imported from commit 26b71579b8f0c31ebdba0eef99afdf0e1772b676)
2013-12-13 16:38:09 -05:00
Jessica McKellar 4740bc7e3a digest: Don't show PMs sent by you.
You can't have unread PMs sent by you, so we weren't explicitly
checking this, but when testing locally we often ignore the unread
check. Filter PMs sent by you to reduce confusion when testing
locally.

(imported from commit 0205c4a3ed67790b9d60d4f2b927e4cb9e720bf3)
2013-12-13 16:38:09 -05:00
Jessica McKellar 35c2804c13 Use PM coloring for PMs in missed message emails.
Previously it used a blue header and white background for everything.

(imported from commit 03b8b0f8fb33a581d67a249f0496cc27aab30ef3)
2013-12-13 16:38:09 -05:00
acrefoot 1bd420f12c Show auto-scroll forever setting in /#settings
(imported from commit fa66cbe52836f230e551f25c47bd755b705387d2)
2013-12-13 11:50:11 -05:00
Tim Abbott 5c9def5be4 Add script to parse user agents with historical data set.
(imported from commit e529c0b914ed3d3d06e9581a6239676f68c97b3f)
2013-12-13 11:26:36 -05:00
Tim Abbott 2e62cd5a74 Add tests for the events_register event application system.
(imported from commit 6c3dbe55d43f528b6a980bba3598ac30c0307a69)
2013-12-12 17:36:41 -05:00
Tim Abbott a3d67a7fe7 Fix apply_events for subscription events to actually match.
Since we changed the initial subscription data to include
user_profile_ids rather than emails, we need to preserve that when
adding in events generated during the page load.

(imported from commit 4f4071b8ba30e57c6f64c9e7b54c1cc754e8f010)
2013-12-12 17:36:40 -05:00
Tim Abbott 8af0fae335 Fix missing muted topics in apply_events.
These events would previously not be applied.

(imported from commit c3e5b9de559b6f744567a0c78e920d34e4d5ae9a)
2013-12-12 17:36:40 -05:00
Tim Abbott afdf7897e6 Split out helper functions from do_events_register.
(imported from commit d838a3303d3ec960d076e755fc8690da5151ac1a)
2013-12-12 17:36:40 -05:00
Tim Abbott a3fe72642b Add save/reload of narrows of server restart.
(imported from commit 2ad77845ce7d8fe365c893ab5cad06e3c725b421)
2013-12-12 17:36:39 -05:00
Tim Abbott 71ad82b6aa Add support for event queues with an included narrow.
This will allow us to substantially decrease the server-side work that
we do to support our Mirroring systems (since the personal mirrors can
request only messages that user sent) and also is what we need to
support a single-stream Zulip widget that we embed in webpages.

(imported from commit 055f2e9a523920719815181f8fdb44d3384e4a34)
2013-12-12 17:36:39 -05:00
Tim Abbott 48963c1cdf Replace accepts_event_type with accepts_event.
This is in preparation for allowing queues to only accept some
messages.

(imported from commit 6bca11a9084aeff52931762085c4c771edbce56e)
2013-12-12 17:36:38 -05:00
Tim Abbott b8579c6848 Disconnect handlers if the client closes a connection.
Tested using the following procedure (run-dev.py won't pass through
the client connect closing)

tabbott@monastery:~/zulip$ curl http://localhost:9991/api/v1/register  -u email:key  -d 'event_types=["message"]'
{"msg":"","max_message_id":2259,"last_event_id":-1,"result":"success","queue_id":"1386884005:0"}
tabbott@monastery:~/zulip$ curl -G http://localhost:9993/api/v1/events  -u email:key -d "last_event_id=0" -d "queue_id=1386884005:0"
(then hit ctrl-C)

(imported from commit 3c4f3d5caac97b3de53da994ff9cd9ef67b2b9ea)
2013-12-12 16:52:25 -05:00
Tim Abbott cc0ce6b437 Finish event handlers when disconnecting from an event queue.
This should help prevent timeouts from clients whose requests have
been supplanted.

(imported from commit fdb3a89c4ec02bb23d0fba50ea558d48cb786916)
2013-12-12 16:52:25 -05:00
Tim Abbott 46760e91cd rest_dispatch: Don't allow session auth on /api URLs.
This caused problems with our tests suite where we were using a logged
in browser session and actually acting as a different user.

(imported from commit 73b8cb39d5d669e682fbacf2f7e574c228885c2f)
2013-12-12 10:31:04 -05:00
Tim Abbott 291506a2b2 Add logging for when we disconnected an active handler.
(imported from commit 18c21a93bdf14485e5656bd7a38f2069eb8f2aad)
2013-12-11 17:23:56 -05:00
Tim Abbott fcf6ec16aa Fix pruning of event queues that only had virtual events.
(imported from commit 7b43d2164df6e95a6feb073b8f90c0862245d629)
2013-12-11 17:20:34 -05:00
Tim Abbott b10278c8e0 events: Fix incorrectly empty event queues with virtual events.
I'm not sure this had much impact, but it's definitely a subtle bug.

(imported from commit 0260d4c5c35ce8eac683e84f1d939a6704ce4e2b)
2013-12-11 16:59:48 -05:00
Tim Abbott df730ec163 Fix json_unauthorized.
(imported from commit 7537b3bbc21b947f91ec49060935e9e2f92be0ac)
2013-12-11 16:45:29 -05:00
Kevin Mehall 662edc2558 [manual] Backend support for Android GCM push notifications
This adds a dependency on gcmclient:
http://gcm-client.readthedocs.org/en/latest/gcmclient.html

pip install gcm-client

or

apt-get install python-gcm-client

(imported from commit 9f1fbf1f793e4a27baed85c6f1aa7a7b03106a10)
2013-12-11 15:37:48 -05:00
Kevin Mehall e4589700b6 [schema] Modify device token to support both iOS and Android
This replaces the AppleDeviceToken table with a generic
PushDeviceToken with a `kind` field to make it easier to add functionality
like per-device/per-stream settings that share code between Android and
iOS devices.

The schema must continue to work on prod with the old table name, so we
add the new table in parallel and can drop the old table once this code
hits prod and any necessary data is copied.

(imported from commit 0209a7013f2850ac6311f23c3d6f92c65ffd19e3)
2013-12-11 15:37:47 -05:00
Leo Franchi 2c39e28ff6 Use realm filters from the database in bugdown
(imported from commit 8e12bfb1b1fe9cad694f80a189efa64b0f102e81)
2013-12-11 14:39:10 -05:00
Zev Benjamin 874d4282c0 Disconnect handlers from event queues via a finally block
(imported from commit ac63a962f978b05a5d8d6bf81861b908b5fd54b3)
2013-12-10 17:44:35 -05:00
Zev Benjamin b566c0f201 Catch a wider range of exceptions when adding events to event queues
We have observed additional exceptions being thrown from zulip_finish and we
need to make sure that the handler is disconnected from the queue, or else the
event queue will keep throwing exceptions due to the handler being closed.

(imported from commit 59273aa14495216430b9eb1525b2cce230d8913d)
2013-12-10 17:40:30 -05:00
Zev Benjamin 0d989a6972 socket: Close old connections before registering new ones under the same id
(imported from commit f9b75288cbf7e2d9baa3254d57443eea5c1f3fc3)
2013-12-09 16:17:26 -05:00
acrefoot 33ddba3d30 extra debugging info for handle_push_notification
(imported from commit c4f63827fd37e7e99853380eaa268ab2e5ae23ce)
2013-12-09 11:15:20 -05:00
Zev Benjamin 5e19ad31bf socket: Increase redis response cache expiration time
(imported from commit 1b18530f01499af9a555c71f0281e62bd3164542)
2013-12-08 12:13:15 -05:00
Zev Benjamin 96bb739157 socket: Add some server-side logging for bug catching
(imported from commit 8579d8bbb414b063ae9cf36993975adaa86cb132)
2013-12-06 16:22:24 -05:00
Zev Benjamin 8535669d46 socket: Include the user in socket close log lines
(imported from commit fb5b9e54a07b6bfd85e913b8c97105305f8210eb)
2013-12-06 16:22:24 -05:00
Leo Franchi f55067a3db Actually use a UserMessage object instead of Message
(imported from commit 9613af82f62a9ea85a85f1ce662234f410457997)
2013-12-06 15:53:42 -05:00
Leo Franchi 4f2cb1bc1c Read read flags from UserMessage not Message
(imported from commit 4c0f6ec90992c61ba861f5f347655673e769bebc)
2013-12-06 15:08:02 -05:00
Zev Benjamin 657a750550 socket: Fix spelling
We don't ever actually check for this value so nothing else has to
change.

(imported from commit e6ccb153df7bd95c9d195aa83395592b309a2778)
2013-12-05 15:53:50 -05:00
Zev Benjamin 002df64811 Remove extraneous semicolons
(imported from commit 2077705c52e7255ab7ea1b4df71d2ae99f6018dc)
2013-12-05 15:53:50 -05:00
Tim Abbott fed3902fc5 Clean up debugging logs from tracking down slow receive.
(imported from commit e4b1f763522406255ce8be3c8d51b5d855a770cd)
2013-12-05 14:36:29 -05:00
Tim Abbott 55047280c9 Include a newline at the end of our JSON responses.
Otherwise, it's super annoying to work with these using curl.

(imported from commit 0047f07786dc07d7ac5cb671492f8ca40986d8cf)
2013-12-04 13:47:48 -05:00
Leo Franchi 1b53a77f1b Expect an ios client instead of iphone
(imported from commit d5d3cd129d7005fd9294096d3899df86155cd366)
2013-12-03 14:27:51 -05:00
Tim Abbott abded5f886 Upgrade event flag collapsing to support all flag changes.
(imported from commit a2ca74b779d67148aee41e92e4a4cb716620e566)
2013-12-03 12:21:04 -05:00
Tim Abbott d2486670a8 event: Add deduplication of server restart events.
(imported from commit e464aa8b68958fb9f9fe8c199b9e73df510d22bc)
2013-12-03 12:20:36 -05:00
Tim Abbott 54b06d046b events: Add support for not storing redundant events.
This should dramatically improve the speed of the dump/load part of
our restart process, especially with large long-lived event queues.

(imported from commit ae4ae20ba2ca4433e25a5e7beeb4fa4882c53972)
2013-12-03 12:19:42 -05:00
Tim Abbott ad9659ea54 events: Fix zulip_finish to not make assumptions about how queues work.
Previously, we had an issue with the ACKing protocol, where if a
virtualizable event (like a "read" flag) was dispatched to a queue
client immediately, we would not properly ACK the change because it
had been made a virtual event.

(imported from commit ea09812f8a5ba1d5aad65f536022e3dbc77b0f9e)
2013-12-03 12:19:42 -05:00
Jessica McKellar e15cc972ec digest: refactor sending the emails into a function for easier testing.
(imported from commit 2f3c8fb8e812c04ab838b1034ae0811309186625)
2013-12-02 18:58:03 -05:00
Jessica McKellar 207cfc449d Add unsubscribe links to digest e-mails.
(imported from commit 4e2a324decf4bc694752cc24b9085361338a08a5)
2013-12-02 18:58:03 -05:00
acrefoot da1466cfa9 handle case of message getting read while processing missed_message
(imported from commit cd53efe3645f5dc8943becc2b8dbd3d607bb8eba)
2013-12-02 16:00:56 -05:00
Tim Abbott 559f0896a1 Revert "events: Add support for not storing redundant events."
This reverts commit cc493fa50b4c339257e060b3f0c0956c682e449d.

(imported from commit 9e8edfc11d415d61581da482d66bc9fa30cbefdf)
2013-12-02 11:03:04 -05:00
Tim Abbott 5b601d8492 Revert "event: Add deduplication of server restart events."
This reverts commit 05467929ba376a0ce173119c5d90d83671fc544b.

(imported from commit 4f24a0f1fe72e71fa368e1d818d9406a76c131be)
2013-12-02 11:03:04 -05:00
Tim Abbott 2c2f404cb6 Revert "Upgrade event flag collapsing to support all flag changes."
This reverts commit dadf3852f2ca7aaa85fb97b3d265c990b4e5a38a.

(imported from commit de5a95572aa955b4a9419b80a99d2446c51730e7)
2013-12-02 11:03:04 -05:00
Tim Abbott 10bce7ffe8 Revert "Fix loading event queues that don't have a virtual_events attribute"
This reverts commit 99c5ae332d253313ed728883a388c61191ed3021.

(imported from commit d7b601a2897bac6e14b675e689ee63ccc6e8d67d)
2013-12-02 11:03:03 -05:00
Tim Abbott 5bc826067e Revert "Add comment in ClientDescriptor.to_dict and EventQueue.to_dict to avoid future bugs"
This reverts commit 45560539d09e9e1922130349be729005b786e632.

(imported from commit aa266941f03cdbe12f4315e65ec6c472cc207204)
2013-12-02 11:03:03 -05:00
acrefoot 717e06d94c remove log message on push notification
(imported from commit 45fc89e67a79aac3484c911d88aff3e470b4eff6)
2013-11-27 18:00:33 -05:00
acrefoot 8800dcdce4 more helpful error message when missing APNS certs
(imported from commit e21dc627844b9e28f93a924097176481d4135947)
2013-11-27 18:00:33 -05:00
acrefoot 8ee7536012 Send push notifications more aggresively
Unbundle the push notifications from the missed message queue processors
and handlers. This makes notifications more immediate, and sets things up
for better badge count handling, and possibly per-stream filtering.

(imported from commit 11840301751b0bbcb3a99848ff9868d9023b665b)
2013-11-27 18:00:31 -05:00
Zev Benjamin 9d0f801ad6 Exempt a particular client name from heartbeats for testing
(imported from commit 3914130384a7a9854b006405ffd2fe379bbd090d)
2013-11-27 10:39:31 -05:00