Commit Graph

514 Commits

Author SHA1 Message Date
Tim Abbott da90d63046 Split out zerver/lib/notifications.py from actions.py.
(imported from commit 784b82834ee4fcb4431e77f8fb1c526f8eec82ad)
2014-01-24 17:33:56 -05:00
Steve Howell 1d95812770 Add is_admin to page_params.
Because full_name and is_admin changes go through many similar,
generic codepaths, it is almost more work at this point to keep
is_admin out of page_params as it is to just put it in.  So
I put it in.  This should pave the path for showing admins in
the GUI.

This commit actually starting by my adding a test
that calling apply_events() with the notification you get
when calling do_change_is_admin() updates
state['realm_users'] to be similar to what you would get
out of fetch_initial_state_data().  We didn't have test coverage
there before.  Making that test pass forced my hand to
either add is_admin to page_params or to special-case
apply_events() not to update page_params with is_admin.  I
chose the former approach.

(imported from commit 1e49d59c66540014284529c29d5007224be6a0c6)
2014-01-24 16:34:12 -05:00
Steve Howell 4ad8ad2b5a Extract get_realm_user_dicts() from fetch_initial_state_data().
(imported from commit 2f0f52d90eba22a4b72bd6173e60ab463482b7d6)
2014-01-24 16:34:12 -05:00
Jason Michalski 1f98c4f4ce Add support for updating the stream description
The stream description can now be updated, but the UI is not yet updated
on success.

(imported from commit f45e0a9d5138d828ae98d6d49645ab3ddc966704)
2014-01-24 14:47:44 -05:00
Jason Michalski 4104f00229 [schema] Add description to streams and display it
A description was added to the streams and it is now displayed on the
subscriptions page. It can not be set in the UI yet.

(imported from commit 81d08b65eee42dba87cd99dd5bd30106c4eb6c6a)
2014-01-24 14:47:44 -05:00
Luke Faraone 1c67d05e5f Handle duplicate GCM registrations by dropping the old ones.
While we're at it, lets comment up the function so I know what this is
doing next time :)

(imported from commit e745be75fcd6dbce9997e1d73464619fc8b73996)
2014-01-23 19:21:28 -05:00
Jessica McKellar ee7a1f0953 tutorial: add helpful in-app links and be consistent with realm welcome message.
(imported from commit 2057dbe2606d32721192021ed2046ed9f412d8a7)
2014-01-23 17:06:29 -05:00
Jessica McKellar 086eab8bb5 bugdown: support modal in-app links in messages.
(imported from commit 0e3c0ff61cf9a84bae81e37b0a6d79831dea9224)
2014-01-23 17:06:29 -05:00
Leo Franchi 56d9446c41 [socket] Use socket req ids that contain queue ids
(imported from commit 781eafe60bf117d79ce3a30deea02ec1e875060a)
2014-01-23 16:28:58 -05:00
Leo Franchi 7743aa191c Pass through local_id and sender_queue_id in non-rabbitmq case
(imported from commit 8989225a0943b39e45bd5debe405acb9768acc8c)
2014-01-23 16:28:58 -05:00
Steve Howell 2283aa52cb Use "stream" as our type for updates to Stream.name.
(imported from commit 2eedbff5fac77b9e654ba88900167048573e4056)
2014-01-23 13:27:11 -05:00
Steve Howell bc6da3717f Fix obscure bug with page_params.is_bot.
If a name change event arrived during the call to
fetch_initial_state_data(), we would call apply_events() to
update the data structure that eventually becomes page_params.
Our update code, instead of surgically updating the fields, was
just overwriting the record, which led to is_bot being taken
out of the record when only full_name was in the event.

Apart from fixing the "update" case to do the right thing,
this commit also does a bit of cleanup on the code handling
"realm_user" events to make it more generic in how it does
add/remove/update.  If we could standardize our events a bit
more, this could eventually lead to DRYing up some of the
apply_events() code.

(imported from commit 772e2fcd6a5605ccb6e8d1bc499b5f336934cf3c)
2014-01-23 13:01:04 -05:00
Steve Howell 3b12e63d8d Send notifications for is_admin changes.
(The client doesn't do anything with them yet.)

(imported from commit df2f406644fe271af53e32a6b3521752b6a1ac41)
2014-01-22 14:43:28 -05:00
Leo Franchi c0fdbf81cf Use the feedback connection for getting APNS feedback
(imported from commit d371188e0437606b3ff435ee9c005cee2849f6ae)
2014-01-22 13:50:15 -05:00
Leo Franchi 4dac0e7d37 Print out APNS tokens in b64 format for ease of use
(imported from commit d38eb1629a599681c42f1f8d817c90229b547516)
2014-01-22 11:59:02 -05:00
Leo Franchi 6903d87809 Retry on APNS failure / retry request
(imported from commit b9d1c414731de7a4ad5d3bc93c3dce1286cbdb38)
2014-01-22 11:59:02 -05:00
Leo Franchi c0d3b2fd89 Remove APNS tokens from our database if we get an invalid token notification
(imported from commit 5ec3c053684f9574a31103e7821309de883baa82)
2014-01-22 11:59:02 -05:00
Leo Franchi 226d894257 Prefix apple-specific APNS logs with APNS:
(imported from commit 2fa866b3a626c3e42f340bc68e8bc27aef6fd521)
2014-01-22 11:59:02 -05:00
Jason Michalski c30a411c10 Add custom markdown tag to render a stream subscribe button
When new streams are created we now send a message with a custom
markdown tag that renders a subscribe button.

(imported from commit 9dfba280b3b4ff4f32f6431ef9227867c8bf4b40)
2014-01-22 11:28:31 -05:00
Leo Franchi 669951b631 Send push notifications regardless of idle status for robinhood.io, zulip.com
(imported from commit 49d2c954f6537d28ef3f8ccad669c56430afdbde)
2014-01-21 14:48:57 -05:00
Jason Michalski 4c90392261 [schema] Add an per-user option to disable notification on new streams
Added a default_desktop_notifications boolean to userprofile with a UI
in Zulip Labs. This flag is used to default the notification flag on new
subscriptions.

(imported from commit a25223cc5ecf09980cf877991e25034bb3fd4046)
2014-01-16 20:13:34 -05:00
Jessica McKellar a1d128b6cb Flush memcached on a stream privacy change.
Otherwise uninvited users can join a stream recently turned
invite-only.

(imported from commit ab43bf3b856a0e3b71e6e06cee1519214b7763cd)
2014-01-15 17:13:56 -05:00
Waseem Daher 9eed2c50b9 email-mirror: Support '.' as an alternate for '+' (and bugfix).
Google Groups won't let you add an email address that has a '+' in it
as a member of a group, so we allow '.' as well.

This commit also fixes a current issue with email mirroring, where it
doesn't work if you had a + in the stream name.

This fixes Trac #2102.

(imported from commit 9a7a5f5d16087f6f74fb5308e170a6f04387599e)
2014-01-15 16:07:19 -05:00
Jason Michalski 60f878b45e Limit cross-realm private messages to zulip.com users
Cross-realm private messages are only used to respond to support
requests. So now cross-realm private messages are only allowed if
exactly two realms are in the private message and one of them is
zulip.com.

(imported from commit f01a2824e214682acb22a6995714a9d1b0d0c66f)
2014-01-15 16:07:19 -05:00
Luke Faraone 0b5036e0fb Let message recipient in process_message be passed as an argument.
(imported from commit cfcfb9edf748a35d279235ee11a05b1a00c934b5)
2014-01-15 16:07:18 -05:00
Zev Benjamin db23674749 Do query time tracking at the psycopg2 level instead of the Django level
This allows us to track the query time of SQLAlchemy and raw queries.

(imported from commit 818a4ee41786ffc57b80d7ed1cfba075f29b6ee5)
2014-01-14 11:47:12 -05:00
Tim Abbott e38ac2f9ac Index the presence data sent from Django to Tornado by ID.
(imported from commit 8ffb70562b49ba93c5b0927b34dc57a743d49564)
2014-01-13 16:54:06 -05:00
Tim Abbott e890f06c1c Move missedmessage hook checks on UserProfile settings to workers.
This does result in a few more rabbitmq events to be processed (though
a negligible number compared to what we already do), but it saves a
database query from inside Tornado whenever we occasionally have a
cache miss looking up the UserProfile, which is far more important.

(imported from commit a553a00a3004ba27bfb54ffbc3e9c9b170ebae4d)
2014-01-13 16:54:05 -05:00
Zev Benjamin 22ae828685 Support Tornado 3
The only thing that needed changing was our instrumented IOLoop.

(imported from commit 17833500f03949f45a546050a1f99675e9f6b1b5)
2014-01-10 21:39:06 -05:00
Steve Howell e832e937c5 Make emojis be 20px tall in missed message emails.
(imported from commit 687b099f19d69f41876571c08a0627f061067f86)
2014-01-10 21:39:05 -05:00
Steve Howell 1a679795e1 Flatten the code for build_message_payload.
(imported from commit b58c91468b36d33c163d4994b501eac796d10e8f)
2014-01-10 21:39:05 -05:00
Waseem Daher fe748f7633 Make Twitter preview look like other inline image previews.
(imported from commit ca4ff9146679a5348bf53d06e992972e9a4b628a)
2014-01-10 21:39:04 -05:00
Steve Howell 3e48b4fd95 Update mentioned flags after message edits.
(imported from commit 67886db571193a0f53cf72aa00af9e5e75729dc0)
2014-01-10 21:39:03 -05:00
Steve Howell 8d79606136 Update has_alert_word flag after message edits.
(imported from commit cad1b5646742506c1e84a42366fd58aaaf6dc17d)
2014-01-10 21:39:02 -05:00
Steve Howell 0326fa92b4 Send UserMessage flags for update_message events.
When we edit a message, send out UserMessage flags to the recipients.
This sets the stage for making sure that changes related to
user-specific alert words or mentions get sent out to users.

(imported from commit bce1de19acef44b5e106352f261203352ece02b9)
2014-01-10 21:39:02 -05:00
Jason Michalski 9e822403bb Linkify twitter media links
Image and video links in the twitter API are media and need to be
handed on separately. We also include a preview image if the media link
is a to a picture.

(imported from commit 2bd00d267e51b29ad0ba681195b2bfea9b991d8c)
2014-01-10 21:39:02 -05:00
Jason Michalski 1352dccea5 Add support to linkify links and mentions on twitter
This converts links in tweets to a tags. We also convert the displayed
text to the target of the twitter short URL. Mentions are linked to the
users twitter page.

(imported from commit 192d5546a7eea82759f9ae30d82c102aed15ff71)
2014-01-10 21:39:01 -05:00
Tim Abbott 1470cb8bf4 do_send_messages: Only pass active recipient users to Tornado.
Previously, we were processing in Tornado mirroring dummy users (and
deactived users) as recipients -- resulting in bugs where the missed
message hooks would fire for these nonexistent users.

Our Tornado real-time delivery system only needs the list of active
users both for delivery and for presence information updates.

(imported from commit b81143f106a4d0eefa4b838e7c074b2963259746)
2014-01-10 21:39:00 -05:00
Tim Abbott 4bf3ace444 [manual] Allow signups for emails held by non-MIT mirror dummy accounts.
Before this is deployed to prod, we need to manually frob our database
to set the is_mirror_dummy=True bit for all existing mirror users.

(imported from commit 39f1938cef091cf1d7d97307f76b137fe1d92b6c)
2014-01-10 21:38:59 -05:00
Jessica McKellar d3f6c4ef36 Log the actual sending of digest emails in addition to queuing possible ones.
(imported from commit e43ac59e71620fe715db462347db228d5a950aed)
2014-01-10 21:38:59 -05:00
Tim Abbott 47500d8352 [schema] Add a bit for whether inactive users are mirror dummies.
(imported from commit bb21bb2c62ac09742484d7a4ca8907e7d864b982)
2014-01-09 11:08:35 -05:00
Jessica McKellar 8e3911269e Strip whitespace from emails before looking up DB entries in a few places.
(imported from commit eafdce98450d16c0ca51c4cb17a139811a6124bc)
2014-01-07 20:24:21 -05:00
Jessica McKellar 511e1a12e3 Add a management command to deactivate old realms.
(imported from commit f6b1d0cb5b10f91683afcffea88b23b1a75a9ad2)
2014-01-07 20:24:20 -05:00
Leo Franchi c7836626ea Accept a pass-through local_id option in send_message
(imported from commit 3f7b8e862a92d8a11b68da9ff23f711b6b25b5e2)
2014-01-07 17:33:34 -05:00
Leo Franchi 76892c52aa Send realm filters to clients
(imported from commit a3ba31701baa4d515ea3bf36c39d7255f1dd20d6)
2014-01-07 17:33:33 -05:00
Luke Faraone df4b2d9f85 Break out non-Twisted email mirror code into separate library
(imported from commit 45c3e70ba81cdb44f1e9db4f162e5f2d2fe5ead6)
2014-01-06 13:36:53 -05:00
Steve Howell 654ce3e0c7 Add make_stream_private endpoint.
(imported from commit b504c24fd651ecf8814e64725a9f303dc085b14e)
2014-01-06 11:59:47 -05:00
Steve Howell e0c62c5eda Add make_stream_public endpoint.
(imported from commit 03d4cff5587d0aa149997f2f6ae28ec4ede95d7a)
2014-01-06 11:59:47 -05:00
Zev Benjamin fb602da502 Don't log exceptions that we know aren't bugs when adding events to event queues
(imported from commit 55074e68603119bd11cc5218453a4e1eab710feb)
2014-01-03 14:40:55 -05:00
Jessica McKellar 30367d0c0f digest: don't include automated messages in hot conversation accounting.
(imported from commit ab8db46c644b3cf66a4bc654d204d870b8fae91f)
2013-12-31 16:45:21 -05:00
Jessica McKellar 8091634814 Move sent_by_human check to a method on the Message class.
(imported from commit 2ef49051932f9633ad11130952591c5c5b147f12)
2013-12-31 16:42:38 -05:00
Jessica McKellar 61103d8349 digest: On an app server, only send digest emails for domains it manages.
(imported from commit 9aea636a19eb6fdefacdc7dc393bf075a05862c5)
2013-12-20 12:50:23 -05:00
Tim Abbott 2264a07c03 Update server-side app checks to handle the new app names.
(imported from commit da7b2c06b413add1881b2d80a89794d0db9c810f)
2013-12-19 17:21:26 -05:00
Tim Abbott a70072f5cf socket: Don't throw traceback when sessions are expired.
(imported from commit 16c876b5bbd0c6ecd5ef48369a7f71ca74d1183f)
2013-12-19 16:52:53 -05:00
Tim Abbott 2ca5f43f05 Report json format 500 errors from all json format views.
Previously, we only did this via rest_dispatch.

(imported from commit b0edfdccea294378292b64677a64d5b01f936b08)
2013-12-19 16:48:51 -05:00
Jessica McKellar 0d7ea7353f Don't show broken images in digest and missed PMs emails.
(imported from commit 92297b4c8f020e30b5c00e93bf32df6e19069dd8)
2013-12-18 14:31:04 -05:00
Jessica McKellar 96ea91056e Support displaying emoji in digest and missed PM emails.
(imported from commit b440da2501ccb6dd95dc79c3fbce1e81b17c38f4)
2013-12-18 14:31:04 -05:00
Jessica McKellar 2f59f69a1f digest: tweak sender information.
(imported from commit fafc54d4b9c9cda447c42f2bd24845aee62624c2)
2013-12-18 14:31:03 -05:00
Steve Howell be1857ceb3 Add blank line to check_list (cosmetic).
(imported from commit 20a9eed98846e976c7fa2cfdb43e25e1c43c850d)
2013-12-18 12:47:32 -05:00
Steve Howell 6f573feaeb Add "length" option to check_list validator.
(imported from commit 4f8e203f964d1c936fe548b2f77a2e4aae745ae9)
2013-12-18 12:45:29 -05:00
Tim Abbott 3b7bf691e7 Add tool to query our usage stats as of a given date.
This contains the various fixes that needed to be made in order to get
accurate statistics.

Most notably, the active_users_between function in the previous
version of zerver/lib/statistics.py was broken for end dates in the
past, because it used the UserActivity table to get its data -- so in
fact it really was querying "users last active between".

This commit isn't super clean, but I figure we're probably better off
having our latest code for historical usage data in git so it doesn't
bitrot and anyone can improve on it.

(imported from commit 24ff2f24a22e5bdc004ea8043d8da12deb97ff2f)
2013-12-17 15:34:44 -05:00
Jessica McKellar 1d11e2cce0 digest: don't show new users for MIT.
(imported from commit b7453cb30e6fcf5b36a5e3e53f67faeb17e74048)
2013-12-17 10:47:16 -05:00
Tim Abbott 2276c6e524 Add support for loading a narrows-only embedded Zulip window.
Features:
* Only shows messages in the narrow
* New messages in the narrow will arrive as they are sent
* Works even for streams you're not subscribed to
* Automatically subscribes you to a stream on send
* Doesn't update your pointer
* All searches etc. automatically have the narrow added

(imported from commit 2e12b76849f6ca0f53dda5985dad477a04f7bbac)
2013-12-16 17:46:02 -05:00
Tim Abbott 7c829c4024 Extract adding clients to the various dicts to a function.
(imported from commit 667e5736710c23becdf4ebe5688a31421ac632f5)
2013-12-16 17:46:02 -05:00
Tim Abbott af347acf99 Fix narrow stream event queues with all event types.
(imported from commit 69c2f6f1be3dd864b5da85104b83235f0dd4df50)
2013-12-16 17:46:02 -05:00
Steve Howell 4184b9d56f Add validators.py (with tests).
This sets up a scheme to validate complex data structures and
give specific error messages for improperly typed parameters.

(imported from commit 33b2f070d993da4ee929119dd41503bd0128c8eb)
2013-12-13 18:19:07 -05:00
Tim Abbott 7800a34d84 Fix json_unhandled_exception handling of JsonableError.
(imported from commit 16ed6c213cbda3b60a16a01af32c56d6802b7acc)
2013-12-13 18:13:43 -05:00
Waseem Daher 14a5c6bf66 twitter: Detect link styles more robustly.
* Deal with shorter tweet IDs
  (some old tweets don't have a full 18-character ID)
* Allow trailing slash
* Deal with old-style #! syntax
* Deal with links that link to a photo

(imported from commit 008a98c806f3b8dddd9e2f18a8f002af6932766f)
2013-12-13 18:13:23 -05:00
Steve Howell 44b738ab75 Have rest_dispatch return JSON when exceptions are thrown.
(imported from commit 587a8f46d406c6358480db9e0ebd5afb69e12abf)
2013-12-13 17:38:20 -05:00
Kevin Mehall ebab4a59c9 Don't proxy https images through Camo on MIT
These images at least load now, but that's because Camo redirects
the browser to the origin server, so the only effect is an extra
round-trip time.

(imported from commit 0d6b9c888a5cdfaa9299272d74a085e872dfa434)
2013-12-13 17:27:01 -05:00
Jessica McKellar 685830b66a Fix broken image links for missed PM and digest emails.
In plaintext e-mails these will be simple links.

In HTML e-mails these become a <link> and <img>, which some web mail
clients may inline.

(imported from commit b1242dfd917008a019981eb2224c1c7f5f84739f)
2013-12-13 16:38:11 -05:00
Jessica McKellar 70b8d09391 digest: Fix erroneous early return giving us only 1 hot conversation.
(imported from commit db1588bf98778ef838edd6de88fb1ba750251eae)
2013-12-13 16:38:10 -05:00
Jessica McKellar e85566cbb2 digest: Always pad out to 4 conversations if possible.
(imported from commit 1b31a472db5a22a017e461ba846524226641ade1)
2013-12-13 16:38:10 -05:00
Jessica McKellar 8651e205be digest: Make it impossible to report new streams for MIT users.
We may never enqueue MIT users, but just in case we do, never report
new streams.

(imported from commit 01885f339d7bbb3c8ee665fd576831206d493b88)
2013-12-13 16:38:10 -05:00
Jessica McKellar 040cdc2f91 Don't mention new invite-only streams in digest e-mails.
(imported from commit 10ca5dd77736a085f0cc771888ed0aa1ba68a37d)
2013-12-13 16:38:09 -05:00
Jessica McKellar 67088b15a4 Sort messages before building a message list for emails.
Otherwise they sometimes appear out of order!

(imported from commit 26b71579b8f0c31ebdba0eef99afdf0e1772b676)
2013-12-13 16:38:09 -05:00
Jessica McKellar 4740bc7e3a digest: Don't show PMs sent by you.
You can't have unread PMs sent by you, so we weren't explicitly
checking this, but when testing locally we often ignore the unread
check. Filter PMs sent by you to reduce confusion when testing
locally.

(imported from commit 0205c4a3ed67790b9d60d4f2b927e4cb9e720bf3)
2013-12-13 16:38:09 -05:00
Jessica McKellar 35c2804c13 Use PM coloring for PMs in missed message emails.
Previously it used a blue header and white background for everything.

(imported from commit 03b8b0f8fb33a581d67a249f0496cc27aab30ef3)
2013-12-13 16:38:09 -05:00
acrefoot 1bd420f12c Show auto-scroll forever setting in /#settings
(imported from commit fa66cbe52836f230e551f25c47bd755b705387d2)
2013-12-13 11:50:11 -05:00
Tim Abbott 5c9def5be4 Add script to parse user agents with historical data set.
(imported from commit e529c0b914ed3d3d06e9581a6239676f68c97b3f)
2013-12-13 11:26:36 -05:00
Tim Abbott 2e62cd5a74 Add tests for the events_register event application system.
(imported from commit 6c3dbe55d43f528b6a980bba3598ac30c0307a69)
2013-12-12 17:36:41 -05:00
Tim Abbott a3d67a7fe7 Fix apply_events for subscription events to actually match.
Since we changed the initial subscription data to include
user_profile_ids rather than emails, we need to preserve that when
adding in events generated during the page load.

(imported from commit 4f4071b8ba30e57c6f64c9e7b54c1cc754e8f010)
2013-12-12 17:36:40 -05:00
Tim Abbott 8af0fae335 Fix missing muted topics in apply_events.
These events would previously not be applied.

(imported from commit c3e5b9de559b6f744567a0c78e920d34e4d5ae9a)
2013-12-12 17:36:40 -05:00
Tim Abbott afdf7897e6 Split out helper functions from do_events_register.
(imported from commit d838a3303d3ec960d076e755fc8690da5151ac1a)
2013-12-12 17:36:40 -05:00
Tim Abbott a3fe72642b Add save/reload of narrows of server restart.
(imported from commit 2ad77845ce7d8fe365c893ab5cad06e3c725b421)
2013-12-12 17:36:39 -05:00
Tim Abbott 71ad82b6aa Add support for event queues with an included narrow.
This will allow us to substantially decrease the server-side work that
we do to support our Mirroring systems (since the personal mirrors can
request only messages that user sent) and also is what we need to
support a single-stream Zulip widget that we embed in webpages.

(imported from commit 055f2e9a523920719815181f8fdb44d3384e4a34)
2013-12-12 17:36:39 -05:00
Tim Abbott 48963c1cdf Replace accepts_event_type with accepts_event.
This is in preparation for allowing queues to only accept some
messages.

(imported from commit 6bca11a9084aeff52931762085c4c771edbce56e)
2013-12-12 17:36:38 -05:00
Tim Abbott b8579c6848 Disconnect handlers if the client closes a connection.
Tested using the following procedure (run-dev.py won't pass through
the client connect closing)

tabbott@monastery:~/zulip$ curl http://localhost:9991/api/v1/register  -u email:key  -d 'event_types=["message"]'
{"msg":"","max_message_id":2259,"last_event_id":-1,"result":"success","queue_id":"1386884005:0"}
tabbott@monastery:~/zulip$ curl -G http://localhost:9993/api/v1/events  -u email:key -d "last_event_id=0" -d "queue_id=1386884005:0"
(then hit ctrl-C)

(imported from commit 3c4f3d5caac97b3de53da994ff9cd9ef67b2b9ea)
2013-12-12 16:52:25 -05:00
Tim Abbott cc0ce6b437 Finish event handlers when disconnecting from an event queue.
This should help prevent timeouts from clients whose requests have
been supplanted.

(imported from commit fdb3a89c4ec02bb23d0fba50ea558d48cb786916)
2013-12-12 16:52:25 -05:00
Tim Abbott 46760e91cd rest_dispatch: Don't allow session auth on /api URLs.
This caused problems with our tests suite where we were using a logged
in browser session and actually acting as a different user.

(imported from commit 73b8cb39d5d669e682fbacf2f7e574c228885c2f)
2013-12-12 10:31:04 -05:00
Tim Abbott 291506a2b2 Add logging for when we disconnected an active handler.
(imported from commit 18c21a93bdf14485e5656bd7a38f2069eb8f2aad)
2013-12-11 17:23:56 -05:00
Tim Abbott fcf6ec16aa Fix pruning of event queues that only had virtual events.
(imported from commit 7b43d2164df6e95a6feb073b8f90c0862245d629)
2013-12-11 17:20:34 -05:00
Tim Abbott b10278c8e0 events: Fix incorrectly empty event queues with virtual events.
I'm not sure this had much impact, but it's definitely a subtle bug.

(imported from commit 0260d4c5c35ce8eac683e84f1d939a6704ce4e2b)
2013-12-11 16:59:48 -05:00
Tim Abbott df730ec163 Fix json_unauthorized.
(imported from commit 7537b3bbc21b947f91ec49060935e9e2f92be0ac)
2013-12-11 16:45:29 -05:00
Kevin Mehall 662edc2558 [manual] Backend support for Android GCM push notifications
This adds a dependency on gcmclient:
http://gcm-client.readthedocs.org/en/latest/gcmclient.html

pip install gcm-client

or

apt-get install python-gcm-client

(imported from commit 9f1fbf1f793e4a27baed85c6f1aa7a7b03106a10)
2013-12-11 15:37:48 -05:00
Kevin Mehall e4589700b6 [schema] Modify device token to support both iOS and Android
This replaces the AppleDeviceToken table with a generic
PushDeviceToken with a `kind` field to make it easier to add functionality
like per-device/per-stream settings that share code between Android and
iOS devices.

The schema must continue to work on prod with the old table name, so we
add the new table in parallel and can drop the old table once this code
hits prod and any necessary data is copied.

(imported from commit 0209a7013f2850ac6311f23c3d6f92c65ffd19e3)
2013-12-11 15:37:47 -05:00
Leo Franchi 2c39e28ff6 Use realm filters from the database in bugdown
(imported from commit 8e12bfb1b1fe9cad694f80a189efa64b0f102e81)
2013-12-11 14:39:10 -05:00
Zev Benjamin 874d4282c0 Disconnect handlers from event queues via a finally block
(imported from commit ac63a962f978b05a5d8d6bf81861b908b5fd54b3)
2013-12-10 17:44:35 -05:00
Zev Benjamin b566c0f201 Catch a wider range of exceptions when adding events to event queues
We have observed additional exceptions being thrown from zulip_finish and we
need to make sure that the handler is disconnected from the queue, or else the
event queue will keep throwing exceptions due to the handler being closed.

(imported from commit 59273aa14495216430b9eb1525b2cce230d8913d)
2013-12-10 17:40:30 -05:00
Zev Benjamin 0d989a6972 socket: Close old connections before registering new ones under the same id
(imported from commit f9b75288cbf7e2d9baa3254d57443eea5c1f3fc3)
2013-12-09 16:17:26 -05:00
acrefoot 33ddba3d30 extra debugging info for handle_push_notification
(imported from commit c4f63827fd37e7e99853380eaa268ab2e5ae23ce)
2013-12-09 11:15:20 -05:00
Zev Benjamin 5e19ad31bf socket: Increase redis response cache expiration time
(imported from commit 1b18530f01499af9a555c71f0281e62bd3164542)
2013-12-08 12:13:15 -05:00
Zev Benjamin 96bb739157 socket: Add some server-side logging for bug catching
(imported from commit 8579d8bbb414b063ae9cf36993975adaa86cb132)
2013-12-06 16:22:24 -05:00
Zev Benjamin 8535669d46 socket: Include the user in socket close log lines
(imported from commit fb5b9e54a07b6bfd85e913b8c97105305f8210eb)
2013-12-06 16:22:24 -05:00
Leo Franchi f55067a3db Actually use a UserMessage object instead of Message
(imported from commit 9613af82f62a9ea85a85f1ce662234f410457997)
2013-12-06 15:53:42 -05:00
Leo Franchi 4f2cb1bc1c Read read flags from UserMessage not Message
(imported from commit 4c0f6ec90992c61ba861f5f347655673e769bebc)
2013-12-06 15:08:02 -05:00
Zev Benjamin 657a750550 socket: Fix spelling
We don't ever actually check for this value so nothing else has to
change.

(imported from commit e6ccb153df7bd95c9d195aa83395592b309a2778)
2013-12-05 15:53:50 -05:00
Zev Benjamin 002df64811 Remove extraneous semicolons
(imported from commit 2077705c52e7255ab7ea1b4df71d2ae99f6018dc)
2013-12-05 15:53:50 -05:00
Tim Abbott fed3902fc5 Clean up debugging logs from tracking down slow receive.
(imported from commit e4b1f763522406255ce8be3c8d51b5d855a770cd)
2013-12-05 14:36:29 -05:00
Tim Abbott 55047280c9 Include a newline at the end of our JSON responses.
Otherwise, it's super annoying to work with these using curl.

(imported from commit 0047f07786dc07d7ac5cb671492f8ca40986d8cf)
2013-12-04 13:47:48 -05:00
Leo Franchi 1b53a77f1b Expect an ios client instead of iphone
(imported from commit d5d3cd129d7005fd9294096d3899df86155cd366)
2013-12-03 14:27:51 -05:00
Tim Abbott abded5f886 Upgrade event flag collapsing to support all flag changes.
(imported from commit a2ca74b779d67148aee41e92e4a4cb716620e566)
2013-12-03 12:21:04 -05:00
Tim Abbott d2486670a8 event: Add deduplication of server restart events.
(imported from commit e464aa8b68958fb9f9fe8c199b9e73df510d22bc)
2013-12-03 12:20:36 -05:00
Tim Abbott 54b06d046b events: Add support for not storing redundant events.
This should dramatically improve the speed of the dump/load part of
our restart process, especially with large long-lived event queues.

(imported from commit ae4ae20ba2ca4433e25a5e7beeb4fa4882c53972)
2013-12-03 12:19:42 -05:00
Tim Abbott ad9659ea54 events: Fix zulip_finish to not make assumptions about how queues work.
Previously, we had an issue with the ACKing protocol, where if a
virtualizable event (like a "read" flag) was dispatched to a queue
client immediately, we would not properly ACK the change because it
had been made a virtual event.

(imported from commit ea09812f8a5ba1d5aad65f536022e3dbc77b0f9e)
2013-12-03 12:19:42 -05:00
Jessica McKellar e15cc972ec digest: refactor sending the emails into a function for easier testing.
(imported from commit 2f3c8fb8e812c04ab838b1034ae0811309186625)
2013-12-02 18:58:03 -05:00
Jessica McKellar 207cfc449d Add unsubscribe links to digest e-mails.
(imported from commit 4e2a324decf4bc694752cc24b9085361338a08a5)
2013-12-02 18:58:03 -05:00
acrefoot da1466cfa9 handle case of message getting read while processing missed_message
(imported from commit cd53efe3645f5dc8943becc2b8dbd3d607bb8eba)
2013-12-02 16:00:56 -05:00
Tim Abbott 559f0896a1 Revert "events: Add support for not storing redundant events."
This reverts commit cc493fa50b4c339257e060b3f0c0956c682e449d.

(imported from commit 9e8edfc11d415d61581da482d66bc9fa30cbefdf)
2013-12-02 11:03:04 -05:00
Tim Abbott 5b601d8492 Revert "event: Add deduplication of server restart events."
This reverts commit 05467929ba376a0ce173119c5d90d83671fc544b.

(imported from commit 4f24a0f1fe72e71fa368e1d818d9406a76c131be)
2013-12-02 11:03:04 -05:00
Tim Abbott 2c2f404cb6 Revert "Upgrade event flag collapsing to support all flag changes."
This reverts commit dadf3852f2ca7aaa85fb97b3d265c990b4e5a38a.

(imported from commit de5a95572aa955b4a9419b80a99d2446c51730e7)
2013-12-02 11:03:04 -05:00
Tim Abbott 10bce7ffe8 Revert "Fix loading event queues that don't have a virtual_events attribute"
This reverts commit 99c5ae332d253313ed728883a388c61191ed3021.

(imported from commit d7b601a2897bac6e14b675e689ee63ccc6e8d67d)
2013-12-02 11:03:03 -05:00
Tim Abbott 5bc826067e Revert "Add comment in ClientDescriptor.to_dict and EventQueue.to_dict to avoid future bugs"
This reverts commit 45560539d09e9e1922130349be729005b786e632.

(imported from commit aa266941f03cdbe12f4315e65ec6c472cc207204)
2013-12-02 11:03:03 -05:00
acrefoot 717e06d94c remove log message on push notification
(imported from commit 45fc89e67a79aac3484c911d88aff3e470b4eff6)
2013-11-27 18:00:33 -05:00
acrefoot 8800dcdce4 more helpful error message when missing APNS certs
(imported from commit e21dc627844b9e28f93a924097176481d4135947)
2013-11-27 18:00:33 -05:00
acrefoot 8ee7536012 Send push notifications more aggresively
Unbundle the push notifications from the missed message queue processors
and handlers. This makes notifications more immediate, and sets things up
for better badge count handling, and possibly per-stream filtering.

(imported from commit 11840301751b0bbcb3a99848ff9868d9023b665b)
2013-11-27 18:00:31 -05:00
Zev Benjamin 9d0f801ad6 Exempt a particular client name from heartbeats for testing
(imported from commit 3914130384a7a9854b006405ffd2fe379bbd090d)
2013-11-27 10:39:31 -05:00
Jessica McKellar 2ba9415178 Factor welcome e-mail enqueuing into its own function.
(imported from commit 85054ee3e567d2ba7a4ed94c52a311d02f5e4d43)
2013-11-27 09:30:00 -05:00
Zev Benjamin 8899c62567 Add comment in ClientDescriptor.to_dict and EventQueue.to_dict to avoid future bugs
(imported from commit 45560539d09e9e1922130349be729005b786e632)
2013-11-26 18:39:31 -05:00
Zev Benjamin ce00fe0cd5 Fix loading event queues that don't have a virtual_events attribute
(imported from commit 99c5ae332d253313ed728883a388c61191ed3021)
2013-11-26 18:32:58 -05:00
Tim Abbott e9f4612281 Upgrade event flag collapsing to support all flag changes.
(imported from commit dadf3852f2ca7aaa85fb97b3d265c990b4e5a38a)
2013-11-26 18:12:52 -05:00
Tim Abbott fb5c42707e event: Add deduplication of server restart events.
(imported from commit 05467929ba376a0ce173119c5d90d83671fc544b)
2013-11-26 18:12:52 -05:00
Tim Abbott 1db8fba5aa events: Add support for not storing redundant events.
This should dramatically improve the speed of the dump/load part of
our restart process, especially with large long-lived event queues.

(imported from commit cc493fa50b4c339257e060b3f0c0956c682e449d)
2013-11-26 18:12:52 -05:00
Leo Franchi 501f063426 Replace email_to_domain with split_email_to_domain and resolve_email_to_domain
Now that we support email aliases, we have to be careful when going from
an email address to a domain that we assume we can use to get a Realm
object for. When we care about the Realm's domain, we want to follow
any RealmAliases that exist for a certain domain.

When we just care about the original email address domain itself,
for comparison or other purposes, use split_email_from_domain

This removes the ambiguity of having to decide when to use
email_to_domain + RealmAlias or just email_to_domain

(imported from commit 0e199495502d946ce2e1aae56263e7e8665be4ed)
2013-11-26 10:45:01 -05:00
Kevin Mehall b8a6100d2e bugdown: Open #-relative links in the same window.
Stream notifications shouldn't leave you with tons of Zulip windows.

(imported from commit e5ebfeff51f1e579d30a3a12eaf7341ef82f0449)
2013-11-25 17:34:59 -05:00
Jessica McKellar dbc1b4b501 Add a one-click unsubscribe link to missed message e-mails.
(imported from commit 719eb61c9b7722eb1b0996b4d5f2f8f5929ad701)
2013-11-25 15:18:40 -05:00
Tim Abbott 7cb261a0ed Set default streams to engineering,social when creating realms.
(imported from commit 4e174fec162be4138da099edfc4cfb8d5662b7f1)
2013-11-25 14:00:28 -05:00
Zev Benjamin bc43981ccf socket: Explain why we disable some transports
(imported from commit 670d2f558a11a6f4fcce3f2e107582c4425b9285)
2013-11-25 11:58:02 -05:00
acrefoot 4cd1d69ea0 Add the ability to unsubscribe all users from a stream
This resolves Trac #2022

(imported from commit 23d5129a6bec40c33eaf71953cd41ec387646a39)
2013-11-22 17:12:51 -05:00
Jessica McKellar d0a49e3a80 Replace decentralized topic truncation with a single check when sending messages.
(imported from commit 291c999147dd59a9c7e1d9de446293755c17e99d)
2013-11-22 13:53:03 -05:00
Jessica McKellar baef4b9b00 Truncate long messages instead of rejecting them.
(imported from commit 41e0ce1e91ab7a441f2b21ad445344a86ae18fc7)
2013-11-22 13:53:03 -05:00
Steve Howell ace12bcab6 Support arbitrarily nested fenced quote/code blocks.
Now we can nest fenced code/quote blocks inside of quote
blocks down to arbitrary depths.  Code blocks are always leafs.
Fenced blocks start with at least three tildes or backticks,
and the clump of punctuation then becomes the terminator for
the block.  If the user ends their message without terminators,
all blocks are automatically closed.

When inside a quote block, you can start another fenced block
with any header that doesn't match the end-string of the outer
block.  (If you don't want to specify a language, then you
can change the number of backticks/tildes to avoid amiguity.)

Most of the heavy lifting happens in FencedBlockPreprocessor.run().
The parser works by pushing handlers on to a stack and popping
them off when the ends of blocks are encountered. Parents communicate
with their children by passing in a simple Python list of strings
for the child to append to.  Handlers also maintain their own
lists for their own content, and when their done() method is called,
they render their data as needed.

The handlers are objects returned by functions, and the handler
functions close on variables push, pop, and processor.  The closure
style here makes the handlers pretty tightly coupled to the outer
run() method.  If we wanted to move to a class-based style, the
tradeoff would be that the class instances would have to marshall
push/pop/processor etc., but we could test the components more
easily in isolation.

Dealing with blank lines is very fiddly inside of bugdown.

The new functionality here is captured in the test
BugdownTest.test_complexly_nested_quote().

(imported from commit 53886c8de74bdf2bbd3cef8be9de25f05bddb93c)
2013-11-21 17:13:17 -05:00
Tim Abbott 272e1c2d92 Revert "Don't send heartbeat events to clients that don't request them."
This reverts commit 1147814b22fb9737a807057ddbdbe0e9554086e0.

This seems to with some probability screw up our Zephyr mirroring
script.

(imported from commit 4f82452f1b0ca98e6b895db020e071d2daa325e4)
2013-11-21 12:53:26 -05:00
Tim Abbott ca8225cf47 [manual] Add endpoint to cleanup a finished events queue.
This requires a puppet apply on each of staging and prod0 to update
the nginx configuration to support the new URL when it is deployed.

(imported from commit a35a71a563fd1daca0d3ea4ec6874c5719a8564f)
2013-11-20 18:34:15 -05:00
acrefoot 5973f20b69 fix use of EXTERNAL_URL to EXTERNAL_HOST, and some typos
(imported from commit 3e6e0f337671b289202bc86c6224854b6ee623c5)
2013-11-20 17:39:07 -05:00
Steve Howell 0d920dae87 Extract FencedBlockPreprocessor.placeholder().
(imported from commit cf6dab74ae97a57498e1e1807c7c4f272f3f448b)
2013-11-20 16:06:53 -05:00
Steve Howell 2955cac47e Extract vars in FencedBlockPreprocessor.process_fence().
(imported from commit 092a8e5f7479f1d918219cae058b6a867743c7f9)
2013-11-20 14:30:56 -05:00
Steve Howell b3344c2f49 Move langclass var into FencedBlockPreprocessor.format_code().
(imported from commit e468a8b6a12494facb0dfa97eff90e02cd14f169)
2013-11-20 14:30:55 -05:00
Steve Howell 46838ec6d0 Simplify FencedBlockPreprocessor.format_fence().
(imported from commit 741f4f0640756d545019b6fe6850af3198b351ba)
2013-11-20 14:30:55 -05:00
Steve Howell c0f312ec29 DRY up m.group('code') in FencedBlockPreprocessor.
(imported from commit b23caae1ab63a9adb80025b628bc1a7c6a988ce0)
2013-11-20 14:30:55 -05:00