Commit Graph

302 Commits

Author SHA1 Message Date
Leo Franchi 0a1a63c87f Blacklist some webreq urls that we don't want to store for metrics
(imported from commit 60def99010e5c55e77acc65aa516459c6e4ffaae)
2013-09-18 11:56:25 -04:00
Jessica McKellar 28f80211ec email mirror: handle non-multipart e-mails when checking for attachments.
(imported from commit 2132d604bb353ef6e8ca2c371479b2b2146d8603)
2013-09-17 16:20:28 -04:00
Steve Howell 30b6f54a36 Don't cache UserPresence info.
(imported from commit ff590bdf5d84bb9b3cedc561c2f2fbf7f7592a66)
2013-09-17 14:51:56 -04:00
Zev Benjamin 3d11d0e420 github: Remove the issue title from message bodies
(imported from commit d7298696011c548c26cc4ec4d13c7f1d83efd785)
2013-09-17 13:55:39 -04:00
Zev Benjamin 4996eb1e61 tests: Add Github issues tests
(imported from commit 1aa7a2c78f148f5e6ea9c7551f25374cbfe32f15)
2013-09-17 13:55:39 -04:00
Zev Benjamin 067997287e tests: Use real data for testing Github pushes
(imported from commit 83b800a8711cf045ffc7da3305227f495819a5da)
2013-09-17 12:59:18 -04:00
Zev Benjamin fc7106aa1b github: Only try to capture Github events when deployed
(imported from commit 99e89f52d6b1653b6bf8b65b8ab44cd06e3fc28f)
2013-09-17 12:59:18 -04:00
Zev Benjamin b1075438d1 github: Add support for issue comments
(imported from commit 1a5317303292de2a7adcb2dc6a3b5bd3d27d1c73)
2013-09-17 12:55:57 -04:00
Zev Benjamin 75196cfe9e github: Improve issues formatting
* Remove the action from the topic and add the issue title
* Only show the issue body on open or reopen

(imported from commit f08eb40f36122d2498fe0c36a69df9e606296ff3)
2013-09-17 12:55:57 -04:00
Zev Benjamin e5cf24b2fd Fully quote payload body contents in integrations
(imported from commit 3cb03eabbd917f878b37aa226eb3f3ecda63d5e7)
2013-09-17 12:55:57 -04:00
Steve Howell e1b881629c Use fold_case:true for subscribers Dict
(imported from commit 8acc794cb5369baadef512a9fcdb3537231dd604)
2013-09-17 10:32:13 -04:00
Steve Howell 4b90f1685d Add stream_data.set_subscribers() method.
This allows us to encapsulate our use of Dict for the internal
data structure.

(imported from commit e8acc50b4c17d339015cb9db9939b9452a62cc8b)
2013-09-17 10:32:11 -04:00
Jessica McKellar 3db51fa17f Upload and display attachments sent through the email mirror.
(imported from commit b3ac24b71a2e7c354d69ecae9174316102ab8f84)
2013-09-17 10:00:01 -04:00
Jessica McKellar 97e6709d49 uploads: have upload_image_to_s3 take file data instead of a file object.
We have to read the data anyway, and we don't have a convenient file
handle for uploads from attachments sent through the e-mail gateway.

(imported from commit 86260a4eaceef85c82707929a80558e11dc54ef6)
2013-09-17 10:00:01 -04:00
Jessica McKellar 8a40fb346c Be able to upload images to S3 from outside the web client.
We'll need this to upload attachments sent through the email gateway.

(imported from commit b2f5eb736b65f1478189c3aebd4cb37f1ad540fb)
2013-09-17 10:00:01 -04:00
Jessica McKellar 2c0ba365ae email mirror: show more debugging information on errors.
(imported from commit 57f5b8b57062501443367953239526f1ec78eb7f)
2013-09-17 10:00:00 -04:00
Steve Howell 1e912c7e22 Exclude bots in the user presence query.
(imported from commit 02668ba5db85e12ac0aefc0fa2480d38ffed4921)
2013-09-16 16:27:16 -04:00
Jessica McKellar cfeaa2be9b Add backend support for realm admins to rename streams.
(imported from commit eaf419f1f68dfd350b8c99e8a5089ec316f0c409)
2013-09-16 16:16:13 -04:00
Steve Howell e83c026a27 Set focus-related flags correctly after a server reload
(imported from commit 408469c0da0cfa5513e6838329816cd88665981f)
2013-09-16 12:29:34 -04:00
Steve Howell 98af7e8664 Avoid unnecessary calls to receiver_is_idle()
(imported from commit 88443f396309def2fa49360337ef5b7748ff6f79)
2013-09-16 11:29:13 -04:00
Steve Howell 1836d8958e Add comment about sending presence updates
(imported from commit da9c521a6ad652e4dbcaabcc3a24b3e23376f545)
2013-09-16 11:29:13 -04:00
Steve Howell 83e517fcf9 Flush cache on all user presence updates.
(imported from commit 130eac36f9932350a29ed88f11dd3e2f6bf3f979)
2013-09-16 11:29:13 -04:00
Steve Howell 7868787ae0 Fix indentation level in cache_save_user_profile().
(imported from commit d4dff41fa7913b15a65e10fbafb5b7b49d6b6837)
2013-09-16 11:29:13 -04:00
Steve Howell 85f9b6695f Optimize user presence/activity query.
The get_status_dict_by_realm helper gets called whenever our
 realm user_presences cache expires, and it used to query these fields:

    "zerver_userpresence"."id", "zerver_userpresence"."user_profile_id", "zerver_userpresence"."client_id", "zerver_userpresence"."timestamp", "zerver_userpresence"."status", "zerver_userprofile"."id", "zerver_userprofile"."password", "zerver_userprofile"."last_login", "zerver_userprofile"."is_superuser", "zerver_userprofile"."email", "zerver_userprofile"."is_staff", "zerver_userprofile"."is_active", "zerver_userprofile"."is_bot", "zerver_userprofile"."date_joined", "zerver_userprofile"."bot_owner_id", "zerver_userprofile"."full_name", "zerver_userprofile"."short_name", "zerver_userprofile"."pointer", "zerver_userprofile"."last_pointer_updater", "zerver_userprofile"."realm_id", "zerver_userprofile"."api_key", "zerver_userprofile"."enable_desktop_notifications", "zerver_userprofile"."enable_sounds", "zerver_userprofile"."enter_sends", "zerver_userprofile"."enable_offline_email_notifications", "zerver_userprofile"."last_reminder", "zerver_userprofile"."rate_limits", "zerver_userprofile"."avatar_source", "zerver_userprofile"."tutorial_status", "zerver_userprofile"."onboarding_steps", "zerver_userprofile"."invites_granted", "zerver_userprofile"."invites_used", "zerver_userprofile"."alert_words", "zerver_userprofile"."muted_topics", "zerver_client"."id", "zerver_client"."name"

Now it queries just the fields it needs:

  "zerver_client"."name", "zerver_userpresence"."status", "zerver_userpresence"."timestamp", "zerver_userprofile"."email" FROM "zerver_userpresence"

Also, get_status_dict_by_realm is now namespaced under UserPresence as a static method.

(imported from commit be1266844b6bd28b6c615594796713c026a850a1)
2013-09-16 11:29:10 -04:00
Steve Howell 0d2c6f756f Remove code to pre-fetch presence objects.
(imported from commit 546e98bedd809e66f506581434402caad5a984de)
2013-09-16 11:28:47 -04:00
Steve Howell e881fdd2ff Change the cache timeout for get_status_dict() back to a minute.
This function gets user presence information, which changes rapidly
and requires a pretty simple query.

(imported from commit f9b9f0f22277335c76eb4371930a4fff2758a240)
2013-09-16 11:24:29 -04:00
Steve Howell 99dd2ed8df Add presences info to the event data for process_new_message().
The do_send_messages() populates the user_presences data structure
for process_new_message(), so that Tornado code never needs to hit
the database or memcached to get the user presence info.

(imported from commit 194aeaead8fa712297a2ee8aff5aa773b92f1207)
2013-09-16 11:24:12 -04:00
Leo Franchi a4744b10ed Better logging for error diagnosis
(imported from commit a2fc108ca1230b671c7e35651e3d53042386b4bd)
2013-09-13 17:56:16 -04:00
Leo Franchi 6e56342cf6 Send missedmessage notifications if user is idle for >1hr
(imported from commit 573f46a77497cb2f73eae3b4a648e466977e6247)
2013-09-13 17:33:34 -04:00
Leo Franchi 7bb96bd36b Extract data from cache in tornado_callbacks with cache_get_many
This reduces the number of memcached calls we make in our time-
slice-limited tornado event handler.

(imported from commit 8903ce4ac754ba82d57e04d1b0356be7533edee2)
2013-09-13 17:33:34 -04:00
Steve Howell 5eaeddea12 Get stream_data.js back to 100% coverage
(imported from commit 5b82a10d50b6599c0c42604fe86275b21f9a6396)
2013-09-12 17:23:52 -04:00
Steve Howell 0165dc54ad Create blueslip error for undefined keys in Dict.
We create a blueslip error for undefined keys in Dict.  This led
to a straightforward change in the unit tests for Dict.  For the
unread test, to avoid the blueslip error, we had to be more specific
in setting up a user in one place, but this reduced our coverage,
leading to another small test being added.

(imported from commit 33e14795500d9283de2a7c03c4c58aec11cea4b8)
2013-09-11 18:07:37 -04:00
Steve Howell 9eefc8ae62 Test fold_case option of Dict
(imported from commit 630643767a21818c1ef4adb557c794e560783c3e)
2013-09-11 18:07:37 -04:00
Steve Howell e9adaaa0c0 Avoid Dict crashes for undefined keys when fold_case is true.
The exceptions were cryptic before, and they were inconsistent with
the fold_case: false behavior.

(imported from commit a40704d1a22bcdc60d91be832ee3c81eb416c6dd)
2013-09-11 18:07:37 -04:00
Steve Howell a8ebd80c89 Test that vanilla Dict handles undefined keys
(imported from commit 09846eafa55f2321ca10db950ad3785ce78ae2c6)
2013-09-11 18:07:37 -04:00
Steve Howell b7ef86bc34 Added /json/set_muted_topics
(imported from commit e9072cd273fabf0e072b6a5e9ed80b07142f1013)
2013-09-11 16:47:37 -04:00
Steve Howell 21456091e4 [schema] Add muted_topics to UserProfile
(imported from commit 5d804259be94ba83dc4f7541e50443dd1a42cc6b)
2013-09-11 16:47:37 -04:00
Steve Howell c6131b630b Handle undefined keys in muting.is_topic_muted
(imported from commit 96a9f0253e1c0e72a2b05577118dac438d190994)
2013-09-11 16:47:36 -04:00
Steve Howell f616148181 Make muting case insensitive for streams and topics
(imported from commit 6e731a5305a6d19896c61300aedf0fb4a387e148)
2013-09-11 16:47:36 -04:00
Steve Howell 0641e693fa Add muting.set_muted_topics
(imported from commit c84cd085e513d57e3913543d40cd8fd6b5d45b67)
2013-09-11 16:47:36 -04:00
Steve Howell 04c6a8198a Add muting.get_muted_topics
(imported from commit 5edb1d36ae66dfc42cf534edc452959df8306661)
2013-09-11 16:47:36 -04:00
Steve Howell 8626276dc3 Add muting.unmute_topic
(imported from commit c3aa08b7d50c70cba86b43c12d618ba1d96832b4)
2013-09-11 16:47:35 -04:00
Leo Franchi a408192f92 Don't re-parse alert words more than necessary
(imported from commit 4fee6ad9592c1dc5f844889fcb6586192de6cb35)
2013-09-11 12:00:37 -04:00
Jessica McKellar 171306a37d Don't include not in home view messages towards bankrupcty unread count.
(imported from commit 64c038f1d41c091c88875f2c90e9b4dd695308a3)
2013-09-11 01:29:45 -04:00
Kevin Mehall 7fea9e0755 Fix race condition in 06-unread
There was nothing to ensure that the changes resulting from scrolling
happened before the unread counts were checked. We already had a long
wait there; might as well do those checks after it to ensure that the
DOM is updated.

(imported from commit 0d4014ae6a74dd684521fecabefc4bf79015f842)
2013-09-10 18:06:57 -04:00
Kevin Mehall 5991b0003f Fix 06-unread PM count selector for changes in 2b82fbb
(imported from commit 0d8e52f17fb06092efff7cdf4fc4b01ab4901e2f)
2013-09-10 18:06:57 -04:00
Leo Franchi 2eb8c6aae3 Add documentation and a duration argument to analyze_user_activity.py
(imported from commit 5668ec02a23dcfeb29760b55d8acee17cfdea9cf)
2013-09-10 13:25:59 -04:00
Leo Franchi 9ebb536c2d Use json_to_bool in REQ instead of manually using ujson.loads
(imported from commit 5a607f7aba4a8e4e5920a35df06c375cff98ff55)
2013-09-10 13:25:59 -04:00
Leo Franchi a1d7e6336c Add stub for .on() in activity test
(imported from commit 82727e18739cb8bed7b2c64f8013e7f0fe386b94)
2013-09-10 13:25:59 -04:00
Tim Abbott 9fcca3df4e [schema] Collect data on when users are active on site.
These engagement data will be useful both for making pretty graphs of
how addicted our users are as well as for allowing us to check whether
a new deployment is actually using the product or not.

This measures "number of minutes during which each user had checked
the app within the previous 15 minutes".  It should correctly not
count server-initiated reloads.

It's possible that we should use something less aggressive than
mousemove; I'm a little torn on that because you really can check the
app for new messages without doing anything active.

This is somewhat tested but there are a few outstanding issues:

* Mobile apps don't report these data.  It should be as easy as having
  them send in update_active_status queries with new_user_input=true.

* The semantics of this should be better documented (e.g. the
  management script should print out the spec above)x.

(imported from commit ec8b2dc96b180e1951df00490707ae916887178e)
2013-09-10 13:25:59 -04:00