Commit Graph

350 Commits

Author SHA1 Message Date
Zev Benjamin 233290d578 Add a management command to purge a rabbitmq queue
This is useful in debugging when you just want to discard all the
messages in a queue because they have the wrong structure.

(imported from commit 8559ac74f11841430b4d0c801d5506ebcb74c3eb)
2013-09-23 11:25:58 -04:00
Zev Benjamin 7d0fd364b2 Remove duplicate call to update_user_activity
(imported from commit e9d2dce02cb6863a3ba66045a7b88bd351fd0c5d)
2013-09-23 11:25:58 -04:00
Zev Benjamin d7d17c993c Fix using SimpleQueueClient with multiple queues
(imported from commit 18bda1430d76f8b436ad42f85d0ce3cab8fd8193)
2013-09-23 11:25:58 -04:00
Steve Howell 660475bd0c Hide muted messages.
Instead of collapsing muted messages, just hide them altogether
in view where it makes sense to hide them.

(imported from commit 1c2c987ff302ceb135a025753cf421b4de1aea71)
2013-09-23 10:53:06 -04:00
Steve Howell 022ef6e9b9 Add narrow.muting_enabled()
(imported from commit 8972d1e3ca347397664094a7f150ec86fd2f8f8e)
2013-09-23 10:53:06 -04:00
Waseem Daher 7afd3ecc81 Custom invite text for MIT invitations.
(imported from commit 21b72861e991159eb7faa74809edb8d01027c6c8)
2013-09-20 12:06:26 -04:00
Steve Howell 6eda9c3314 Rename function to send_pm_if_empty_stream().
This used to have a really long name.

(imported from commit d3027a75f49db428239d432cfaa8821aa733eb04)
2013-09-20 10:19:07 -04:00
Steve Howell a457135a25 Use efficient stream.num_subscribers() for bot PMs.
Use stream.num_subscribers() in check_if_a_bot_is_sending_a_message_to_an_empty_stream().

The num_subscribers() function using Django's count() method, which returns
a single row, vs. len() on an iterator of query rows.

(imported from commit 6157fe248945e9288ee71d8cc39fb6dda4e9a247)
2013-09-20 10:19:07 -04:00
Steve Howell 4b75dd0d85 Add Stream.num_subscribers
(imported from commit 9b79ab84516c3634ef6795ef2d90db37553cc0d1)
2013-09-20 10:19:07 -04:00
Steve Howell 5ae5e3f881 Disable empty-stream PMs for MIT bots.
(imported from commit a855d612b6ba316fc183200c4f72102831b6179c)
2013-09-20 10:19:07 -04:00
Steve Howell 098720b6d1 Extract check_if_a_bot_is_sending_a_message_to_an_empty_stream().
(imported from commit 3312d70f6b94f4a13a42bbe0320f4bb1062262e7)
2013-09-20 10:19:07 -04:00
Steve Howell a8f6c2fe55 Add CheckMessageTest.test_bot_pm_feature() test.
This tests that a bot's owner gets sent a message if the bot
sends a message to a stream with no subscribers. (Presumably
the message will be a PM; we could make the test more precise
in the future.)

(imported from commit 0aaf931a90cb9c7bc3fde8ac545c6b6ad0a55668)
2013-09-20 10:19:06 -04:00
Steve Howell 7b748fed7a Add CheckMessageTest class
(imported from commit 07c4ba895570853c923703b1332c4fc403cc858e)
2013-09-20 10:19:06 -04:00
Jessica McKellar ee2cd51c4c generate_invite_links: work for open realms too.
(imported from commit ec2143d25cb2f6f4181ece7a25c79666334a1634)
2013-09-20 10:00:10 -04:00
Zev Benjamin 9b84b2dd67 github: Reduce code duplication
This includes:
* Merging the pull request and issue subject creation functions
* Factoring out pull request and issue content creation

(imported from commit 9cf90e999482a1998431e6483788522101607167)
2013-09-19 23:44:21 -04:00
Zev Benjamin e5e7e2f7bf github: Rearrange code to put the pull request and issue handlers next to each other
This makes their parallelism clearer.

(imported from commit 328d6df9443ed68ff5f462b405a930ecb6d90155)
2013-09-19 23:44:21 -04:00
Zev Benjamin 0394459f04 github: Put pull request comments on the 'commits' stream
(imported from commit d50c1f577737a07da1f08a1fbfc4242ca9d6d4a1)
2013-09-19 23:44:21 -04:00
Zev Benjamin 1c82a9ecfa github: Format pull request messages more like issue messages
(imported from commit cba3a84c3abb6f33495e5afbe8e03d1a7a7d046c)
2013-09-19 23:44:21 -04:00
Zev Benjamin f99cf15358 github: Add tests for pull requests
(imported from commit dd099cd0e630c09f490ecd5aac761ce131b0a8e2)
2013-09-19 23:44:21 -04:00
Zev Benjamin df5c4bb202 github: Enable pull requests for zulip.com so we can test them
(imported from commit ab13951a77d6563a9ec41d6ed7f991579623c565)
2013-09-19 23:44:20 -04:00
Zev Benjamin c5d4a640be tests: Refactor Github tests to reduce code duplication
(imported from commit 7c34bd2eefe706898479f96a03685f6e027bf43f)
2013-09-19 23:44:20 -04:00
Zev Benjamin f47258c740 github: Factor out pull request and issue subject creation
(imported from commit 539238f78b0ee49672a1338fc96d00bb21ce539d)
2013-09-19 23:44:20 -04:00
Waseem Daher 190390d8ea Allow MIT to invite coworkers.
(imported from commit 273b3abdd3dcc49c01c1a04001450983d4acb000)
2013-09-19 17:14:41 -04:00
Steve Howell e79e74ade4 Add narrowed_to_topic() helper function.
(imported from commit 9190caa9b183893fb7ae883686b9bdbd661d5895)
2013-09-19 15:41:00 -04:00
Steve Howell a4e2ed9fd2 Add test_narrowed() to narrow.js unit tests.
(imported from commit 060032ac00d0c28a9625e2885ee3156bc94ff130)
2013-09-19 15:40:59 -04:00
Steve Howell 51b8caf442 Add set_filter() to narrow unit tests.
(imported from commit 60734fa1a9d0d28e8c0277a41cc69186bc2b6c0e)
2013-09-19 15:40:59 -04:00
Kevin Mehall 6695105bb9 Fix 2499d7b to not break user registration.
Some bots created by us do not have owners. Don't try to send a
message to the nonexistent owner.

(imported from commit ab952eccd7d6c4728e9477a106142214b5c81ca9)
2013-09-19 13:42:20 -04:00
Jessica McKellar 7e733a1b91 Don't rate-limit missed message e-mails.
Instead just rely on the 2-minute delay in the management command to
batch conversations.

We've had people report being confused or thinking the feature was
broken when they didn't get e-mails because of our rate-limiting, so
let's see if this is not too overwhelming.

(imported from commit 706ddb07b906b5c2edea1159c04acc2ee6f06e29)
2013-09-19 11:55:42 -04:00
Steve Howell de0d50d31a [important] Fix recent regression w/maybe_get_subscribers
(imported from commit 8069f729dcb2bfec1c1df8f50a88fe49545a9a87)
2013-09-19 09:37:39 -04:00
Steve Howell c6a9297f1a Add blueslip warnings to stream_data functions.
Warn inside these functions when you get data on streams that you
are not subscribed to:

  add_subscriber
  remove_subscriber
  user_is_subscribed

The back end should be smart enough not to spam us with subscriber
info that we don't care about.

(imported from commit b27644be2abc37c11ddff884ef392ea208bd1bd3)
2013-09-19 08:52:54 -04:00
Steve Howell 3c69a6dd26 Bulk-send peer_add events in bulk_add_subscriptions().
Send a single peer_add event per stream that has multiple users.

(imported from commit 1027b212557f848d437128a03698ecd54575c3fd)
2013-09-19 08:52:54 -04:00
Steve Howell c8c64984d3 Eliminate redundant peer_add notifications.
Don't send peer_add notifications to users who are already
getting add notifications, because they will already know
about subscribers.

(imported from commit 726b54ae0e30b71440b17d9c51b026872ea96218)
2013-09-19 08:52:54 -04:00
Steve Howell c58d366178 Optimize notifications from do_add_subscription().
Instead of sending one notification per user, bulk them up with
the users parameter.

(imported from commit c296f23599a214a63ef6f09406a9deaea469a7f3)
2013-09-19 08:52:54 -04:00
Steve Howell 8182d60961 Optimize query used by do_add_subscription().
It only grabs the user_profile_id column now.  This leads to a
speedup of about 16x between grabbing large ORM objects vs.
small 1-column dictionaries.

(imported from commit 95150bff3fdcbe250b04f014062224af42a6644f)
2013-09-19 08:52:54 -04:00
Steve Howell e89c6f64bd Extract notify_peers().
Splitting out notify_peers() will give us flexibility for cleaning
up how we notify peers for bulk adds.

(imported from commit e108fa2c432cc1fe54d788c58c82c983e0f2394e)
2013-09-19 08:52:54 -04:00
Steve Howell d24d8e4b36 Optimize showing subscribers on the settings page.
If you expand subscribers on your settings page, you will now see
a query like this in your postgres logs:

SELECT "zerver_userprofile"."email"
FROM "zerver_subscription" INNER JOIN "zerver_recipient" ON ("zerver_subscription"."recipient_id" = "zerver_recipient"."id") INNER JOIN "zerver_userprofile" ON ("zerver_subscription"."user_profile_id" = "zerver_userprofile"."id") WHERE ("zerver_recipient"."type" = 2  AND "zerver_subscription"."active" = true  AND "zerver_recipient"."type_id" = 40  AND "zerver_userprofile"."is_active" = true )

The join's still complicated, but the list of fields is one instead of 40+.

(imported from commit 48de1f888193a4d23fcea52d0b633d134e4a3ff7)
2013-09-19 08:52:53 -04:00
Steve Howell 9cdedf7e6c Optimize adding subscriptions with maybe_get_subscriber_emails().
To get emails, query one field instead of 40-ish fields.

(imported from commit d2dbbe23871df76bc7d431dcd9b19ebd7a58c161)
2013-09-19 08:52:53 -04:00
Steve Howell 986dd8289e Optimize get_subscribers back end.
get_subscribers_backend() now calls the new get_subscriber_emails()
function, which just queries the email field:

  "zerver_userprofile"."email"

...instead of querying about 40 fields that it never uses.

I was able to verify the query slimming by watching my postgres server log.

Also, you can verify that the ORM does roughly 16x less work using values():

>>> def f(): return [sub.user_profile.email for sub in list(Subscription.objects.all().select_related())]
...
>>> def g(): return [row['user_profile__email'] for row in list(Subscription.objects.all().values('user_profile__email'))]
...
>>> def timeit(func): t = time.time(); func(); return time.time() - t
...
>>> timeit(f)
0.045198917388916016
>>> timeit(g)
0.002752065658569336

(imported from commit a69f690a96d076b323fdfc2f4821b0548bdfac7f)
2013-09-19 08:52:53 -04:00
Steve Howell db2c9ff8c2 Extract get_subscribers_query
(imported from commit fea253591093a1f0cf722c1d4bcfb0df4f881984)
2013-09-19 08:52:53 -04:00
Steve Howell b2a7860d56 Add test_multi_user_subscription() to backend tests.
(imported from commit aa6849f7c01f083144acd3ecb72cf7e31794dad7)
2013-09-19 08:52:53 -04:00
Steve Howell b02770440a Add tornado_redirected_to_list() to tests.py
(imported from commit e515d3e4b9f04b5db56d4c9433bc9ba15fd3b4da)
2013-09-19 08:52:52 -04:00
Kevin Mehall 8c7f1ecffa bugdown: Don't generate garbage on invalid links.
LinkPattern returned a string which contained a placeholder if the URL was
considered invalid. AtomicLinkPattern wrapped this in an AtomicString,
where the placeholder doesn't get removed properly.

m.group(0) is always incorrect because python-markdown modifies your regex
to include more than you specified (this is why part of the message got
duplicated).

(imported from commit 576bdf09c2b677cf4bc56484c363eb05f2110158)
2013-09-18 16:22:13 -04:00
Kevin Mehall 63cff6eb71 Make populate_db set the is_bot flag on bot accounts.
(imported from commit 93a507faa2b19fde05376fd73ed3a65149cfce68)
2013-09-18 16:16:17 -04:00
Leo Franchi d88266b41d Warn bot owners when a bot sends a message to a nonexistent stream
(imported from commit 2499d7bd9d176a7ab43f751eb855813d0a479bd4)
2013-09-18 15:54:13 -04:00
Jessica McKellar c2e704e4b6 Instead of erroring out, do our best to format HTML-only e-mails.
(imported from commit a58e65864ebb3ef539cf15902b59636cd3912b69)
2013-09-18 15:41:27 -04:00
Jessica McKellar 8ab3ae9da2 email mirror: handle non-ASCII message bodies.
(imported from commit 1a8e21b7ff5441c21147bab85f5a460a7567c5e3)
2013-09-18 15:41:27 -04:00
Leo Franchi 5441e36167 Bulk update messages in deprecated cache when updating message
(imported from commit 5e68e4eec83c0ed02b9997b0cb3f93ce8f0a7fdd)
2013-09-18 13:27:58 -04:00
Leo Franchi c588c7938c Allow message topic changing to propagate backwards
(imported from commit 470178ef5f7aaf09d4528c88ae6e33f17538fcaf)
2013-09-18 13:27:58 -04:00
Leo Franchi 0a1a63c87f Blacklist some webreq urls that we don't want to store for metrics
(imported from commit 60def99010e5c55e77acc65aa516459c6e4ffaae)
2013-09-18 11:56:25 -04:00
Jessica McKellar 28f80211ec email mirror: handle non-multipart e-mails when checking for attachments.
(imported from commit 2132d604bb353ef6e8ca2c371479b2b2146d8603)
2013-09-17 16:20:28 -04:00