Commit Graph

122 Commits

Author SHA1 Message Date
Leo Franchi 08ae641dd2 Pre-fetch data from the DB and hand to markdown thread
We want to avoid opening a DB connection in the markdown thread
as its DB connection might live for a long time

(imported from commit 7700b2ca793ee5e9add7f071b92f22a4bf576b3d)
2013-10-10 10:58:21 -04:00
Leo Franchi f0f4ff61f0 Attempt to convert JIRA mentions to Zulip mentions
(imported from commit 0d763c8fdefd8cc2c2ae527fdb80e66635ded3d6)
2013-10-07 13:59:36 -04:00
Leo Franchi 02a48e94f0 Turn JIRA users into mentions where possible
(imported from commit faddc2127fe3fdb95e278df06a834cda700b97f4)
2013-10-07 13:59:36 -04:00
Zev Benjamin 4129541206 github: Add support for commit comments
(imported from commit 43abd740e563952d1291939e6a530b8366232eb3)
2013-10-04 15:53:21 -04:00
Steve Howell 95321a9845 Clean up a few @slow decorators in tests.py.
A few "slow" tests aren't as slow any more, for whatever reason,
so we're setting a higher bar going forward.

(imported from commit 642137cebb7826f4512b5635da9d7b75bd5c35f4)
2013-10-04 00:25:49 -04:00
Kevin Mehall 72a0b67c80 Prevent other markdown operators from touching URLs.
The text of manual links are already AtomicStrings, so linkified strings
should be too.

Moves emoji detection to happen after linkification, so the emoji rule
won't look at links.

(imported from commit 9c56bce6a0e873b398255e0762dfb312a4a9a64e)
2013-10-03 17:26:47 -04:00
Kevin Mehall 8842349629 Fix links with invalid emoji in them.
InlinePatterns should return None on failure, not text that may
have placeholders in it.

(imported from commit f9d8d22b2b8cfa7a92ecf3e52a6c76b48e6f0175)
2013-10-03 17:26:47 -04:00
Steve Howell f6794ce849 Remove stray print statement in tests.py
(imported from commit 2be8ceb83664cfb9f3158e23894637ac513efa41)
2013-10-03 13:03:33 -04:00
Tim Abbott 3422b02e52 Add some unit tests for gather_subscriptions.
(imported from commit 10c362e36e2e9044918e22449b6d82acb9c867f3)
2013-10-02 16:37:56 -04:00
Steve Howell 00cf695af2 Add "Durations" tab to /activity.
This tab shows how long each user has been on during the last 24
hours, using data from UserActivityInterval.  Much of the code
is borrowed from analyze_user_activity.py, but in this version
we set the time interval to be the last 24 hours and sort by
realm and email.  I also ensure that it only executes one
query to get all the data (and there's test coverage for that).

(imported from commit 7a2b80f52679054b03c5f5f42b2cda07d5599432)
2013-10-02 11:27:23 -04:00
Steve Howell 786c2cf759 Remove client-specific tabs from activity overview page.
Waseem is ok with removing the client-specific tabs on the
main /activity page.  This reduces the number of queries from
25 to 1.  We might eventually restore some of that logic, but
we will do it more efficiently.  A lot of the data for
non-website clients is kind of unreliable, anyway.

The page looks kind of funny with only one tab, but that
will be fixed in the next commit.

(imported from commit 54f08f89d5242ad3e045d8ca0d97b86617c15380)
2013-10-02 11:27:23 -04:00
Steve Howell 374a64acf8 Add test for /activity page.
For now we just verify that we don't explode, and we count
the number of queries (25).

(imported from commit 145c3373e2f6d005827af743729e03e524167904)
2013-10-02 11:27:22 -04:00
Leo Franchi 028711baef Handle some JIRA formatting and make it like our own
(imported from commit 1e8369067668b0c375f226c1911814f496637d16)
2013-10-02 11:18:17 -04:00
Steve Howell c23ab96fb9 Add simulated_empty_cache() test decorator
(imported from commit 2dd03a522f9ee01a802c8e5ff4116db04763ed35)
2013-10-02 10:45:18 -04:00
Steve Howell 53b5768b14 Double speed of message fetching in get_old_messages_backend().
When we don't already have old messages in cache, we need to
fetch data from the database and create dictionaries for the
cache.  This commit makes that process work in 50ms, instead
of 130ms, for the data set in test_bulk_message_fetching(),
which is 602 records.  Before this commit we had about 132
microseconds of unnecessary churn per message, because we
were fetching DB fields we didn't need and incurring the cost
of the Django ORM.  Now we use values() to get only the columns
we need, and we take advantage of previous commits that make
our code less OO and more function-driven, so we can pass the
values directly to build_message_dict() without having to create
objects.

A couple caveats on this commit:

1) I haven't been able to get good measurements on the overall
effect on get_old_messages_backend().  If you kill the cache to
force DB queries, you introduce noise related to sessions and
user profiles.

2) Look at the long comment in this commit related to
re-rendering messages in this codepath.  The problem precedes
this commit.

(imported from commit dcb64aa9416f0e9583355ddd6dc3adfa746b9fc7)
2013-09-30 18:22:07 -04:00
Steve Howell 3c802e1b02 Add MessageExtractorTest.test_applying_markdown()
(imported from commit 0e88f999a871e93ae14d157e0087a75ae81d60a3)
2013-09-30 18:08:32 -04:00
Steve Howell 8ae789eaf8 Add test_message_extractor_basics().
(imported from commit 37e64cd4db1850e04bd9fdc37de1f38400436736)
2013-09-30 18:08:32 -04:00
Leo Franchi 4102b8e844 Add query count tests for bulk_add_subscriptions
(imported from commit 84a68e3e50178df383e66bb29008da6086709eda)
2013-09-27 10:26:11 -04:00
Steve Howell a99d2bebab Assert that registration involves <= 53 queries.
This isn't a very strong test, to be sure, but it's something.

(imported from commit 578dbc6601f38186fec9bfb061a8bbc322201ae7)
2013-09-26 09:55:43 -04:00
Steve Howell 6890c89936 Harden the queries_captured() test decorator.
Before, it was trying to use connection.queries, but Django
could pull the rug out from under us.  Now we monkeypatch
the CursorDebugWrapper methods instead.

(imported from commit 25d5bab47673f2b66a6325f48d33e66c31055ab3)
2013-09-26 09:55:42 -04:00
Zev Benjamin f6500d691e github: Make pull request comments go to the user-specified stream
(imported from commit dfce76bcba43dd4a0beba6b6af7642363193dcde)
2013-09-24 16:32:13 -04:00
Steve Howell 50641c3337 [perf] Reduce the number of DB queries in send_message.
For a 4-person stream, we were hitting the DB 8 times, and 4 of
those queries were to lazily get user.email for the 4 recipients
due to upstream code using only().  I added user_profile__email
to the only() call.

I believe this regression started 9/18, and after pushing this
to prod, we would should look at this graph:

https://stats1.zulip.net/graphs/8274cd84588

(imported from commit 70629cb69fe5955c674ba76482609dfe78e5faaf)
2013-09-24 16:19:52 -04:00
Steve Howell 63de50d6f0 Add StreamMessagesTest.test_not_too_many_queries
(imported from commit bc49d7a0f932aecb1be7cecd0f6c5a9c95b2767d)
2013-09-24 16:19:52 -04:00
Steve Howell 9ee2f186ca Add queries_captured() context manager for tests.
(imported from commit 14a9529aef057e79bc12c979d88b595828a8b3a3)
2013-09-24 16:19:52 -04:00
Steve Howell 4b75dd0d85 Add Stream.num_subscribers
(imported from commit 9b79ab84516c3634ef6795ef2d90db37553cc0d1)
2013-09-20 10:19:07 -04:00
Steve Howell a8f6c2fe55 Add CheckMessageTest.test_bot_pm_feature() test.
This tests that a bot's owner gets sent a message if the bot
sends a message to a stream with no subscribers. (Presumably
the message will be a PM; we could make the test more precise
in the future.)

(imported from commit 0aaf931a90cb9c7bc3fde8ac545c6b6ad0a55668)
2013-09-20 10:19:06 -04:00
Steve Howell 7b748fed7a Add CheckMessageTest class
(imported from commit 07c4ba895570853c923703b1332c4fc403cc858e)
2013-09-20 10:19:06 -04:00
Zev Benjamin 0394459f04 github: Put pull request comments on the 'commits' stream
(imported from commit d50c1f577737a07da1f08a1fbfc4242ca9d6d4a1)
2013-09-19 23:44:21 -04:00
Zev Benjamin 1c82a9ecfa github: Format pull request messages more like issue messages
(imported from commit cba3a84c3abb6f33495e5afbe8e03d1a7a7d046c)
2013-09-19 23:44:21 -04:00
Zev Benjamin f99cf15358 github: Add tests for pull requests
(imported from commit dd099cd0e630c09f490ecd5aac761ce131b0a8e2)
2013-09-19 23:44:21 -04:00
Zev Benjamin c5d4a640be tests: Refactor Github tests to reduce code duplication
(imported from commit 7c34bd2eefe706898479f96a03685f6e027bf43f)
2013-09-19 23:44:20 -04:00
Steve Howell 3c69a6dd26 Bulk-send peer_add events in bulk_add_subscriptions().
Send a single peer_add event per stream that has multiple users.

(imported from commit 1027b212557f848d437128a03698ecd54575c3fd)
2013-09-19 08:52:54 -04:00
Steve Howell c8c64984d3 Eliminate redundant peer_add notifications.
Don't send peer_add notifications to users who are already
getting add notifications, because they will already know
about subscribers.

(imported from commit 726b54ae0e30b71440b17d9c51b026872ea96218)
2013-09-19 08:52:54 -04:00
Steve Howell c58d366178 Optimize notifications from do_add_subscription().
Instead of sending one notification per user, bulk them up with
the users parameter.

(imported from commit c296f23599a214a63ef6f09406a9deaea469a7f3)
2013-09-19 08:52:54 -04:00
Steve Howell b2a7860d56 Add test_multi_user_subscription() to backend tests.
(imported from commit aa6849f7c01f083144acd3ecb72cf7e31794dad7)
2013-09-19 08:52:53 -04:00
Steve Howell b02770440a Add tornado_redirected_to_list() to tests.py
(imported from commit e515d3e4b9f04b5db56d4c9433bc9ba15fd3b4da)
2013-09-19 08:52:52 -04:00
Kevin Mehall 8c7f1ecffa bugdown: Don't generate garbage on invalid links.
LinkPattern returned a string which contained a placeholder if the URL was
considered invalid. AtomicLinkPattern wrapped this in an AtomicString,
where the placeholder doesn't get removed properly.

m.group(0) is always incorrect because python-markdown modifies your regex
to include more than you specified (this is why part of the message got
duplicated).

(imported from commit 576bdf09c2b677cf4bc56484c363eb05f2110158)
2013-09-18 16:22:13 -04:00
Leo Franchi c588c7938c Allow message topic changing to propagate backwards
(imported from commit 470178ef5f7aaf09d4528c88ae6e33f17538fcaf)
2013-09-18 13:27:58 -04:00
Zev Benjamin 3d11d0e420 github: Remove the issue title from message bodies
(imported from commit d7298696011c548c26cc4ec4d13c7f1d83efd785)
2013-09-17 13:55:39 -04:00
Zev Benjamin 4996eb1e61 tests: Add Github issues tests
(imported from commit 1aa7a2c78f148f5e6ea9c7551f25374cbfe32f15)
2013-09-17 13:55:39 -04:00
Zev Benjamin 067997287e tests: Use real data for testing Github pushes
(imported from commit 83b800a8711cf045ffc7da3305227f495819a5da)
2013-09-17 12:59:18 -04:00
Zev Benjamin e5cf24b2fd Fully quote payload body contents in integrations
(imported from commit 3cb03eabbd917f878b37aa226eb3f3ecda63d5e7)
2013-09-17 12:55:57 -04:00
Steve Howell b7ef86bc34 Added /json/set_muted_topics
(imported from commit e9072cd273fabf0e072b6a5e9ed80b07142f1013)
2013-09-11 16:47:37 -04:00
Scott Feeney 1c2479c807 Fix overly-tight coupling in set_stream_color test
The test will fail if a new attribute is added to the structure that
gather_subscriptions() returns. It should only be concerned with the
subscription's color.

(imported from commit fd5bad97bbce2544e0078ee029f54d4e45da9c15)
2013-09-06 23:13:47 -04:00
Leo Franchi 252045fe2e Fix assignee in JIRA created message
(imported from commit 5ebadd14dee58c20746c80dfd99ba70a3d772ab6)
2013-09-06 11:24:03 -04:00
Zev Benjamin cb78014eef bugdown: Allow block-level block quotes
It is triggered by specifying the "language" of a code block to
"quote" or "quoted":
    Hamlet said:
    ~~~ quote
    To be or **not** to be.

    That is the question
    ~~~

(imported from commit 847a0602e335e9f2955e32d9955adf8ac8de068c)
2013-09-05 17:34:28 -04:00
Zev Benjamin 5c81148839 Switch our trac realm filter to use trac.zulip.net
(imported from commit 19163f5be01a1d9370d9ed45c55bedd670772dac)
2013-09-05 15:43:07 -04:00
Tim Abbott ce710292ca Always use external-content.zulipcdn.net for mit.edu realm.
(imported from commit 21d1101185bf52f21fcc345e791217311e9c0a13)
2013-09-05 14:35:42 -04:00
Tim Abbott b557b94d0b bugdown: Rewrite image urls to avoid mixed-content warnings.
(imported from commit fc0a41befb04f2a8aad2937a856366ac3cadb192)
2013-09-05 14:35:29 -04:00
Tim Abbott cdfc6dc10a bugdown: Fetch youtube preview images over HTTPS.
(imported from commit 9dbf3c21083cbbcf1288789f400c33ec735d4416)
2013-09-05 14:35:29 -04:00
Leo Franchi b2ddd670e4 [schema] Add backend support for per-user alert words
(imported from commit 7a9c596a010cbedbddf594c5d9c68bb9ed46d122)
2013-09-05 10:18:40 -04:00
Kevin Mehall 6b820038ad Server side tests for message editing, including subject propagation.
(imported from commit 0a68bdc337eda1b04976fb41a50a29f9f626ef25)
2013-09-04 14:51:46 -04:00
Kevin Mehall be5c90c217 Add test for realm-specific emoji
(imported from commit 2148038b300b71afe0a7517709ed0404536b69fd)
2013-09-03 11:01:01 -04:00
Scott Feeney 09ff770435 Make invite-only stream invite notifications less confusing
Previously, when added to an invite-only stream, you got notifications
like this:

   Hi there! We thought you'd like to know that Some Person just subscribed
you to the invite-only stream 'Secret stuff'

   You can see historical content on a non-invite-only stream by narrowing
to it.

Note that the second line is irrelevant and confusing in light of the
first!

This commit leaves this out, and also, to make sure I didn't mess
anything up (and because I needed to change the tests anyway), adds a
test for invite-only stream notifications.

(imported from commit 49c333629c78fc06f6d2f1ec8a627c6d38e7716a)
2013-08-29 10:27:34 -04:00
Jessica McKellar cf0f205f75 tests: check that non-subscribers can't send messages to an invite-only stream.
(imported from commit 5ee0fb49e994c4d283348d89a25606ca6d314005)
2013-08-28 16:43:40 -04:00
Jessica McKellar 2cc06aafe1 tests: don't skip message sending restriction checks!
We were using the internal do_send_message, which does not do any
security checks.

(imported from commit 14a31a39a392dd2bc6b79f4345aa80217d5356f2)
2013-08-28 16:43:40 -04:00
Tim Abbott 9e372118a9 Simplify stream access denied error message.
Otherwise, we'll report that a stream is invite-only when it isn't.

(imported from commit 441db1c84fb19bdf564251a1bd80ace23151783d)
2013-08-22 12:29:04 -04:00
Tim Abbott f5f95e5f43 [manual] Extend /api/v1/streams API endpoint.
Previously it only provided the list of all public streams; now it
allows one to specify any union of some of the following:
* all public streams
* all streams the user subscribed to

(the most relevant being the union of those two, which is what we want
for the "streams" page).

Or:
* all streams in realm (superuser only)

The manual task required is that when this is pushed to prod, we need
to also deploy the new sync-public-streams version to zmirror.

(imported from commit 27848b8bd136e2777f399b7d05b2fdcec35e4e21)
2013-08-22 12:29:04 -04:00
Kevin Mehall 00fc004be2 Server-side check for empty (or all-whitespace) messages.
(imported from commit 49054dd7a8050705f579b4b7a1bda7eb7bfc25be)
2013-08-21 10:37:36 -04:00
Tim Abbott 517c98d56f add_subscriptions: add authorization_errors_fatal parameter.
For syncing streams between Zephyr and Zulip, we need to be able to
have the API client send the server a long list of streams, some of
which might be invite-only, and add the ones that it can add and not
the ones it cannot without a bunch of annoying round trips dropping
individual streams one is not authorized to one by one.  This argument
makes that possible.

We might find other applications as well.

(imported from commit 9236d185897c42218ab6cac3d8f3ddcb1bbc94e9)
2013-08-20 14:57:15 -04:00
Kevin Mehall 8a741ed3d4 Ensure that the pointer is valid before saving it.
Trac #1716

Fix the tests to use real message IDs for the pointer. One helper didn't
use the pointer ID it was passed at all, so the test didn't actually test
what it was supposed to before.

(imported from commit 457bcdb04a6c3873e224b68cd9d79c9a26612010)
2013-08-19 16:46:52 -04:00
Jessica McKellar f530e3b930 Display the email address for a stream in its stream page detail.
For now, only show it on staging.

(imported from commit fd07fad1c34578d8ddc2cddd1bb6bdcb72f354de)
2013-08-13 14:28:47 -04:00
Zev Benjamin 6843cc4479 Add test for zephyr mirror CC de-duplication
(imported from commit 1452c67818f8aa577a95a219fc6b86e9690033eb)
2013-08-13 13:49:09 -04:00
Luke Faraone 8c35db91c0 Unit testing for MIT user name calculation / mailing list protection.
(imported from commit ae7d4c0a1fea0b0887861a71828eae7c3022ec6f)
2013-08-13 10:21:45 -07:00
Steve Howell 164215bc78 Support API for changing user's API key.
(imported from commit b9f5594cf809965c996115c8420a359820dad3ff)
2013-08-08 13:03:09 -04:00
Tim Abbott 7b9305b06f Rename Django project to zproject.
This includes a hack to preserve humbug/backends.py as a symlink, so
that we don't need to regenerate all our old sessions.

(imported from commit b7918988b31c71ec01bbdc270db7017d4069221d)
2013-08-07 11:04:03 -04:00
Tim Abbott d59ba39518 Rename humbug_finish to zulip_finish.
(imported from commit a73ef9302d16a4068c7d050d4882d2eba699488d)
2013-08-07 10:00:08 -04:00
Tim Abbott 661ca38b58 Change humbug => zulip in some local variables.
(imported from commit 88caa4a87ea0fd269ab741645c124c5d07d69c0a)
2013-08-07 10:00:08 -04:00
Tim Abbott 1f1af26048 Change Humbug => Zulip in text/comments.
(imported from commit 2f9d73431ae40e1b9e9e11bc2f4f62f566ae758a)
2013-08-07 10:00:07 -04:00
Tim Abbott e05979d4c4 Rename humbug_realm to zulip_realm.
(imported from commit b36dcc7e5240159357a0f30036394a2af9b85844)
2013-08-07 10:00:07 -04:00
Steve Howell 9f9c7c3e3b Loosen up emoji regex. (i.e. More emojis! 👍)
Now parsed: 🍺,🍺;🍺!

 If \w characters surround :foo:, we still say it's NOT an
 emoji, but we used to do this for \S characters, so it's loosened up.

(imported from commit 49b33d2f0ffdcfde8947ae411a4addcf4c24af9c)
2013-08-06 18:20:30 -04:00
Tim Abbott e111a2f9a5 [manual] Rename Django app from zephyr to zerver.
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).

At the time it is deployed, we need to make a few changes directly in
the database:

(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';

(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
2013-08-06 07:39:36 -04:00