This includes:
* Merging the pull request and issue subject creation functions
* Factoring out pull request and issue content creation
(imported from commit 9cf90e999482a1998431e6483788522101607167)
Some bots created by us do not have owners. Don't try to send a
message to the nonexistent owner.
(imported from commit ab952eccd7d6c4728e9477a106142214b5c81ca9)
Instead just rely on the 2-minute delay in the management command to
batch conversations.
We've had people report being confused or thinking the feature was
broken when they didn't get e-mails because of our rate-limiting, so
let's see if this is not too overwhelming.
(imported from commit 706ddb07b906b5c2edea1159c04acc2ee6f06e29)
Warn inside these functions when you get data on streams that you
are not subscribed to:
add_subscriber
remove_subscriber
user_is_subscribed
The back end should be smart enough not to spam us with subscriber
info that we don't care about.
(imported from commit b27644be2abc37c11ddff884ef392ea208bd1bd3)
Don't send peer_add notifications to users who are already
getting add notifications, because they will already know
about subscribers.
(imported from commit 726b54ae0e30b71440b17d9c51b026872ea96218)
It only grabs the user_profile_id column now. This leads to a
speedup of about 16x between grabbing large ORM objects vs.
small 1-column dictionaries.
(imported from commit 95150bff3fdcbe250b04f014062224af42a6644f)
Splitting out notify_peers() will give us flexibility for cleaning
up how we notify peers for bulk adds.
(imported from commit e108fa2c432cc1fe54d788c58c82c983e0f2394e)
If you expand subscribers on your settings page, you will now see
a query like this in your postgres logs:
SELECT "zerver_userprofile"."email"
FROM "zerver_subscription" INNER JOIN "zerver_recipient" ON ("zerver_subscription"."recipient_id" = "zerver_recipient"."id") INNER JOIN "zerver_userprofile" ON ("zerver_subscription"."user_profile_id" = "zerver_userprofile"."id") WHERE ("zerver_recipient"."type" = 2 AND "zerver_subscription"."active" = true AND "zerver_recipient"."type_id" = 40 AND "zerver_userprofile"."is_active" = true )
The join's still complicated, but the list of fields is one instead of 40+.
(imported from commit 48de1f888193a4d23fcea52d0b633d134e4a3ff7)
get_subscribers_backend() now calls the new get_subscriber_emails()
function, which just queries the email field:
"zerver_userprofile"."email"
...instead of querying about 40 fields that it never uses.
I was able to verify the query slimming by watching my postgres server log.
Also, you can verify that the ORM does roughly 16x less work using values():
>>> def f(): return [sub.user_profile.email for sub in list(Subscription.objects.all().select_related())]
...
>>> def g(): return [row['user_profile__email'] for row in list(Subscription.objects.all().values('user_profile__email'))]
...
>>> def timeit(func): t = time.time(); func(); return time.time() - t
...
>>> timeit(f)
0.045198917388916016
>>> timeit(g)
0.002752065658569336
(imported from commit a69f690a96d076b323fdfc2f4821b0548bdfac7f)
LinkPattern returned a string which contained a placeholder if the URL was
considered invalid. AtomicLinkPattern wrapped this in an AtomicString,
where the placeholder doesn't get removed properly.
m.group(0) is always incorrect because python-markdown modifies your regex
to include more than you specified (this is why part of the message got
duplicated).
(imported from commit 576bdf09c2b677cf4bc56484c363eb05f2110158)
* Remove the action from the topic and add the issue title
* Only show the issue body on open or reopen
(imported from commit f08eb40f36122d2498fe0c36a69df9e606296ff3)
We have to read the data anyway, and we don't have a convenient file
handle for uploads from attachments sent through the e-mail gateway.
(imported from commit 86260a4eaceef85c82707929a80558e11dc54ef6)
The get_status_dict_by_realm helper gets called whenever our
realm user_presences cache expires, and it used to query these fields:
"zerver_userpresence"."id", "zerver_userpresence"."user_profile_id", "zerver_userpresence"."client_id", "zerver_userpresence"."timestamp", "zerver_userpresence"."status", "zerver_userprofile"."id", "zerver_userprofile"."password", "zerver_userprofile"."last_login", "zerver_userprofile"."is_superuser", "zerver_userprofile"."email", "zerver_userprofile"."is_staff", "zerver_userprofile"."is_active", "zerver_userprofile"."is_bot", "zerver_userprofile"."date_joined", "zerver_userprofile"."bot_owner_id", "zerver_userprofile"."full_name", "zerver_userprofile"."short_name", "zerver_userprofile"."pointer", "zerver_userprofile"."last_pointer_updater", "zerver_userprofile"."realm_id", "zerver_userprofile"."api_key", "zerver_userprofile"."enable_desktop_notifications", "zerver_userprofile"."enable_sounds", "zerver_userprofile"."enter_sends", "zerver_userprofile"."enable_offline_email_notifications", "zerver_userprofile"."last_reminder", "zerver_userprofile"."rate_limits", "zerver_userprofile"."avatar_source", "zerver_userprofile"."tutorial_status", "zerver_userprofile"."onboarding_steps", "zerver_userprofile"."invites_granted", "zerver_userprofile"."invites_used", "zerver_userprofile"."alert_words", "zerver_userprofile"."muted_topics", "zerver_client"."id", "zerver_client"."name"
Now it queries just the fields it needs:
"zerver_client"."name", "zerver_userpresence"."status", "zerver_userpresence"."timestamp", "zerver_userprofile"."email" FROM "zerver_userpresence"
Also, get_status_dict_by_realm is now namespaced under UserPresence as a static method.
(imported from commit be1266844b6bd28b6c615594796713c026a850a1)
This function gets user presence information, which changes rapidly
and requires a pretty simple query.
(imported from commit f9b9f0f22277335c76eb4371930a4fff2758a240)
The do_send_messages() populates the user_presences data structure
for process_new_message(), so that Tornado code never needs to hit
the database or memcached to get the user presence info.
(imported from commit 194aeaead8fa712297a2ee8aff5aa773b92f1207)
This reduces the number of memcached calls we make in our time-
slice-limited tornado event handler.
(imported from commit 8903ce4ac754ba82d57e04d1b0356be7533edee2)
We create a blueslip error for undefined keys in Dict. This led
to a straightforward change in the unit tests for Dict. For the
unread test, to avoid the blueslip error, we had to be more specific
in setting up a user in one place, but this reduced our coverage,
leading to another small test being added.
(imported from commit 33e14795500d9283de2a7c03c4c58aec11cea4b8)
The exceptions were cryptic before, and they were inconsistent with
the fold_case: false behavior.
(imported from commit a40704d1a22bcdc60d91be832ee3c81eb416c6dd)
There was nothing to ensure that the changes resulting from scrolling
happened before the unread counts were checked. We already had a long
wait there; might as well do those checks after it to ensure that the
DOM is updated.
(imported from commit 0d4014ae6a74dd684521fecabefc4bf79015f842)
These engagement data will be useful both for making pretty graphs of
how addicted our users are as well as for allowing us to check whether
a new deployment is actually using the product or not.
This measures "number of minutes during which each user had checked
the app within the previous 15 minutes". It should correctly not
count server-initiated reloads.
It's possible that we should use something less aggressive than
mousemove; I'm a little torn on that because you really can check the
app for new messages without doing anything active.
This is somewhat tested but there are a few outstanding issues:
* Mobile apps don't report these data. It should be as easy as having
them send in update_active_status queries with new_user_input=true.
* The semantics of this should be better documented (e.g. the
management script should print out the spec above)x.
(imported from commit ec8b2dc96b180e1951df00490707ae916887178e)
The test will fail if a new attribute is added to the structure that
gather_subscriptions() returns. It should only be concerned with the
subscription's color.
(imported from commit fd5bad97bbce2544e0078ee029f54d4e45da9c15)
We found that since bugdown processes are threaded, the cost of
doing a db query in a markdown processor is quite high---each
thread must start up a new db connection including a SSL handshake
etc. We should strive to keep our rendering pipeline free of mandatory
DB queries.
(imported from commit 555066bd03da6c681b74ce6137acc264eb41c55d)
It is triggered by specifying the "language" of a code block to
"quote" or "quoted":
Hamlet said:
~~~ quote
To be or **not** to be.
That is the question
~~~
(imported from commit 847a0602e335e9f2955e32d9955adf8ac8de068c)
The new version is more accurate (doesn't rely on UserMessage IDs
being sorted, which they aren't necessarily) and simpler.
(imported from commit 671dd89dc8881ae2dcb8d0e804fd65458e074a29)
This will allow mail with an implicit destination (mailing lists, complex
forwarding) to be received by our system.
See http://support.google.com/a/bin/answer.py?hl=en&answer=2368151 for
documentation of Google's behaviour that adds this header.
(imported from commit f8fd500e3c27e12af5941c63c91d5c796a2cd24a)
Previously the email gateway had to be the only address in a recipient
field or we'd mis-parse the recipient.
This commit also makes the mirror correctly handle addresses of the
form "Jessica McKellar <jesstess@zulip.com>".
(imported from commit 7435f2b59b8f47dc599cc869f64597a730af7d12)
The mirror will use INBOX when deployed and Test locally. Send an
e-mail to the Test mailbox by including the word "localhost" in the
subject; a GMail filter will place it in Test on receipt.
(imported from commit bacf9a9554c8c5e1f3ec8497761edf2c15d3745d)
Previously, when added to an invite-only stream, you got notifications
like this:
Hi there! We thought you'd like to know that Some Person just subscribed
you to the invite-only stream 'Secret stuff'
You can see historical content on a non-invite-only stream by narrowing
to it.
Note that the second line is irrelevant and confusing in light of the
first!
This commit leaves this out, and also, to make sure I didn't mess
anything up (and because I needed to change the tests anyway), adds a
test for invite-only stream notifications.
(imported from commit 49c333629c78fc06f6d2f1ec8a627c6d38e7716a)
It was getting hard to follow and is going to get more complicated
with a new super user check in a later commit.
(imported from commit 8d5cfa960824519d87ce0f09aab3a120ba9ef357)
An important part of this is updating the various caches that cache
the display_recipient.
(imported from commit 888bf54fd205516cf31a25ba3f4e45ad11bbd4d5)
This cache was created to make recipient lookups within a single
request (e.g. when fetching old messages) cheaper. To support stream
name changes, we need to invalidate this cache on every request so
that users get a consistent view of the name change.
(imported from commit 801051b9f6a108c1f50be7eca9a1242d661919b1)
This shows up when you're not running a Zephyr mirroring bot and lets
you use Webathena to have us run it. Obviously needs more docs.
Current problems include:
* supervisorctl reload ends up recreating /var/run/supervisor.sock
with the wrong permissions, so it only works once in a row before
you need to chmod that.
* /etc/supervisor/conf.d needs to be humbug-writeable; this is a clear
local root vulnerability
* This uses SSH and thus is kinda slow.
(imported from commit 7029979615ffd50b10f126ce2cf9a85a5eefd7a2)
Before this it was [deleted]. Using parens is consistent with how we put
in (no topic) if you don't specify a topic.
(imported from commit 931c06a1096cf7b0d226336cbe82535abd2e6032)
This is experimental, for staging only. There might be a better
way to model this than dueling force_expand/force_collapse flags,
but it works for now. The code in collapse_recipient_group()
could also be DRYed up relative to expand_summary_row().
(imported from commit 107151d1ecd640970fb7700d41278a003bd1abaa)
I believe the test data to be incorrect, but we'll find out when
we've captured some real data.
(imported from commit ae32af0436b6273d0b2a60b2c55047aace6adcad)
I was saying bar.d in places where I wasn't really specifically
testing the .d feature, and it was distracting and just an
unintentional consequence of copy/paste.
(imported from commit 7b137b28cb33c72b83f02fe1d2961c5c6accc263)
This change will allow us to test the muting feature on
staging. Any topic named "muted" will automatically be
muted. You can also mute any other topic on the console:
muting.mute_topic('devel', 'ios');
current_msg_list.rerender();
More UI around this experiment will be coming soon, as well
as support for muting entire streams.
The muting module keeps track of which topics are muted, but a
user can expand muted messages, and once that happens, the
messages are marked with the "force_expand" flag that gets
persisted to the back end.
Muted messages are rendered in similar fashion to the summarized
rows, and as part of unifying some of that code, we have
made it so that expanding a summarized section doesn't remove
individual flags related to summaries; instead, the messages
get the force_expand flag set.
(imported from commit acee4190e63813d46850415c41ff8ebfae4a6953)
I regressed this recently, thinking that all our operators are
strings, but I forgot about the "near:" operator used in the
"Narrow messages around this time" feature. The user facing
symptom was that the search bar showed up empty instead
of saying near:50, which might actually be the better
behavior, but it certainly was not intentional. :)
(imported from commit fcb93cecbe9a052bb9bc1af7fcac5aecaba5aafb)
I'm trying to move well-isolated methods out of narrow.js, so that
narrow.js is more strongly focused on UI/ajax interactions and
big, heavy lifting stuff. The logical home for parse/unparse
seemed to be Filter, and they brought along two private methods
with them. The big code moves involved trivial follow ups
like s/exports/Filter/.
(imported from commit ace0fe5aa1c7abce0334d079ba9eb8d9a57bd10f)
This is hard to break up into separate commits; sorry about that.
Before this commit we had 3 tests:
1. Claims to check that an unread count was 3, but actually doesn't.
2. Checks that scrolling down causes the left-sidebar stream unread
count to decrease.
3. Claims to check that unread counts are correct, but actually doesn't.
From talking to Leo, it seems that he originally tried to actually do
what tests 1 and 3 claim to do, but found it too fragile to check an
exact unread count because of font sizes, layout, etc.
We now have 4 tests. For each of the stream sidebar and user sidebar, we
test that:
1. Scrolling down causes the unread count to decrease
2. Logging out and back in again leaves the unread count unchanged
I've removed the two bogus tests and some other code that didn't seem to
serve a purpose.
(imported from commit 9f8e4b521e2765099510426d0b7e2960885e6f19)
You used to have to call casper.test.done(N) where N was the number of
tests run. This is no longer required and is deprecated in CasperJS 1.1.
(imported from commit 0de9ecb1930cbce416fa02c24a882e926cdc8e87)
Have ui.set_presence_list() only touch the presence list.
Before this change, it was calling update_unread_counts(), which
has a bunch of side effects unrelated to the presence list.
(imported from commit 690f754d78874a03fa36f8ff8765d5a63e431d28)
Previously it only provided the list of all public streams; now it
allows one to specify any union of some of the following:
* all public streams
* all streams the user subscribed to
(the most relevant being the union of those two, which is what we want
for the "streams" page).
Or:
* all streams in realm (superuser only)
The manual task required is that when this is pushed to prod, we need
to also deploy the new sync-public-streams version to zmirror.
(imported from commit 27848b8bd136e2777f399b7d05b2fdcec35e4e21)
Github's documentation is a little unclear on what data is actually
sent and in some cases conflicts with the examples in the
github-services repo. The hope here is to capture a good sample of
different events to develop against and then use it in testing.
(imported from commit 356935894b87ca21591f5682c15f9ed27289a5b6)
Zev Benjamin
Waseem Daher
Steve Howell
Kevin Mehall
Jessica McKellar
Leo Franchi
Tim Abbott
Scott Feeney
acrefoot
acrefoot
Luke Faraone
Allen Rabinovich