We also now separate out the times for the socket overhead, the
request service time, and the queuing delays.
(imported from commit e1683f7f28b968b86ebb701b0ac29b00ac6d67c3)
One quirk here is that the Request object is built in the
message_sender worker, not Tornado. This means that the request time
only counts time taken for the actual sending and does not account
for socket overhead. For this reason, I've left the fake logging in
for now so we can compare the two times.
(imported from commit b0c60a3017527a328cadf11ba68166e59cf23ddf)
If a user types "/me runs to the store", we put "runs to the store"
in bold after their name.
(imported from commit fbc11e99244e1c8fa1c03e4753e706957fcd449e)
The Freshdesk API is bonkers, but we do the best we can with it to
support notifications on ticket creation and ticket updates.
(imported from commit 2023622b274ef83f4e1544d0df286fe2e68581b3)
Add a method that lets us know what percentage of a huddle is
present. (We can use this later to set the opacity of huddles
in the UI.)
(imported from commit 8a2383951807d7bfbf9d730a8980d977cf23b379)
This logging is kinda excessive since it adds like 4 log lines per
recipient, so I expect we'll end up reverting it once we've debugged
the proximal issue.
(imported from commit 5e6ab3e230f32b65ad9cf0d95f20ffbc0fe7397e)
This will hopefully help with the send dialog being stuck on
"sending" as well as allowing us to not show errors to the user on
reconnect.
(imported from commit 31ee889853f348e486863073dc130cdfb4e1338d)
Clients can only have one connection at a time, anyway, so we can
just keep track of a client id, instead. This makes reconnections
easier.
It's a little funny to use queue ids for the client id, but we know
they should exist by the time the client is connecting and they are
guaranteed to already be unique and authenticatable. We will also
eventually be integrating the event system and the socket code closer
anyway.
(imported from commit 1f60e06fb16d31d6c121deafd493fb304d19a6c2)
This reverts commit c10d9c1a0d23891acce88bf8d79866c08cb75681.
This reverts commit 9259a246946cd968a8725c38ff5ef2d4b4793717.
(imported from commit 50e9e0136c2487cc63d75ae2b78df0c70a1b0be1)
This is the amount of time between when it is sent, and when it is
rendered into the user's home view.
(imported from commit 468c28e77ba16c7256c359e90ab5aacf9d497585)
The simulated people_dict in the activity.js test was not
matching the production code, and going forward, we'll want to
share the people_dict setup for all of our tests.
(imported from commit fc21a02216b9422130b9fe9c11bcf80590612844)
Activity.js now has the capability to track huddles that
come through in loaded messages and return them in reverse
chronological order by their most recent message. Right
now this only connected to a unit test, not any production
code.
(imported from commit 59957086fa2e454e5711472df091f178217aed2b)
The main Activity page counts users as active if they have either
sent a message or updated a pointer. In the unlikely event that
somebody sent a message but never updated their pointer, we were
undercounting them, if they went through send_messages_backend.
(imported from commit 5f112be87a239980c38a18c13f9cd68e90d2e905)
This should help with determining the prevalence of slow sends as
experienced by users.
(imported from commit f00797679315c928af3c87ad8fdf0112f1dfa900)
The "desktop" counts aggregate all desktop clients, but on the
Clients tab, we are only interested in specific versions.
(imported from commit eea2d8da584a6fa32fa1f3a2bae71ef5daaba738)
We only needed a transaction here to workaround problems associated
with not having database-level autocommit.
(imported from commit 240ba05a4a4a846a7ff62e6e59e403ab0d78ab11)
I'm pretty sure this code was a hack to workaround the fact that
without database-level autocommit, .get_or_create is racy.
(imported from commit bfdd476a354aee439feeaa0030ae59cc43c165b9)
This was a precursor to UserMessage.flags.read that never got used
because we decided to use django-bitfield.
(imported from commit 868754723c07ee9b85ae951aee785e571ccfef97)
I added the @wraps decorator, and I point request._query at the
function name to override the URL, since some of the internal
URLs have realm names and domain names in them. I basically
prefer the function names in most cases, so I just made this
automatic for zulip_internal functions, rather than having to
remember to address URL-vs.-function-name for every new endpoint.
(imported from commit 5583607f395be4dfae0bac31e1cdbffdf51fb3e8)
This report will eventually replace the per-realm report that is
now accessible through /activity. In order not to disrupt Waseem,
I'm leaving the old reports around until we've polished the new
ones.
The old report does 24 different queries to get per-realm user data.
The new approach gets all the data at once, and it slices and dices
the data in Python to accomodate our slightly quirky data model.
On localhost, this is a typical query:
LOG: duration: 5.668 ms statement: SELECT "zerver_useractivity"."id", "zerver_useractivity"."user_profile_id", "zerver_useractivity"."client_id", "zerver_useractivity"."query", "zerver_useractivity"."count", "zerver_useractivity"."last_visit", "zerver_userprofile"."id", "zerver_userprofile"."email", "zerver_client"."id", "zerver_client"."name" FROM "zerver_useractivity" INNER JOIN "zerver_userprofile" ON ("zerver_useractivity"."user_profile_id" = "zerver_userprofile"."id") INNER JOIN "zerver_realm" ON ("zerver_userprofile"."realm_id" = "zerver_realm"."id") INNER JOIN "zerver_client" ON ("zerver_useractivity"."client_id" = "zerver_client"."id") WHERE "zerver_realm"."domain" = 'zulip.com' ORDER BY "zerver_userprofile"."email" ASC, "zerver_useractivity"."last_visit" DESC
(imported from commit 0c71f4e32fe5a40f4496749dc29ad3463868d55e)
This page shows aggregate activity for a user on various
clients. This allows Waseem to troubleshoot things like users
switching between website and desktop, etc.
This particular page probably won't be used too much, but some of the
logic is gonna be reused in the per-realm activity pages.
(imported from commit b8c1fad5bfa45daab40954f92319f6f89a3fa433)
If you don't have a cookie or basic auth and the request looks like
a top-level page in the browser, redirect to the login page.
(imported from commit fc1bcb1080591522bd1b694664255f7049a5d443)
The register_json_consumer() function now expects its callback
function to accept a single argument, which is the payload, as
none of the callbacks cared about channel, method, and properties.
This change breaks down as follows:
* A couple test stubs and subclasses were simplified.
* All the consume() and consume_wrapper() functions in
queue_processors.py were simplified.
* Two callbacks via runtornado.py were simplified. One
of the callbacks was socket.respond_send_message, which
had an additional caller, i.e. not register_json_consumer()
calling back to it, and the caller was simplified not
to pass None for the three removed arguments.
(imported from commit 792316e20be619458dd5036745233f37e6ffcf43)
Looking at the historical data, fewer than 50% of active users have
completed the checklist, which means that it is just persistent
clutter. We also have other better ways of encouraging people to send
traffic and get the apps now.
This commit removes both the frontend UI and backend work but leaves
the db row for now for the historical data.
(imported from commit e8f5780be37bbc75f794fb118e4dd41d8811f2bf)
We now ensure `create_realm` adds you to a default deployment and that
`create_deployment` removes the old deployment association when
performed.
(imported from commit 5b94fb07b8e11332765b057dc640a5ed873ec99e)
Subclasses of QueueProcessingWorker that don't override start() will
have their consume() functions wrapped by consume_wrapper(), which
will catch exceptions and log data from troublesome events to a log
file.
We need to do a puppet apply to create /var/log/zulip/queue_error.
(imported from commit 3bd7751da5fdef449eeec3f7dd29977df11e2b9c)
The DigestWorker isn't do any batching of events, so it can
be implemented with a simple consume() handler.
(imported from commit 0560eaf1a6e510adf97448fbe8933d9903d30016)
We were using Gravatar for user avatars, but now users can
upload their avatars directly to Zulip, and we will store
their avatar for them. This removes the old Gravatar-related
interface and polling code.
This commit does not attempt to update the avatars in
messages that have already been loaded, either for the user
making the change or other users.
(imported from commit 301dc48f96f83de0136c93de57055638c79e0961)
The "Your Account" and "Notifications" boxes on the Settings
page each had their own border and their own "Save changes"
button, but they were within the same form and sending to the
same back end point.
This commit creates a separate form and endpoint for each
of the two boxes.
(imported from commit 04d4d16938f20749a18d2c6887da3ed3cf21ef74)
This prevents the ones with external side-effects (like sending real
email) from being accidentally run in dev instances.
(imported from commit 6d9861d721abb29136bfff974de01a9264051436)
Before we were removing items individually from the queue. We now
directly use RabbitMQ's queue purging mechanism.
(imported from commit 62ab52c724c5a221b4c81a967154a4046a579f84)
This commit must be simultaneously deployed on both staging and
prod0. It also requires completely taking down the app.
To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0
* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0
* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
actually require a restart)
* run tools/migrate-db on postgres0 as root
* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1
* do a puppet apply on nagios
(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
This will allow us to redirect clients to the correct local site.
To apply this migration, just run:
python manage.py migrate zilencer 0002
(imported from commit 7bd39b5f035145b6b52e1b2cb2ad5f6720d598ce)
Zev Benjamin
Leo Franchi
Steve Howell
Kevin Mehall
Jessica McKellar
Tim Abbott
Luke Faraone
Jessica McKellar
Waseem Daher