Commit Graph

103 Commits

Author SHA1 Message Date
Steve Howell 95dc0894d8 Add /realm_activity report.
This report will eventually replace the per-realm report that is
now accessible through /activity.  In order not to disrupt Waseem,
I'm leaving the old reports around until we've polished the new
ones.

The old report does 24 different queries to get per-realm user data.
The new approach gets all the data at once, and it slices and dices
the data in Python to accomodate our slightly quirky data model.

On localhost, this is a typical query:

LOG:  duration: 5.668 ms  statement: SELECT "zerver_useractivity"."id", "zerver_useractivity"."user_profile_id", "zerver_useractivity"."client_id", "zerver_useractivity"."query", "zerver_useractivity"."count", "zerver_useractivity"."last_visit", "zerver_userprofile"."id", "zerver_userprofile"."email", "zerver_client"."id", "zerver_client"."name" FROM "zerver_useractivity" INNER JOIN "zerver_userprofile" ON ("zerver_useractivity"."user_profile_id" = "zerver_userprofile"."id") INNER JOIN "zerver_realm" ON ("zerver_userprofile"."realm_id" = "zerver_realm"."id") INNER JOIN "zerver_client" ON ("zerver_useractivity"."client_id" = "zerver_client"."id") WHERE "zerver_realm"."domain" = 'zulip.com'  ORDER BY "zerver_userprofile"."email" ASC, "zerver_useractivity"."last_visit" DESC

(imported from commit 0c71f4e32fe5a40f4496749dc29ad3463868d55e)
2013-11-01 11:46:26 -04:00
Steve Howell 3577ba7585 Add /user_activity page.
This page shows aggregate activity for a user on various
clients.  This allows Waseem to troubleshoot things like users
switching between website and desktop, etc.

This particular page probably won't be used too much, but some of the
logic is gonna be reused in the per-realm activity pages.

(imported from commit b8c1fad5bfa45daab40954f92319f6f89a3fa433)
2013-11-01 11:46:26 -04:00
Tim Abbott 21663601b4 Remove obsolete zulip.com/what-is-humbug page.
(imported from commit 0e2f317cf0bf262eb113caa7daab7d71ce564e9e)
2013-10-31 15:29:49 -04:00
Tim Abbott 0e35424a7c Rename humbug => zulip in some log file names.
(imported from commit 0bb5284566861e5a2f260cc66dcb53ebac0d7b97)
2013-10-31 15:29:49 -04:00
Jessica McKellar f1e90086f5 Remove the onboarding checklist.
Looking at the historical data, fewer than 50% of active users have
completed the checklist, which means that it is just persistent
clutter. We also have other better ways of encouraging people to send
traffic and get the apps now.

This commit removes both the frontend UI and backend work but leaves
the db row for now for the historical data.

(imported from commit e8f5780be37bbc75f794fb118e4dd41d8811f2bf)
2013-10-31 11:34:15 -04:00
Steve Howell d7d1888ce0 Add TestWorker queue processor.
TestWorker is for troubleshooting, and it simply appends lines to a file
in /tmp.

(imported from commit 96b646a193a5474f3222b41725742e359b5301ec)
2013-10-30 16:47:16 -04:00
Zev Benjamin 57bef07832 [manual] Move /etc/humbug-* files into /etc/zulip
The moved files are:
humbug-server
humbug-local
humbug-machinetype

Their new names are their old names with 'humbug-' removed.

zulip-puppet-apply must be run before this commit is deployed

(imported from commit f4eb523244d3409b5809c279301225d3fdf0c230)
2013-10-30 15:42:25 -04:00
Jessica McKellar 8e838f4db3 Add a first pass at Stash integration via their webhook mechanism.
(imported from commit 6173379f72aeef318550b35e51d4aa1fd69997ed)
2013-10-29 22:40:42 -04:00
Steve Howell 884e602185 [puppet] Handle exceptions from queue workers.
Subclasses of QueueProcessingWorker that don't override start() will
have their consume() functions wrapped by consume_wrapper(), which
will catch exceptions and log data from troublesome events to a log
file.

We need to do a puppet apply to create /var/log/zulip/queue_error.

(imported from commit 3bd7751da5fdef449eeec3f7dd29977df11e2b9c)
2013-10-29 17:45:31 -04:00
Leo Franchi 8e05f76511 Only enable analytics and dropbox on non-LOCAL_SERVER
(imported from commit 1ba877550b3afde51bec6f344762ea998800c5b6)
2013-10-29 17:33:36 -04:00
Kevin Mehall 06590f4332 Upload to local filesystem on local server.
(imported from commit 6c38a8cae721b2e7a0863470692cc56425006ecb)
2013-10-29 16:19:04 -04:00
Kevin Mehall 209ff75af8 Remove duplicate MANDRILL_API_KEY setting
(imported from commit c71928b5bfd71e9fcda1f5d57e9c7b99361ce070)
2013-10-29 15:47:13 -04:00
Steve Howell 7cf66b5bb8 Allow users to upload avatars on the Settings page.
We were using Gravatar for user avatars, but now users can
upload their avatars directly to Zulip, and we will store
their avatar for them.  This removes the old Gravatar-related
interface and polling code.

This commit does not attempt to update the avatars in
messages that have already been loaded, either for the user
making the change or other users.

(imported from commit 301dc48f96f83de0136c93de57055638c79e0961)
2013-10-29 10:59:41 -04:00
Steve Howell 68942a8f3a Properly split up "Your Account" and "Notifications".
The "Your Account" and "Notifications" boxes on the Settings
page each had their own border and their own "Save changes"
button, but they were within the same form and sending to the
same back end point.

This commit creates a separate form and endpoint for each
of the two boxes.

(imported from commit 04d4d16938f20749a18d2c6887da3ed3cf21ef74)
2013-10-29 10:59:41 -04:00
Tim Abbott 56e9ad230e [manual] Move our deployment scripts to scripts/.
This will require updating the post-receive code on git.zulip.net to
work.

(imported from commit 2e51fa2d7b891c1138d3f22ae534cfb8a6cf174c)
2013-10-28 10:54:48 -04:00
Tim Abbott 0a44ba2a28 Move servers/configure-rabbitmq to scripts/setup/.
(imported from commit 2b4d5ccb88675447ae744fb985246211deef7486)
2013-10-28 10:54:48 -04:00
Steve Howell 09979252dd Add Steve's laptop to USING_RABBITMQ list.
(imported from commit 3527edd27e3f386eb8b668fe8ee7e3875282ead8)
2013-10-27 11:27:27 -04:00
Zev Benjamin cdcbe61cb0 [manual] Change references to the humbug_test user, schema, and database to zulip_test
This requires no changes in production, but is tagged as manual to
remind developers that they need to edit and run the tools/migrate-db
script to fix up their local database instances.

(imported from commit fbf764fb61592ef994d6d2ad56edad65ff01f14b)
2013-10-26 04:16:28 -04:00
Zev Benjamin 7b6f6823a5 [manual] Change references to the humbug user, schema, and database to zulip
This commit must be simultaneously deployed on both staging and
prod0.  It also requires completely taking down the app.

To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0

* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0

* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
  actually require a restart)
* run tools/migrate-db on postgres0 as root

* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1

* do a puppet apply on nagios

(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
2013-10-26 04:16:27 -04:00
Waseem Daher 66f48288b4 Rename LOCALSERVER -> LOCAL_SERVER.
(imported from commit b3abdd10d54d2ad7a9c463af9a291d2e2127707f)
2013-10-25 17:37:06 -04:00
Leo Franchi 8e21d70afe Disable statsd on local server
(imported from commit cac2c95c2de18b613a0900fe1c04fd97963b9350)
2013-10-25 16:37:29 -04:00
Leo Franchi c12a97577e Rework local_settings_template.py to be clear about what the user needs to do
(imported from commit f69ea0d57e885509abc70dc4f2bee6e36169691f)
2013-10-25 16:35:27 -04:00
Luke Faraone 238d19e1f1 Include a rest_dispatch in tornadoviews and call it from urls.py
(imported from commit aefe66b44be60b39d0f6bb3076502e964f1ed6bd)
2013-10-25 14:13:31 -04:00
Luke Faraone e552307511 Send feedback to a queue to be forwarded to staging.
(imported from commit 4a9a1bfc6c95763a816263a726cc61b3ca90bf15)
2013-10-25 14:13:30 -04:00
Luke Faraone 81d7dd1fda [schema] Support for authenticating Deployments via the API.
Here we introduce a new Django app, zilencer. The intent is to not have
this app enabled on LOCALSERVER instances, and for it to grow to include
all the functionality we want to have in our central server that isn't
relevant for local deployments.

Currently we have to modify functions in zerver/* to match; in the
future, it would be cool to have the relevant shared code broken out
into a separate library.

This commit inclues both the migration to create the models as well as a
data migration that (for non-LOCALSERVER) creates a single default
Deployment for zulip.com.

To apply this migration to your system, run:
   ./manage.py migrate zilencer

(imported from commit 86d5497ac120e03fa7f298a9cc08b192d5939b43)
2013-10-25 14:13:30 -04:00
Leo Franchi 0bc5b13648 Fix Zulip username of email gateway bot
(imported from commit f24c5fd3fc5888bf7dc4d9501723777ad8447704)
2013-10-25 11:51:13 -04:00
Leo Franchi a9b9c0ec7b Add Email Mirror default settings and error out with readable warning if not configured
(imported from commit adac321d09561b51277ec1e314a1d480dc01e0c8)
2013-10-25 11:51:13 -04:00
Leo Franchi 2d276179d0 Refactor email-mirror to handle running on any machine
(imported from commit 2971449ceaacb564770e66874fc095f77e68d445)
2013-10-25 11:37:34 -04:00
Kevin Mehall 3bad5acc71 Fix upload auth redirect for user_activity.
Use rest_dispatch for upload auth redirect so it doesn't send the
long URL to user_activity.

(imported from commit ab327bbd529412e43eee6d109f8550180544dbbb)
2013-10-24 17:33:12 -04:00
Kevin Mehall 0a3a22cb3d Support authenticated upload URLs.
Trac #1734

This is implemented by bouncing uploaded file links through a view
that checks authentication and redirects to an expiring S3 URL.

This makes file uploads return a domain-relative URI. The client converts
this to an absolute URI when it's in the composebox, then back to relative
when it's submitted to the server.

We need the relative URI because the same message may be viewed across
{staging,www,zephyr}.zulip.com, which have different cookies.

(imported from commit 33acb2abaa3002325f389d5198fb20ee1b30f5fa)
2013-10-24 17:01:06 -04:00
Zev Benjamin c13114c7e3 [manual] Change rabbitmq user to 'zulip'
Before deploying this commit, the following commands must be run:
  # rabbitmqctl add_user zulip $(tools/get-django-setting RABBITMQ_PASSWORD)
  # rabbitmqctl set_user_tags zulip administrator
  # rabbitmqctl set_permissions -p / zulip '.*' '.*' '.*'

(imported from commit 76d66fa6ac69aa39c91f55b9b5d5a954f9e94d73)
2013-10-24 16:54:27 -04:00
Zev Benjamin ec21275b19 Move rabbitmq username into settings.py
(imported from commit f9a8f80f53b8774781b8558444e601567cb59835)
2013-10-24 16:40:38 -04:00
Waseem Daher 94c0b2fe70 Comparison page.
(imported from commit 4e2ce843bd066fd20e5840b2fd5cd2d991be02e7)
2013-10-24 16:22:00 -04:00
Leo Franchi 410ee44eb6 Send users push notifications when they miss messages
(imported from commit 6c54fe44a82c5796268e56d3f5577bf4cfc8163a)
2013-10-24 14:54:31 -04:00
Leo Franchi a9bd2743f0 Initialize default settings if they are missing in the setting module
(imported from commit 97eb19c603436d460ad1832827dc3ec34c36bf31)
2013-10-24 10:19:24 -04:00
Tim Abbott 391de2d592 Add local settings template for local server instances.
(imported from commit 96f59e6a041992f5b2c467558a2fcc14a951a8a9)
2013-10-23 17:36:34 -04:00
Zev Benjamin ca16644152 Add client-side SockJS wrapper
The wrapper handles our RPC protocol, authentication scheme, and
reconnections.

(imported from commit 1fed2d160582c235a32de80a80b3e451c13a7b1c)
2013-10-22 18:45:11 -04:00
Zev Benjamin 5979af3a45 [manual] Add asynchronous message sender via sockjs-tornado
New dependency: sockjs-tornado

One known limitation is that we don't clean up sessions for
non-websockets transports.  This is a bug in Tornado so I'm going to
look at upgrading us to the latest version:
https://github.com/mrjoes/sockjs-tornado/issues/47

(imported from commit 31cdb7596dd5ee094ab006c31757db17dca8899b)
2013-10-22 18:45:11 -04:00
Steve Howell 0662300e9e Remove /json/subscriptions/list endpoint.
(imported from commit db2c38f6b12ebee5250737a6cdd193863a4605af)
2013-10-22 15:51:19 -04:00
Zev Benjamin 06390de20a Load underscore before blueslip
blueslip depends on underscore.

(imported from commit 13a379d62bdb9824c052d8695ec030c69f41a5db)
2013-10-22 15:39:17 -04:00
Steve Howell 8711f7450d Add History tab to /activity for realm pages.
(imported from commit 8d51da01ff8a9f4668aca7408b3ccbf9d862b81c)
2013-10-22 14:05:12 -04:00
Steve Howell 3be4641625 Subsume /queries into the main /activity report.
This moves the reports from /queries into six new tabs on the
main /queries report.

(imported from commit e4ccf6e7edfc900d929e85613f21437b40f046ae)
2013-10-22 11:11:54 -04:00
Steve Howell 1f4b70938f Add /sent_messages_report to show sent messages per realm per day.
This shows the number of messages sent by users in a realm going
back two weeks.

(imported from commit 9bf677b908f947efef4e00a43d323e1049a9157a)
2013-10-22 10:15:08 -04:00
Steve Howell f2be9f046a Create /queries page with canned queries for Waseem.
These are some queries on API usage, desktop usage, and
Android usage that would be of interest to Waseem.  These
will eventually be subsumed into /activity, but some interim
data issues may make them easier to keep separate for now.

(imported from commit 697a8496cbf4447d557a3fc89f64c1c4d3e67e70)
2013-10-16 23:40:04 -04:00
Jessica McKellar d4e2cdd09e Determine DEPLOYED only based on /etc/humbug-server.
The other zulip.net check was for a transition to using that file that
has now completed.

(imported from commit 991d9165515b5611865957255f9da7a69a75fd7b)
2013-10-16 13:18:49 -04:00
Leo Franchi f7386c3f58 [schema] Add support for keeping track of iOS APNS device tokens
In order to support iOS Push Notifications, we need to keep track
of a device's unique APNS Token. These are delivered to our iOS
code after registering for remote notifications

(imported from commit bbe34483e1380dc20a1c93e3ffa1fcfdb9087e67)
2013-10-16 12:54:28 -04:00
Jessica McKellar a7e35c0830 Fix /apps/ being a 404.
I looked around the Django docs for a more future-proof way of
addressing this problem of wanting both /apps and /apps/ to work. It
doesn't seem like there is.

Note that APPEND_SLASH is default True, so we're getting a / appended
automatically if there's no URL match without the /:

https://docs.djangoproject.com/en/dev/ref/settings/#append-slash

http://stackoverflow.com/a/11690144 points out making the slash
optional in the URL regex, but a commenter points out that this is bad
for SEO.

(imported from commit 8e883fb786c583fe5f561f14473211f6b100cecf)
2013-10-14 12:49:49 -04:00
acrefoot f8662c16f7 Add Mandrill decorators, credentials, actions
You can queue email for future delivery or send immediately via mandrill now

(imported from commit e6b6d11a2d94fcdeaffab80793e7ba31955b9031)
2013-10-10 19:32:21 -04:00
Luke Faraone 1d9391e867 Initial local server configuration.
(imported from commit ac9b9896b74b78c6ca03af7f411d0788ae402cff)
2013-10-10 14:14:14 -04:00
Luke Faraone 7d03614a1b Store the feedback bot key in local_settings.py
(imported from commit 3322d8976328db61cd382acb06775c6a6df3fea0)
2013-10-10 11:31:15 -04:00