Commit Graph

121 Commits

Author SHA1 Message Date
Steve Howell f5bb2409ef Create "analytics" app with activity reports.
(imported from commit 6385935c3d7894fe52bcc265faecc30b07629717)
2013-11-06 12:07:32 -05:00
Steve Howell c3d176a480 Move marketing pages to corporate app.
(imported from commit 70528b544ceb4be12fb02be32889e1b9ca8caeb4)
2013-11-06 12:07:32 -05:00
Steve Howell 6193f60d45 Move static MIT-related pages to corporate app.
(imported from commit e2c5f72d3f3e4d8f8fa888326b2ba455beb9156c)
2013-11-06 12:07:31 -05:00
Steve Howell a827d727cc Create non-localserver "corporate" app with jobs pages.
The corporate "app" is not a full-fledged Django app, but it has
a urls.py and a templates directory.  This commit creates the app
and moves the jobs pages into it.  Localserver deployments will
not see any of the corporate code.

(imported from commit 35889c3cf92329258c30741fdfa564769a4fac1a)
2013-11-06 12:07:31 -05:00
Luke Faraone c11b65590b SSO / REMOTE_USER support
(imported from commit 4f4fad7af5d3c6099cac95d7708338c182626d72)
2013-11-05 16:14:13 -05:00
Luke Faraone a34731ed00 Factor out get_user in zproject/backends.py
(imported from commit d60b5440722ed596ffbcb81086b2f62d535288dd)
2013-11-05 16:14:13 -05:00
Steve Howell 5ff6bc7330 Move activity reports to zilencer.
This commit moves the view code and the URLs.  It doesn't touch
the templates yet.

(imported from commit dbef5eafe4956b29154c1134c05aa586c9e417b2)
2013-11-05 15:41:03 -05:00
Tim Abbott e06722657a [manual] Remove /messages/latest API and related legacy code.
This requires doing a puppet apply on our servers to take effect
properly.

(imported from commit 19dc56f071f07a5d2571eef49dd835121b2e82b6)
2013-11-05 14:19:40 -05:00
Zev Benjamin 0073014ea9 Remove redundant EXTERNAL_HOST settings
(imported from commit dcf9709bc5c2a2c3f00410fd1b2849d3ab879fb9)
2013-11-05 14:14:19 -05:00
Zev Benjamin 787215d743 [manual] Switch over to new /etc/zulip/zulip.conf config file
Run the following commands as root before deploying this branch:
 # /root/zulip/tools/migrate-server-config
 # rm /etc/zulip/machinetype /etc/zulip/server /etc/zulip/local /etc/humbug-machinetype /etc/humbug-server /etc/humbug-local

(imported from commit aa7dcc50d2f4792ce33834f14761e76512fca252)
2013-11-05 14:14:19 -05:00
Tim Abbott 792b283dcd Report end-to-end sending times as experienced by clients.
This should help with determining the prevalence of slow sends as
experienced by users.

(imported from commit f00797679315c928af3c87ad8fdf0112f1dfa900)
2013-11-04 18:32:34 -05:00
Leo Franchi 6765ec0795 Use a site-specific Zulip Admin email in django error pages
(imported from commit 2d5415d7cd81befc3051b5de3835c0cd258b6375)
2013-11-04 16:35:50 -05:00
Kevin Mehall 78b45b61fb Minify JS in the localserver upgrade tarball.
This moves the list of removed files from .gitattributes to
tools/build-local-server-tarball because static/ and tools/ are
necessary for update-prod-static, and it seemed best to keep the
entire list in one place.

(imported from commit 2a447cbde29e90d776da43bb333650a40d4d363c)
2013-11-04 13:40:43 -05:00
Tim Abbott 68dcc760c3 Clean up some unused imports.
(imported from commit 0c5d8e2a55ba1b8909ba807fee3afe863dcdc226)
2013-11-04 11:51:17 -05:00
Tim Abbott 2844a8fb97 Enable postgres database-level autocommit.
This is something we'll want to do before we switch to Django 1.6,
since it's the default there, and I'm not aware of any significant
problems this switch will cause.

(imported from commit fadea41f09e2179735328f99a3ec55b46c92041e)
2013-11-04 07:31:11 -05:00
Leo Franchi ca8187b8cf Email gateway bot variable name changed to EMAIL_GATEWAY_BOT
(imported from commit 01c430bd9b916c9bb152ee821214495333f1d37e)
2013-11-01 16:46:17 -04:00
Leo Franchi 9b4491db6a Don't use hardcoded bots in nagios' check_send_receive
(imported from commit 82add135bf5b819bcc992af8420eec14cf829ccc)
2013-11-01 14:13:05 -04:00
Leo Franchi db6550e99a Make built-in bots come from settings.py, and allow localserver-specific ones
(imported from commit e21933e37487314ac986147562817a19227e8960)
2013-11-01 14:13:05 -04:00
Steve Howell 95dc0894d8 Add /realm_activity report.
This report will eventually replace the per-realm report that is
now accessible through /activity.  In order not to disrupt Waseem,
I'm leaving the old reports around until we've polished the new
ones.

The old report does 24 different queries to get per-realm user data.
The new approach gets all the data at once, and it slices and dices
the data in Python to accomodate our slightly quirky data model.

On localhost, this is a typical query:

LOG:  duration: 5.668 ms  statement: SELECT "zerver_useractivity"."id", "zerver_useractivity"."user_profile_id", "zerver_useractivity"."client_id", "zerver_useractivity"."query", "zerver_useractivity"."count", "zerver_useractivity"."last_visit", "zerver_userprofile"."id", "zerver_userprofile"."email", "zerver_client"."id", "zerver_client"."name" FROM "zerver_useractivity" INNER JOIN "zerver_userprofile" ON ("zerver_useractivity"."user_profile_id" = "zerver_userprofile"."id") INNER JOIN "zerver_realm" ON ("zerver_userprofile"."realm_id" = "zerver_realm"."id") INNER JOIN "zerver_client" ON ("zerver_useractivity"."client_id" = "zerver_client"."id") WHERE "zerver_realm"."domain" = 'zulip.com'  ORDER BY "zerver_userprofile"."email" ASC, "zerver_useractivity"."last_visit" DESC

(imported from commit 0c71f4e32fe5a40f4496749dc29ad3463868d55e)
2013-11-01 11:46:26 -04:00
Steve Howell 3577ba7585 Add /user_activity page.
This page shows aggregate activity for a user on various
clients.  This allows Waseem to troubleshoot things like users
switching between website and desktop, etc.

This particular page probably won't be used too much, but some of the
logic is gonna be reused in the per-realm activity pages.

(imported from commit b8c1fad5bfa45daab40954f92319f6f89a3fa433)
2013-11-01 11:46:26 -04:00
Tim Abbott 21663601b4 Remove obsolete zulip.com/what-is-humbug page.
(imported from commit 0e2f317cf0bf262eb113caa7daab7d71ce564e9e)
2013-10-31 15:29:49 -04:00
Tim Abbott 0e35424a7c Rename humbug => zulip in some log file names.
(imported from commit 0bb5284566861e5a2f260cc66dcb53ebac0d7b97)
2013-10-31 15:29:49 -04:00
Jessica McKellar f1e90086f5 Remove the onboarding checklist.
Looking at the historical data, fewer than 50% of active users have
completed the checklist, which means that it is just persistent
clutter. We also have other better ways of encouraging people to send
traffic and get the apps now.

This commit removes both the frontend UI and backend work but leaves
the db row for now for the historical data.

(imported from commit e8f5780be37bbc75f794fb118e4dd41d8811f2bf)
2013-10-31 11:34:15 -04:00
Steve Howell d7d1888ce0 Add TestWorker queue processor.
TestWorker is for troubleshooting, and it simply appends lines to a file
in /tmp.

(imported from commit 96b646a193a5474f3222b41725742e359b5301ec)
2013-10-30 16:47:16 -04:00
Zev Benjamin 57bef07832 [manual] Move /etc/humbug-* files into /etc/zulip
The moved files are:
humbug-server
humbug-local
humbug-machinetype

Their new names are their old names with 'humbug-' removed.

zulip-puppet-apply must be run before this commit is deployed

(imported from commit f4eb523244d3409b5809c279301225d3fdf0c230)
2013-10-30 15:42:25 -04:00
Jessica McKellar 8e838f4db3 Add a first pass at Stash integration via their webhook mechanism.
(imported from commit 6173379f72aeef318550b35e51d4aa1fd69997ed)
2013-10-29 22:40:42 -04:00
Steve Howell 884e602185 [puppet] Handle exceptions from queue workers.
Subclasses of QueueProcessingWorker that don't override start() will
have their consume() functions wrapped by consume_wrapper(), which
will catch exceptions and log data from troublesome events to a log
file.

We need to do a puppet apply to create /var/log/zulip/queue_error.

(imported from commit 3bd7751da5fdef449eeec3f7dd29977df11e2b9c)
2013-10-29 17:45:31 -04:00
Leo Franchi 8e05f76511 Only enable analytics and dropbox on non-LOCAL_SERVER
(imported from commit 1ba877550b3afde51bec6f344762ea998800c5b6)
2013-10-29 17:33:36 -04:00
Kevin Mehall 06590f4332 Upload to local filesystem on local server.
(imported from commit 6c38a8cae721b2e7a0863470692cc56425006ecb)
2013-10-29 16:19:04 -04:00
Kevin Mehall 209ff75af8 Remove duplicate MANDRILL_API_KEY setting
(imported from commit c71928b5bfd71e9fcda1f5d57e9c7b99361ce070)
2013-10-29 15:47:13 -04:00
Steve Howell 7cf66b5bb8 Allow users to upload avatars on the Settings page.
We were using Gravatar for user avatars, but now users can
upload their avatars directly to Zulip, and we will store
their avatar for them.  This removes the old Gravatar-related
interface and polling code.

This commit does not attempt to update the avatars in
messages that have already been loaded, either for the user
making the change or other users.

(imported from commit 301dc48f96f83de0136c93de57055638c79e0961)
2013-10-29 10:59:41 -04:00
Steve Howell 68942a8f3a Properly split up "Your Account" and "Notifications".
The "Your Account" and "Notifications" boxes on the Settings
page each had their own border and their own "Save changes"
button, but they were within the same form and sending to the
same back end point.

This commit creates a separate form and endpoint for each
of the two boxes.

(imported from commit 04d4d16938f20749a18d2c6887da3ed3cf21ef74)
2013-10-29 10:59:41 -04:00
Tim Abbott 56e9ad230e [manual] Move our deployment scripts to scripts/.
This will require updating the post-receive code on git.zulip.net to
work.

(imported from commit 2e51fa2d7b891c1138d3f22ae534cfb8a6cf174c)
2013-10-28 10:54:48 -04:00
Tim Abbott 0a44ba2a28 Move servers/configure-rabbitmq to scripts/setup/.
(imported from commit 2b4d5ccb88675447ae744fb985246211deef7486)
2013-10-28 10:54:48 -04:00
Steve Howell 09979252dd Add Steve's laptop to USING_RABBITMQ list.
(imported from commit 3527edd27e3f386eb8b668fe8ee7e3875282ead8)
2013-10-27 11:27:27 -04:00
Zev Benjamin cdcbe61cb0 [manual] Change references to the humbug_test user, schema, and database to zulip_test
This requires no changes in production, but is tagged as manual to
remind developers that they need to edit and run the tools/migrate-db
script to fix up their local database instances.

(imported from commit fbf764fb61592ef994d6d2ad56edad65ff01f14b)
2013-10-26 04:16:28 -04:00
Zev Benjamin 7b6f6823a5 [manual] Change references to the humbug user, schema, and database to zulip
This commit must be simultaneously deployed on both staging and
prod0.  It also requires completely taking down the app.

To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0

* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0

* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
  actually require a restart)
* run tools/migrate-db on postgres0 as root

* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1

* do a puppet apply on nagios

(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
2013-10-26 04:16:27 -04:00
Waseem Daher 66f48288b4 Rename LOCALSERVER -> LOCAL_SERVER.
(imported from commit b3abdd10d54d2ad7a9c463af9a291d2e2127707f)
2013-10-25 17:37:06 -04:00
Leo Franchi 8e21d70afe Disable statsd on local server
(imported from commit cac2c95c2de18b613a0900fe1c04fd97963b9350)
2013-10-25 16:37:29 -04:00
Leo Franchi c12a97577e Rework local_settings_template.py to be clear about what the user needs to do
(imported from commit f69ea0d57e885509abc70dc4f2bee6e36169691f)
2013-10-25 16:35:27 -04:00
Luke Faraone 238d19e1f1 Include a rest_dispatch in tornadoviews and call it from urls.py
(imported from commit aefe66b44be60b39d0f6bb3076502e964f1ed6bd)
2013-10-25 14:13:31 -04:00
Luke Faraone e552307511 Send feedback to a queue to be forwarded to staging.
(imported from commit 4a9a1bfc6c95763a816263a726cc61b3ca90bf15)
2013-10-25 14:13:30 -04:00
Luke Faraone 81d7dd1fda [schema] Support for authenticating Deployments via the API.
Here we introduce a new Django app, zilencer. The intent is to not have
this app enabled on LOCALSERVER instances, and for it to grow to include
all the functionality we want to have in our central server that isn't
relevant for local deployments.

Currently we have to modify functions in zerver/* to match; in the
future, it would be cool to have the relevant shared code broken out
into a separate library.

This commit inclues both the migration to create the models as well as a
data migration that (for non-LOCALSERVER) creates a single default
Deployment for zulip.com.

To apply this migration to your system, run:
   ./manage.py migrate zilencer

(imported from commit 86d5497ac120e03fa7f298a9cc08b192d5939b43)
2013-10-25 14:13:30 -04:00
Leo Franchi 0bc5b13648 Fix Zulip username of email gateway bot
(imported from commit f24c5fd3fc5888bf7dc4d9501723777ad8447704)
2013-10-25 11:51:13 -04:00
Leo Franchi a9b9c0ec7b Add Email Mirror default settings and error out with readable warning if not configured
(imported from commit adac321d09561b51277ec1e314a1d480dc01e0c8)
2013-10-25 11:51:13 -04:00
Leo Franchi 2d276179d0 Refactor email-mirror to handle running on any machine
(imported from commit 2971449ceaacb564770e66874fc095f77e68d445)
2013-10-25 11:37:34 -04:00
Kevin Mehall 3bad5acc71 Fix upload auth redirect for user_activity.
Use rest_dispatch for upload auth redirect so it doesn't send the
long URL to user_activity.

(imported from commit ab327bbd529412e43eee6d109f8550180544dbbb)
2013-10-24 17:33:12 -04:00
Kevin Mehall 0a3a22cb3d Support authenticated upload URLs.
Trac #1734

This is implemented by bouncing uploaded file links through a view
that checks authentication and redirects to an expiring S3 URL.

This makes file uploads return a domain-relative URI. The client converts
this to an absolute URI when it's in the composebox, then back to relative
when it's submitted to the server.

We need the relative URI because the same message may be viewed across
{staging,www,zephyr}.zulip.com, which have different cookies.

(imported from commit 33acb2abaa3002325f389d5198fb20ee1b30f5fa)
2013-10-24 17:01:06 -04:00
Zev Benjamin c13114c7e3 [manual] Change rabbitmq user to 'zulip'
Before deploying this commit, the following commands must be run:
  # rabbitmqctl add_user zulip $(tools/get-django-setting RABBITMQ_PASSWORD)
  # rabbitmqctl set_user_tags zulip administrator
  # rabbitmqctl set_permissions -p / zulip '.*' '.*' '.*'

(imported from commit 76d66fa6ac69aa39c91f55b9b5d5a954f9e94d73)
2013-10-24 16:54:27 -04:00
Zev Benjamin ec21275b19 Move rabbitmq username into settings.py
(imported from commit f9a8f80f53b8774781b8558444e601567cb59835)
2013-10-24 16:40:38 -04:00