This was recommended by:
https://docs.djangoproject.com/en/dev/howto/deployment/checklist/
Since we don't change our deployments without restarting Django and
don't use any custom template magic, this should be a free performance
win.
(imported from commit fd498ab97d0669c3a14b342b2d2f01994a1f1ee1)
I figure we can start with 600s as a maximum age -- our threads do
many dozens of requests per minute, so I figure we'll get most of the
benefit of permanently persisting connections this way. I could also
be convinced to do just 60s, though the impact will likely to be less
visible on staging. 600s seems to be what Django originally had for
this parameter before they disabled it by default. See:
https://groups.google.com/forum/#!msg/django-developers/rH0QQP7tI6w/yBusiFTNBR4J
for discussion, which also suggests we might have issues with
runserver that we should watch out for.
(imported from commit 0ae09fa4f1b39cc88c76fa58258aaf20ab168dcf)
We can revert this commit once any previously valid password reset
links would have expired.
(imported from commit ffac4d36e914e77ad1516ccfac26318c79ba0f7f)
This is for backwards compatible. Later we should actually switch to
the JSON serializer, because it means having our SECRET_KEY stolen
isn't an immediate arbitrary code execution vulnerability.
(imported from commit e68ba5cfdb79c0c1f7b178279ecd0307016f5eff)
Move commands related to stats collection and reporting from
zilencer to analytics. To do this, we had to make "analytics"
officially an app.
(imported from commit 63ef6c68d1b1ebb5043ee4aca999aa209e7f494d)
The corporate "app" is not a full-fledged Django app, but it has
a urls.py and a templates directory. This commit creates the app
and moves the jobs pages into it. Localserver deployments will
not see any of the corporate code.
(imported from commit 35889c3cf92329258c30741fdfa564769a4fac1a)
Run the following commands as root before deploying this branch:
# /root/zulip/tools/migrate-server-config
# rm /etc/zulip/machinetype /etc/zulip/server /etc/zulip/local /etc/humbug-machinetype /etc/humbug-server /etc/humbug-local
(imported from commit aa7dcc50d2f4792ce33834f14761e76512fca252)
This should help with determining the prevalence of slow sends as
experienced by users.
(imported from commit f00797679315c928af3c87ad8fdf0112f1dfa900)
This moves the list of removed files from .gitattributes to
tools/build-local-server-tarball because static/ and tools/ are
necessary for update-prod-static, and it seemed best to keep the
entire list in one place.
(imported from commit 2a447cbde29e90d776da43bb333650a40d4d363c)
This is something we'll want to do before we switch to Django 1.6,
since it's the default there, and I'm not aware of any significant
problems this switch will cause.
(imported from commit fadea41f09e2179735328f99a3ec55b46c92041e)
This report will eventually replace the per-realm report that is
now accessible through /activity. In order not to disrupt Waseem,
I'm leaving the old reports around until we've polished the new
ones.
The old report does 24 different queries to get per-realm user data.
The new approach gets all the data at once, and it slices and dices
the data in Python to accomodate our slightly quirky data model.
On localhost, this is a typical query:
LOG: duration: 5.668 ms statement: SELECT "zerver_useractivity"."id", "zerver_useractivity"."user_profile_id", "zerver_useractivity"."client_id", "zerver_useractivity"."query", "zerver_useractivity"."count", "zerver_useractivity"."last_visit", "zerver_userprofile"."id", "zerver_userprofile"."email", "zerver_client"."id", "zerver_client"."name" FROM "zerver_useractivity" INNER JOIN "zerver_userprofile" ON ("zerver_useractivity"."user_profile_id" = "zerver_userprofile"."id") INNER JOIN "zerver_realm" ON ("zerver_userprofile"."realm_id" = "zerver_realm"."id") INNER JOIN "zerver_client" ON ("zerver_useractivity"."client_id" = "zerver_client"."id") WHERE "zerver_realm"."domain" = 'zulip.com' ORDER BY "zerver_userprofile"."email" ASC, "zerver_useractivity"."last_visit" DESC
(imported from commit 0c71f4e32fe5a40f4496749dc29ad3463868d55e)
This page shows aggregate activity for a user on various
clients. This allows Waseem to troubleshoot things like users
switching between website and desktop, etc.
This particular page probably won't be used too much, but some of the
logic is gonna be reused in the per-realm activity pages.
(imported from commit b8c1fad5bfa45daab40954f92319f6f89a3fa433)
Looking at the historical data, fewer than 50% of active users have
completed the checklist, which means that it is just persistent
clutter. We also have other better ways of encouraging people to send
traffic and get the apps now.
This commit removes both the frontend UI and backend work but leaves
the db row for now for the historical data.
(imported from commit e8f5780be37bbc75f794fb118e4dd41d8811f2bf)
The moved files are:
humbug-server
humbug-local
humbug-machinetype
Their new names are their old names with 'humbug-' removed.
zulip-puppet-apply must be run before this commit is deployed
(imported from commit f4eb523244d3409b5809c279301225d3fdf0c230)
Subclasses of QueueProcessingWorker that don't override start() will
have their consume() functions wrapped by consume_wrapper(), which
will catch exceptions and log data from troublesome events to a log
file.
We need to do a puppet apply to create /var/log/zulip/queue_error.
(imported from commit 3bd7751da5fdef449eeec3f7dd29977df11e2b9c)
We were using Gravatar for user avatars, but now users can
upload their avatars directly to Zulip, and we will store
their avatar for them. This removes the old Gravatar-related
interface and polling code.
This commit does not attempt to update the avatars in
messages that have already been loaded, either for the user
making the change or other users.
(imported from commit 301dc48f96f83de0136c93de57055638c79e0961)
The "Your Account" and "Notifications" boxes on the Settings
page each had their own border and their own "Save changes"
button, but they were within the same form and sending to the
same back end point.
This commit creates a separate form and endpoint for each
of the two boxes.
(imported from commit 04d4d16938f20749a18d2c6887da3ed3cf21ef74)
This requires no changes in production, but is tagged as manual to
remind developers that they need to edit and run the tools/migrate-db
script to fix up their local database instances.
(imported from commit fbf764fb61592ef994d6d2ad56edad65ff01f14b)
This commit must be simultaneously deployed on both staging and
prod0. It also requires completely taking down the app.
To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0
* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0
* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
actually require a restart)
* run tools/migrate-db on postgres0 as root
* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1
* do a puppet apply on nagios
(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
Here we introduce a new Django app, zilencer. The intent is to not have
this app enabled on LOCALSERVER instances, and for it to grow to include
all the functionality we want to have in our central server that isn't
relevant for local deployments.
Currently we have to modify functions in zerver/* to match; in the
future, it would be cool to have the relevant shared code broken out
into a separate library.
This commit inclues both the migration to create the models as well as a
data migration that (for non-LOCALSERVER) creates a single default
Deployment for zulip.com.
To apply this migration to your system, run:
./manage.py migrate zilencer
(imported from commit 86d5497ac120e03fa7f298a9cc08b192d5939b43)
Use rest_dispatch for upload auth redirect so it doesn't send the
long URL to user_activity.
(imported from commit ab327bbd529412e43eee6d109f8550180544dbbb)
Trac #1734
This is implemented by bouncing uploaded file links through a view
that checks authentication and redirects to an expiring S3 URL.
This makes file uploads return a domain-relative URI. The client converts
this to an absolute URI when it's in the composebox, then back to relative
when it's submitted to the server.
We need the relative URI because the same message may be viewed across
{staging,www,zephyr}.zulip.com, which have different cookies.
(imported from commit 33acb2abaa3002325f389d5198fb20ee1b30f5fa)
Before deploying this commit, the following commands must be run:
# rabbitmqctl add_user zulip $(tools/get-django-setting RABBITMQ_PASSWORD)
# rabbitmqctl set_user_tags zulip administrator
# rabbitmqctl set_permissions -p / zulip '.*' '.*' '.*'
(imported from commit 76d66fa6ac69aa39c91f55b9b5d5a954f9e94d73)
New dependency: sockjs-tornado
One known limitation is that we don't clean up sessions for
non-websockets transports. This is a bug in Tornado so I'm going to
look at upgrading us to the latest version:
https://github.com/mrjoes/sockjs-tornado/issues/47
(imported from commit 31cdb7596dd5ee094ab006c31757db17dca8899b)
These are some queries on API usage, desktop usage, and
Android usage that would be of interest to Waseem. These
will eventually be subsumed into /activity, but some interim
data issues may make them easier to keep separate for now.
(imported from commit 697a8496cbf4447d557a3fc89f64c1c4d3e67e70)
The other zulip.net check was for a transition to using that file that
has now completed.
(imported from commit 991d9165515b5611865957255f9da7a69a75fd7b)
In order to support iOS Push Notifications, we need to keep track
of a device's unique APNS Token. These are delivered to our iOS
code after registering for remote notifications
(imported from commit bbe34483e1380dc20a1c93e3ffa1fcfdb9087e67)
I looked around the Django docs for a more future-proof way of
addressing this problem of wanting both /apps and /apps/ to work. It
doesn't seem like there is.
Note that APPEND_SLASH is default True, so we're getting a / appended
automatically if there's no URL match without the /:
https://docs.djangoproject.com/en/dev/ref/settings/#append-slashhttp://stackoverflow.com/a/11690144 points out making the slash
optional in the URL regex, but a commenter points out that this is bad
for SEO.
(imported from commit 8e883fb786c583fe5f561f14473211f6b100cecf)
This requires renaming the account in Google Apps at the time we
deploy this; we'll probably want to do this during off hours to avoid
any user-visible downtime.
This also updates some related email addresses.
(imported from commit fce7629b359a4f278bbf7815c8d177a8fa0484fe)
This may require just doing an mv on the home directory, plus changing
the home directory in /etc/passwd. It should of course be done carefully.
(imported from commit 660997d897ee6d33563af74f0fc5d4267a911755)
This has been deployed, so the current code is now properly
minified, and we don't need the force_minify flag any more.
(The flag is needed when you remove a JS file from JS_SPECS.)
(imported from commit 8b3c7ffdbc875011d59c2560034750f0077db616)
The minify logic doesn't have an easy way to detect that you
removed a file since the last deployment.
(imported from commit 50d05fcdad382a586073c06d29d279433d1bba81)
We are still showing the same data points, but the logic to drill
down on details for a particular realm is now all server side,
not client side, and we are smarter about omitting fields. In
summary mode, we don't show empty Name or Email columns. In
detailed mode, we show the realm as a headline instead of a column.
In this version you do lose the ability to see all system users in
the same view, but Waseem is ok with this.
(imported from commit edd2e646ab4cf5783ea64232d0cd621debece8d4)
Tim Abbott
Leo Franchi
acrefoot
Steve Howell
Luke Faraone
Zev Benjamin
Kevin Mehall
Jessica McKellar
Waseem Daher
acrefoot