Commit Graph

6 Commits

Author SHA1 Message Date
rht a311678190 zerver/management: Use python 3 syntax for typing. 2017-10-26 15:24:56 -07:00
rht 32650c8fdc zerver/management: Remove print_function. 2017-09-27 18:05:45 -07:00
rht e239e97351 zerver/management: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott d43f5ceeec database: Add database index for wildcard mentions. 2017-08-16 13:28:04 -07:00
Tim Abbott 8bb812c8a9 database: Add database index for alert words. 2017-08-16 12:39:01 -07:00
Steve Howell 3a825ef22c Add create_large_indexes management command.
This management command creates the same indexes as migrations
82, 83, and 95, which are all indexes on the huge UserMessage
table. (*)

This command quickly no-ops with clear messaging when the
indexes already exist, so it's idempotent in that regard. (If
somebody somehow creates an index by the same name incorrectly,
they can always drop it in dbshell and re-run this command.)

If any of the migrations have not been run, which we detect simply
by the existence of the indexes, then we create them using a
`CREATE INDEX CONCURRENTLY` command.  This functionality in
postgres allows you to create indexes against large tables
without disrupting queries against those tables.  The tradeoff
here is that creating indexes concurrently takes significantly
longer than doing them non-concurrently.

Since most tables are small, we typically just use regular
Django migrations and run them during a brief interval while
the app is down.

For indexes on big tables, we will want to run this command
as part of the upgrade process, and we will want to run
it while the app is still up, otherwise it's pointless.

All the code in create_indexes() is literally copy/pasted
from the relevant migrations, and that scheme should work
going forward.  (It uses a different implementation of
create_index_if_not_exist than the migrations use, but the
code is identical lexically in the function.)

If we ever do major restructuring of our large tables, such
as UserMessage, and we end up droppping some of these indexes,
then we will need to make this command migrations-aware.  For
now it's safe to assume that indexes are generally additive in
nature, and the sooner we create them during the upgrade process,
the better.

(*) UserMessage is huge for large installations, of course.
2017-08-16 12:38:51 -07:00