process_fts_updates: Fix docker-zulip support.

In the series of migrations to this tool's configuration to support
specifying an arbitrary database name
(e.g. c17f502bb0), we broke support for
running process_fts_updates on the application server, connected to a
remote database server. That workflow is used by docker-zulip and
presumably other settings like Amazon RDS.

The fix is to import the Zulip virtualenv (if available) when running
on an application server.  This is better than just supporting this
case, since both docker-zulip and an Amazon RDS database are setting
where it would be inconvenient to run process-fts-updates directly on
the database server. (In the former case, because we want to avoid
having a strong version dependency on the postgres container).

Details are available in this conversation:
https://chat.zulip.org/#narrow/stream/49-development-help/topic/Logic.20in.20process_fts_updates.20seems.20to.20be.20broken/near/1251894

Thanks to Erik Tews for reporting and help in debugging this issue.
This commit is contained in:
Tim Abbott 2021-09-23 14:37:03 -07:00 committed by Alex Vandiver
parent bed7c672ce
commit 33b5fa633a
1 changed files with 21 additions and 7 deletions

View File

@ -7,7 +7,7 @@
# program processes `fts_update_log`, updating the PostgreSQL full-text
# search column search_tsvector in the main zerver_message.
#
# There are three cases this has to cover:
# There are four cases this has to cover:
#
# 1. Running in development, with a venv but no
# /home/zulip/deployments/current nor /etc/zulip/zulip.conf
@ -20,6 +20,10 @@
# frontend, with a /etc/zulip/zulip.conf but no venv nor
# /home/zulip/deployments/current
#
# 4. Running in production, on an application frontend server connected
# to a remote postgresql server, because we cannot run code _on_ the
# PostgreSQL server, such as in docker-zulip.
#
# Because of case (3), we cannot rely on functions from outside this
# file (e.g. provided by scripts.lib.zulip_tools). For case (1),
# however, we wish to import `zerver.settings` since we have no
@ -103,10 +107,21 @@ USING_PGROONGA = False
try:
# Case (1); we insert the path to the development root.
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../..")))
# Case (2) could take this branch if it set up the venv first, but
# there is no reason to unnecessarily bring in the Django
# dependency; it will fall into the path below due the venv not
# being configured.
# Cases (2) and (4); we insert the path to the production root.
# This likely works out the same as the above path.
#
# We insert this path after the above, so that if running this
# command from a specific non-current Zulip deployment, we prefer
# that deployment's libraries.
sys.path.insert(1, "/home/zulip/deployments/current")
# For cases (2) and (4), we also need to set up the virtualenv, so we
# can read the Django settings.
from scripts.lib.setup_path import setup_path
setup_path()
os.environ["DJANGO_SETTINGS_MODULE"] = "zproject.settings"
from django.conf import settings
@ -119,8 +134,7 @@ try:
pg_args["connect_timeout"] = "600"
USING_PGROONGA = settings.USING_PGROONGA
except ImportError:
# Case (2) and (3); we know that the PostgreSQL server is on this
# host.
# Case (3): we know that the PostgreSQL server is on this host.
pg_args["user"] = "zulip"
config_file = configparser.RawConfigParser()