This solves a common migration problem for folks who cut corners when first setting up Zulip. Fixes #11294.