The readthedocs theme overrides a few settings in their layout template.
We might want to change some settings back to their default values.
This commit copies the original readthedocs layout file from
https://github.com/rtfd/sphinx_rtd_theme/blob/master/sphinx_rtd_theme/layout.html
to _templates/layout.html, and excludes it from lint and template checks.
Addresses #7417.
This doesn't touch the main path through the install docs; that will
see a broader rewrite soon as we make outbound email optional for a
nascent server, make the hostname and admin email into mandatory
installer flags, and then radically simplify the instructions by
removing mandatory editing of `settings.py` and folding most of what's
left into the installer.
This commit renames various source requirements files like `dev.txt`,
`mypy.txt` etc to `dev.in`, `mypy.in` etc and various locked requirements
files like `dev_lock.txt`, `mypy_lock.txt` etc to `dev.txt`, `mypy.txt`
etc. This will help in emphasizing to the user that *.in are actually
input to `update-locked-requirements` tool which should be run after
updating any of these.
This adds a few missing entries to the TOC, which hadn't made sense
back when Zulip's ReadTheDocs didn't have the new collapsing feature.
Tweaked by tabbott to also give the SSL certificates doc an
appropriate title for its new role.
Expand bullets "[+]" show up on locals instance, but they
do not show up online on readthedocs. The expand icon is valuable
to indicate that each section has more content.
The problem is readthedocs appends code to conf.py that overrides the
html_theme_options prior to a build. This commit prevents this from
happening so that collapse_navigation remains set to False.
In response to merged PR 7326.
This updates the developer documentation to recommend `git clone -c`
to set the git pull mode to be rebase by default in the repository
configuration.
The hope is that this will help minimize how often new contributors
end up accidentally messing up their PRs by using `git pull`.
Tweaked by tabbott to fix broken links and optimize the language.
Fixes: #7022.
This should get test-documentation and test-help-documentation passing
again after we moved everything around in the last commit.
There are a lot more absolute links (generally in code comments) to
update, but they are lower-priority and can be done in a follow-up
pass.
This commit helps reduce clutter on the navigation sidebar.
Creates new directories and moves relevant files into them.
Modifies index.rst, symlinks, and image paths accordingly.
This commit also enables expandable/collapsible navigation items,
renames files in docs/development and docs/production,
modifies /tools/test-documentation so that it overrides a theme setting,
Also updates links to other docs, file paths in the codebase that point
to developer documents, and files that should be excluded from lint tests.
Note that this commit does not update direct links to
zulip.readthedocs.io in the codebase; those will be resolved in an
upcoming follow-up commit (it'll be easier to verify all the links
once this is merged and ReadTheDocs is updated).
Fixes#5265.
The warning here means that the admin can't really act on this yet if
they want to disable email auth, which is likely among admins that
want to make any changes here. And for admins who don't, this is an
extra thing to read and make a decision about before they can get a
server running. See #6985.
Conversely, we already discuss auth backends right at the top of the
`prod-customize` doc, which is linked under "Next steps" at the end of
these instructions.
The warning about EmailAuthBackend is important; but we can move it to
the config file right next to the setting, and then it's available
right when it's actionable, which is if the admin is actually thinking
about changing the setting.
LXC on Debian does not ship with a default network config for
containers, so we now recommend setting up the lxc network config
manually.
Tweaked by tabbott to make these details much harder to miss.
The left navigation bar defaults to "sticking" to the screen as you
scroll. This commit adds the sticky_navigation setting to conf.py to
disable the default behavior.
This addresses part of #5265.
The control panel on the Google side doesn't seem to match the
instructions we have; it looks pretty 2017 to me, so I imagine
it's had a redesign since the instructions were written.
Also, in dev, EXTERNAL_HOST is now a port on zulipdev.com, not on
localhost.
Update these instructions for those developments, and edit lightly.
In dev, recommend setting in `dev_settings` instead of in
`prod_settings_template`; that feels to me a little more reflective of
the actual intent, and the effect should be equivalent.
In some minutes of searching yesterday afternoon on the docs site,
I couldn't find docs on how to set up our OAuth integrations for
the dev environment -- even though I was pretty sure they should
be there somewhere, because I'd just been told that. Wasn't until
I considered the problem fresh today, and grepped the docs source
instead, that I found them.
So, move them to a separate page so they're in the nav.
This means one fewer thing the admin typically needs to read, absorb,
and make a decision about at install time.
The one way this change could hypothetically cause trouble is if the
admin wants to keep subdomains of EXTERNAL_HOST out of ALLOWED_HOSTS.
But while the subdomains often won't exist as domain names, it's hard
to imagine the situation in which they would exist but be under
someone else's control, or be doing something other than serving
Zulip realms.
The "subdomain" label is redundant, to the extent it's even
accurate -- this is really just the URL we want to display,
which may or may not involve a subdomain. Similarly "external".
The former `external_api_path_subdomain` was never a path -- it's a
host, followed by a path, which together form a scheme-relative URL.
I'm not quite convinced that value is actually the right thing in
2 of the 3 places we use it, but fixing that can start by giving an
accurate name to the thing we have.
This was part of the logic to handle EXTERNAL_API_PATH varying.
But also it was already no longer used -- it was only ever passed
into template contexts, as `external_api_uri`, and it'd been
overtaken there by `external_api_uri_subdomain`.
So, update our dev docs to reflect that, and eliminate the variable.
http://zulip.readthedocs.io/en/latest/architecture-overview.html
mentions "Our mobile clients are separate code repositories: Android
and React Native iOS app. " However Zulip Mobile is the official
mobile Zulip client supporting both iOS and Android.
Edited by tabbott to also fix confusion about the desktop.
The Vagrant environment setup tutorial anchor links for the
Troubleshooting & Common Errors section don't work in GitHub flavored
markdown because the ampersand in the heading gets removed and the
resulting space is filled with a hyphen. (See here:
https://gist.github.com/asabaylus/3071099#gistcomment-1593627)
To make anchor links work with ReadTheDocs as well as with GitHub
markdown for the time being, this commit changes ampersand in the
heading to 'and'.
Fixes#7056.
Except in:
- docs/writing-bots-guide.md, because bots are supposed to be Python 2
compatible
- puppet/zulip_ops/files/zulip-ec2-configure-interfaces, because this
script is still on python2.7
- tools/lint
- tools/linter_lib
- tools/lister.py
For the latter two, because they might be yanked away to a separate repo
for general use with other FLOSS projects.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful. For configuration and the core
accept/reject logic, it's better to use units that are readily
understood. Switch to those.
I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses. And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft". So, "guesses" it is.
And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting. So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.
(Thanks to @YJDave for a prototype that the code changes in this
commit are based on.)
This has a ton of exclude rules, for two reasons:
(1) We haven't been particularly systematic about avoiding unnecessary
inline style in the past, so there's a lot of code we need to fix.
(2) There are cases where one wants to dynamically compute style
rules. For the latter category, ideally we'd figure out a way to
exclude these automatically (e.g. checking for mustache tags in the
style tag).
We just learned we should be using the "onlytranslated" mode of
Transifex. Since the command is getting a bit complex (and you need
to remember to run `makemessages` first), it makes sense to have a
tool for it.
This commit updates various places where check_send_message had
been previously recommended to recommend check_send_stream_message
for sending messages to a public stream.
This accumulates several changes in recent commits: decimal point
rather than comma, compact log level, and logger names, the latter
abbreviated `zr` in the case of `zulip.request`.
Collaboration in area label teams is only available to members of the
Zulip organization on GitHub. For non-members the related links are
not working, which can be confusing. Address this by explaining the
links won't work and also that anyone can join.
This is just enough of a quick fix to work with a stock Zulip 1.6
server. We should really also make this robust to arbitrary input
from the remote Zulip server, even though it'll be a little tedious.
This is a start and fixes the most glaring problems from not updating
this documentation; I'd like Harshit to do a helpful pass on updating
this to cover some of the more subtle details about how our emoji
picker works, emoji aliases, etc.
This change means that almost every Zulip server out there will now be
using subdomains for every realm. There are a few complications noted
in the release notes.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.
This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
Since we've found that it's fairly frequent that we want to recommend
to developers that they upgrade to a version of Zulip from Git, it
makes sense to include that by default.
This keeps the examples in line with our actual codebase.
Also while I'm here revise it to explain the actual motivation for our
use of `env`, and to correct some subtle details -- it's actually the
kernel that interprets the shebang (as visible in e.g. a `strace` log),
not the shell, and when the program is executed as `./my_program.py`
the exact name including `./` is passed to the interpreter.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2. In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.
One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2. See discussion on the respective previous
commits that made those explicit. There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
This works fine in my testing -- I followed it on a fresh `trusty` VM,
after just getting SSL certs with our LetsEncrypt instructions, and
the install completed successfully.
And in the source tree, the only evidence I can find of a potential
remaining dependency on the `/root/zulip` path is the Nagios config in
`puppet/zulip_ops/`. That's actually already broken, in that it
depends on `/root/zulip/api/`, so we'll have to sort that out;
and in any case, it doesn't matter to most people installing Zulip.
Add a mention that the install script will move the just-unpacked
directory out from under you. While we're here, add a few words about
where the deployed code is laid out.
Text of those last words tweaked by tabbott.
This follows up on 207cf6302 from last year to clean up cases that
have apparently popped up since then. Invoking the scripts directly
makes a cleaner command line in any case, and moreover is essential
to how we control running a Zulip install as either Python 2 or 3
(soon, how we always ensure it runs as Python 3.)
One exception: we're currently forcing `provision` in dev to run
Python 3, while still running both Python 2 and Python 3 jobs in CI.
We use a non-shebang invocation to do the forcing of Python 3.
Since we're auto-detecting the value anyway, there's no reason it
can't be moved to DEFAULT_SETTINGS.
This lets us remove some clutter from the installation documentation.
This field is convenient for bankruptcy checks. Clients could
calculate it from page_params.unread_msgs before this change, but
it would kind of a painful calculation.
To add count, we had to simplify the mypy annotations, which weren't
really accurate before.
Update Email, Beanstalk, Hubot, JIRA, and Trello integrations
links.
The Hubot integrations section (/integrations#hubot-integrations)
was removed in an earlier redesign of /integrations. This commit
replaces the link with the hubot-scripts organization on
Github, which displays the comprehensive list of all integrations
available via Hubot.
Fixes#5875.
In some of these contexts, we may still be *using* the Python 2
version, but at least this should eliminate running into
`ImportError`s one by one in scripts that run outside a virtualenv,
as we update their shebangs to refer to Python 3.
Several Python libraries we use don't come in Python 3 versions on
trusty: gevent, boto, twisted, django, django-tagging, whisper.
The latter two don't come in Python 3 versions even on xenial.
So some work required before we can actually switch the code that
relies on those libraries to run as Python 3 -- probably the best
solution will be to backport them all in our apt repo. (All but
`whisper` are packaged in zesty; `whisper` upstream just grew Python 3
support this year.)
We are phasing out the following in tests:
add_dependencies - this is just kind of a clunky UI
require - normal JS requires cause test leaks
In order to plug require leaks, we are effectively doing what
we always have done inside of add_dependencies, which is to
keep track of which modules we have done `require` on, and
these get cleared between tests.
Now we just use `zrequire` every time we want to pull in real
code to our global namespace.
I think soon we'll put the Python 3 venv at `/srv/zulip-venv` and
make `/srv/zulip-py3-venv` just an alias to that (then remove the
alias too), but for now the old name is helpful for spotting places
that need an update.
The `/srv/zulip-venv` name still appears in codepaths used by Travis
tests.
From the line in tools/provision it should trickle to the rest of the
scripts. This works since almost all the python scripts have been linted
to be generic.
Proof that the setup is python3 only: with this commit, within the
vagrant container env, /srv/zulip-venv is no longer present and
`./tools/run-dev.py` runs just fine.
[gnprice: Added `rm -f` and warning message, and made small edits.]
This makes supervisor see the service as cheerfully running
and let it alone, rather than constantly retry starting it.
Because the crash/restart loop means repeatedly spending a
couple of seconds loading Django and the app, separated by
brief periods while supervisor notices the crash and acts
on it, it was actually consuming about 30-50% CPU on the
zulipchat.com staging server.
This pulls together what's covered in detail in several
longer pages, and gives us a page that can serve as a good
drop-in replacement for https://zulip.org/server.html .
Also tweak a couple of related bits for clarity and orthography.
We are adding a new list of unread message ids grouped by
conversation to the queue registration result. This will allow
clients to show accurate unread badges without needing to load an
unbound number of historic messages.
Jason started this commit, and then Steve Howell finished it.
We only identify conversations using stream_id/user_id info;
we may need a subsequent version that includes things like
stream names and user emails/names for API clients that don't
have data structures to map ids -> attributes.
After editing this file for the previous commit, I couldn't unsee
"low-traffic the". So I fixed that, saw more issues, and then just
went through the page for everything that was similarly trivial to fix.
This allow the webbpack dev server to properly reload JavaScript modules
while running in dev without restarting the server. We need to connect
to webpack-dev-server directly because SockJS doesn't support more than
one connection on the same host/port.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization. It controls access through 2
key mechanisms:
* For long-ago edited messages, get_messages removes the edit history
content from messages it sends to clients.
* For newly edited messages, clients are responsible for checking the
setting and not saving the edit history data. Since the webapp was
the only client displaying it before this change, this just required
some changes in message_events.js.
Significantly modified by tabbott to fix some logic bugs and add a
test.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
`emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
package.
Bumps PROVISION_VERSION.
Fixes: #4730.
NPM packages should be installed at the beginning of the provisioning
process so that later in the provisioning process if a script requires
any NPM package it can use it. Earlier, we were installing NPM packages
in the last as the installation process can fail due to network issues
but since we now retry in case the installation fails, they can be
installed safely at the beginning of the process as well just like apt
packages.
David Rosa Tamsen
rht
Greg Price
Harshit Bansal
Tim Abbott
Balaji2198
Tommy Ip
derAnfaenger
Sarah
Mohd Ali Rizwi
KiranS
Rishi Gupta
Vishnu Ks
Eeshan Garg
Brock Whittaker
Steve Howell
Rhea Parekh
Jack Zhang
Nikhil Kumar Mishra
Alexis La Porte
Priscilla
Garvit Dewan
Joshua Pan
neiljp (Neil Pilgrim)
Alena Volkova
vbNETonIce
zeneli
Supermanu
Kir Chou
Kir Chou
Hobson Lane
Vaida Plankyte
Umair Khan
David Taylor
Greg Price
Aaron Tse
Pweaver (Paul Weaver)
Steve Howell
Jason Michalski
Cynthia Lin
Shubham Gupta
Vishnu Ks
Durga Akhil Mundroy
Rohitt Vashishtha