This fixes exceptions when sending PMs in development (where we were
trying to connect to the localhost push bouncer, which we weren't
authorized for, but even if we were, it wouldn't work, since there's
no APNS/GCM certs).
At the same time, we also set and order of operations that ensures one
has the opportunity to adjust the server URL before submitting
anything to us.
This will likely change pretty quickly as we build this feature; I
wrote this just to have a central place to keep track of what we'll
need to document when we ship this feature.
We started doing this for install docs in de2a2d0df, because `latest`
wasn't suitable and because I didn't know about readthedocs's `stable`
feature. The result has been that even with a checklist item, we
don't reliably update the link.
Instead, use the special `stable` version identifier on readthedocs to
link automatically to the highest version it knows about.
This should make it easier to find the templates that are actually
part of the core webapp, instead of having them all mixed together
with the portico pages.
In the very early days of Zulip, we didn't have unread counts; just
the pointer, and the correct behavior when opening a new tab was to
place you near the pointer. That doesn't make any sense now that we
do have unread counts, and this corner case has been a wart for a long
time.
This commit does the main behavior change here. However, there's a
bug we need to fix, where we might end up trying to pre-render a view
of the narrow based on the `all_msg_list` data before `all_msg_list`
is caught up). We need to fix that bug before we can merge this; it
should be possible to determine that using `FetchStatus` on
`all_msg_list`, or with better performance by using the `unread_msgs`
structure to determine whether the message we should be selecting is
present locally.
Fixes#789.
Fixes#9070.
This is a first pass at fixing node docs. This commit eliminates
some text that is either obsolete or just overly confusing, and
it fixes some of the code samples to reflect how the API has
evolved in the last couple years. We also prominently tell
you how to run the tests.
I don't think anybody ever really used this feature, which I
developed but don't even use myself. It kind of runs counter
to the minimalist approach of the rest of node tests.
I would eventually like to re-think the template tests altogether.
They're slow, and we could solve that somewhat by replacing
jsdon/jquery with an HTML parser library to verify structural
things.
It's also possible that we can just rely on our template linters
to catch the biggest class of errors (malformed tags) and let
code review do the rest.
And it's also possible that we should make a second attempt to
ramp up tooling on making it easy to verify templates, but it
doesn't have to be part of the node tests. If we did that, we
would also potentially use tooling for Python-side templates.
These aren't perfect -- in particular "core chat experience" can
probably be broken up -- but I think they help in making a quick skim
work for getting some sense of what the changes are.
This change just reorders and adds headings, with virtually no wording
changes.
This is kind of easy to gloss over, especially with the framing
as a "format"; surely if things work at all, the file format
must have been right, right? It's really a bit more substantive
than that; say so and also add a bit more description.
In addition to many small edits for formatting and clarity, a few more
significant changes:
* In the main instructions, refer specifically to restarting the
server and to testing that the config works.
* Add SendGrid to the recommended list, as it seems like people
give it a somewhat stronger reputation these days than Mailgun.
* Discuss EMAIL_USE_TLS and EMAIL_PORT along with host, user, and
password in the "free services" section. Though those bullets feel
kind of duplicative to me already.
Let's get right to the point of how to configure SMTP once you know
what you want. That section is pretty short anyway; and we can have
a first step direct the reader to our suggestions if they don't know
what service they want to use.
Also adjust the hierarchy of the headings: group the various
alternative email services under one heading, and group
troubleshooting together under an independent heading.
Also correct what we say about EMAIL_PORT: the Django default is
apparently 25, so if the provider *does* use the usual port 587
then we'll need the port to be set.
In our new system for updating realm settings, we don't need to create
separate functions to update the input elements for each feature.
Most of the work is done automatically by
`settings_org.sync_realm_settings`.
Usually, to debug a small change, you have to remove some tests from JSON
because of lack of support for comments in JSON. This commit allows to
ignore some tests by setting `"ignore" : true` in the bugdown fixtures.
Also, since this is only for while developing, the complete test suite will
throw an error if we leave an 'ignored' test in a commit.
This will allow realm admins to remove others from private stream to
which the realm administrator is not subscribed; this is important for
managing those streams, because previously nobody could remove users
from private streams that didn't have any realm administrators
subscribed.
This will allow realm admins to access subscribers of unsubscribed
private stream. This is a preparatory commit for letting realm admins
remove those users.
This will allow realm admins to update the names and descriptions of
private streams even if they are not subscribed, which fixes the buggy
behavior that previously nobody could(!).
The main point of this change is to streamline the core
code for input pills, and we use also modify user groups.
The main change to input_pill.js is that you now
configure a function called `create_item_from_text`, and
that can return an arbitrary object, and it just needs
a field called `display_value`.
Other changes:
* You now call `input.create(opts)` to create the
widget.
* There is no longer a cache, because we can
write smarter code in typeahead `source` functions
that exclude ids up front.
* There is no value/optinalKey complexity, because
the calling code can supply arbitrary objects and
do their own external data management on the pill
items.
* We eliminate `prependPill`.
* We eliminate `data`, `keys`, and `values`, and just
have `items`.
The summary already has a qualifier that basically says it shouldn't
matter for most people -- making it simultaneously the most
complicated bullet there, and among the least likely to matter.
And in fact, this requirement shouldn't matter for *anyone* when first
experimenting with Zulip. If certbot won't work in a given admin's
environment, and the available ways to get a cert aren't convenient,
they can always let the installer generate a self-signed cert to get
going, and circle back to the issue later.
So, make that option clear in the main requirements text, and then
just cut the whole bullet from the summary.
This further reduces the wall of text on the install instructions.
Simultaneously it lightens up the pressure on this summary to be quite
so terse; expand a couple of items into multiple bullets (yet with
fewer words!) for greater readability.
Now down to just 4 steps!
This version tries to prioritize: just two items that we really want
all admins to look at even if they aren't already mentally committed
to running a big production service and reading all the docs.
Namely, the two required in order to really try out Zulip effectively
with one's colleagues.
The screenshots weren't doing much good without being embedded in the
text... and in fact, looking at them for I think the first time,
they're badly out of date with the app. So cut them.
We might add screenshots later, but on the other hand if we do a good
job with the forms themselves, they should be superfluous.
This further shortens the wall of text inside the instructions.
Note that thanks to embedded reST, we now have the power to provide
custom anchors at section headings! Which in particular means this
link won't break if we later tweak the wording of this heading.
This helps shorten the wall of text between the start of the
instructions and the end. Conversely, now that there are fewer
followup steps, the troubleshooting section at the end isn't so
far away to point.
This flips the experimental `--express` option to be the default.
We retain the old behavior, where the script exits before
`initialize-database`, as an option `--no-init-db`; it might be useful
in e.g. a migration scenario (from a Zulip install elsewhere, or
another chat system) where the admin wants to set up the database
separately.
The install instructions are adjusted to match, getting shorter by two
steps and a bunch of words. I think this opens up opportunities to
refactor the text to simplify things further, too, but leaving that
for another commit.
Also tweak the "production" test suite to match.
This didn't really make much sense - it presented a quickstart summary
of the instructions as an item in the list of differences from the
Ubuntu case, plus that quickstart only incorporated one of the two
actual differences so it didn't work anyway. Given the networking
stuff, an actual quickstart isn't in the cards, so just cut that.
Also fix some small things while we're here.
From here on we start to authenticate uploaded file request before
serving this files in production. This involves allowing NGINX to
pass on these file requests to Django for authentication and then
serve these files by making use on internal redirect requests having
x-accel-redirect field. The redirection on requests and loading
of x-accel-redirect param is handled by django-sendfile.
NOTE: This commit starts to authenticate these requests for Zulip
servers running platforms either Ubuntu Xenial (16.04) or above.
Fixes: #320 and #291 partially.
Update version of Vagrant from 1.8.6 to 2.0.2.
Update version of VirtualBox from 5.1.8 to 5.2.6.
We needed to update Vagrant because older versions have networking
problems on modern Macs, resulting in weird `apt` errors when
provisioning.
Commit message and some text tweaked by tabbott.
This commit also adds a tool to push translation sources to Transifex.
This tool makes sure that we don't push mobile source file. Mobile
source file is supposed to be handled from Zulip-Mobile repo.
This quiets the last of the warnings that Sphinx was giving us about
documents not being in any toctree, now that we've explicitly told it
with `:orphan:` about the documents we intentionally don't link there.
Now that we have `eval_rst` and can explicitly exclude pages from the
toctree completely, we no longer need to set `includehidden`, and we
can return to using upstream's template.
(Meanwhile, our feature request upstream was successful! See
rtfd/sphinx_rtd_theme#485, which upstream implemented just a week
after we requested it. So that would have been another option.)
This reverts commit 11b8b8f48 "docs: Add rtd layout template."
This is what the Sphinx docs recommend when you actually don't want
the page to be included in navigation:
http://www.sphinx-doc.org/en/stable/markup/toctree.html
And now that we have `eval_rst`, we're able to take advantage of it!
One difference between doing this and the old way of making "hidden"
toctree entries is that with the latter, the "previous" and "next"
links at the bottom of each page would thread through the hidden
entries; which gets kind of confusing when they don't appear in the nav.
The careful testing with dozens of realms was a discrete thing
that happened, not a general state that the tool was in;
certainly not the last state it was known to be in, as the rest
of the sentence explains how it hasn't been so carefully tested
since then. So "as of" doesn't fit.
This is easy now, so make it known to admins who are looking for a
fast path for a test install.
Also totally cut the painfully complicated steps for generating a
self-signed cert by hand. Anyone who actually wants that can find a
hundred explanations on the Web, or can look at our script if they
want to specifically mirror how we do it (which is mercifully much
simpler than this.)
Take the core of the logic from how Debian generates the system's
/etc/ssl/certs/ssl-cert-snakeoil.pem ; that gives me more confidence
in the various config choices, and it also demonstrates a much cleaner
way to use the `openssl` tool. Also replace the outer shell logic for
CLI and logging with a cleaner version.
This allows us to selectively use some of the powerful features of
ReST which Python projects with high-quality documentation (like
Python core, and Django) rely on.
It's now January 2018, so we can delete this caveat, right?
Not quite yet -- the original post we link to now has an
update saying 2018-02-27. Let's make it less specific,
in case the date changes again.
Bulleted information instead of prose, huzzah! Also I think we need
to explain the options a bit right here, or at least link to where
they're documented somewhere. (If the list gets much longer, we'll
want to shift toward the latter.)
Also reorganize existing information a bit, and clean up a couple
of nits.
What I really want is to give these sections nice stable slugs
to put on the anchors and use as the URL fragment, independent of
any wording tweaks on the text headings. But I don't think we
have that feature with Markdown and our current docs infrastructure.
At least for Certbot, the brevity helps make this heading clearer
than the previous one.
Hopefully this version makes it somewhat clearer how the different
methods relate to each other, how to choose between them, what
`ZulipRemoteUserBackend` is for, and how the latter works.
Previous versions of these links were removed earlier today because
Docker upstream had broken them.
To see what the links had been intended to point to, I used the
Wayback Machine (web.archive.org/web), the super-handy project of the
Internet Archive. These are the current equivalents.
We made this change because users often unnecessarily click "Home"
first in their use of Zulip, because it seems appealing. While "All
messages" isn't quite precise (it doesn't include muted streams), it
does describe relatively simply the interleaved view that this
represents.
This commit leaves everything as "home" in the code, and only changes
user-visible strings and docs. Changing the code will be a big project;
there are hundreds of relevant occurrences in variable names, etc.
Further, we'll probably want to convert those various variable names
in different ways.
Tweaked by tabbott to extend the commit message and update a few comments.
Sphinx was displaying "WARNING: document isn't included in any toctree"
for files we just don't want in the TOC. We can hide them from the index,
but the rtd theme defaults to display hidden index entries in the nav bar.
This commit excludes these files from such warnings, and patches layout.html
so that hidden index entries stay hidden from the navigation sidebar.
This commit also moves password-strength.md under docs/production and
adds it as a hidden entry in production/index.rst.
Fixes#7417.
The readthedocs theme overrides a few settings in their layout template.
We might want to change some settings back to their default values.
This commit copies the original readthedocs layout file from
https://github.com/rtfd/sphinx_rtd_theme/blob/master/sphinx_rtd_theme/layout.html
to _templates/layout.html, and excludes it from lint and template checks.
Addresses #7417.
This doesn't touch the main path through the install docs; that will
see a broader rewrite soon as we make outbound email optional for a
nascent server, make the hostname and admin email into mandatory
installer flags, and then radically simplify the instructions by
removing mandatory editing of `settings.py` and folding most of what's
left into the installer.
This commit renames various source requirements files like `dev.txt`,
`mypy.txt` etc to `dev.in`, `mypy.in` etc and various locked requirements
files like `dev_lock.txt`, `mypy_lock.txt` etc to `dev.txt`, `mypy.txt`
etc. This will help in emphasizing to the user that *.in are actually
input to `update-locked-requirements` tool which should be run after
updating any of these.
This adds a few missing entries to the TOC, which hadn't made sense
back when Zulip's ReadTheDocs didn't have the new collapsing feature.
Tweaked by tabbott to also give the SSL certificates doc an
appropriate title for its new role.
Expand bullets "[+]" show up on locals instance, but they
do not show up online on readthedocs. The expand icon is valuable
to indicate that each section has more content.
The problem is readthedocs appends code to conf.py that overrides the
html_theme_options prior to a build. This commit prevents this from
happening so that collapse_navigation remains set to False.
In response to merged PR 7326.
This updates the developer documentation to recommend `git clone -c`
to set the git pull mode to be rebase by default in the repository
configuration.
The hope is that this will help minimize how often new contributors
end up accidentally messing up their PRs by using `git pull`.
Tweaked by tabbott to fix broken links and optimize the language.
Fixes: #7022.
This should get test-documentation and test-help-documentation passing
again after we moved everything around in the last commit.
There are a lot more absolute links (generally in code comments) to
update, but they are lower-priority and can be done in a follow-up
pass.
This commit helps reduce clutter on the navigation sidebar.
Creates new directories and moves relevant files into them.
Modifies index.rst, symlinks, and image paths accordingly.
This commit also enables expandable/collapsible navigation items,
renames files in docs/development and docs/production,
modifies /tools/test-documentation so that it overrides a theme setting,
Also updates links to other docs, file paths in the codebase that point
to developer documents, and files that should be excluded from lint tests.
Note that this commit does not update direct links to
zulip.readthedocs.io in the codebase; those will be resolved in an
upcoming follow-up commit (it'll be easier to verify all the links
once this is merged and ReadTheDocs is updated).
Fixes#5265.
The warning here means that the admin can't really act on this yet if
they want to disable email auth, which is likely among admins that
want to make any changes here. And for admins who don't, this is an
extra thing to read and make a decision about before they can get a
server running. See #6985.
Conversely, we already discuss auth backends right at the top of the
`prod-customize` doc, which is linked under "Next steps" at the end of
these instructions.
The warning about EmailAuthBackend is important; but we can move it to
the config file right next to the setting, and then it's available
right when it's actionable, which is if the admin is actually thinking
about changing the setting.
LXC on Debian does not ship with a default network config for
containers, so we now recommend setting up the lxc network config
manually.
Tweaked by tabbott to make these details much harder to miss.
The left navigation bar defaults to "sticking" to the screen as you
scroll. This commit adds the sticky_navigation setting to conf.py to
disable the default behavior.
This addresses part of #5265.
The control panel on the Google side doesn't seem to match the
instructions we have; it looks pretty 2017 to me, so I imagine
it's had a redesign since the instructions were written.
Also, in dev, EXTERNAL_HOST is now a port on zulipdev.com, not on
localhost.
Update these instructions for those developments, and edit lightly.
In dev, recommend setting in `dev_settings` instead of in
`prod_settings_template`; that feels to me a little more reflective of
the actual intent, and the effect should be equivalent.
In some minutes of searching yesterday afternoon on the docs site,
I couldn't find docs on how to set up our OAuth integrations for
the dev environment -- even though I was pretty sure they should
be there somewhere, because I'd just been told that. Wasn't until
I considered the problem fresh today, and grepped the docs source
instead, that I found them.
So, move them to a separate page so they're in the nav.
This means one fewer thing the admin typically needs to read, absorb,
and make a decision about at install time.
The one way this change could hypothetically cause trouble is if the
admin wants to keep subdomains of EXTERNAL_HOST out of ALLOWED_HOSTS.
But while the subdomains often won't exist as domain names, it's hard
to imagine the situation in which they would exist but be under
someone else's control, or be doing something other than serving
Zulip realms.
The "subdomain" label is redundant, to the extent it's even
accurate -- this is really just the URL we want to display,
which may or may not involve a subdomain. Similarly "external".
The former `external_api_path_subdomain` was never a path -- it's a
host, followed by a path, which together form a scheme-relative URL.
I'm not quite convinced that value is actually the right thing in
2 of the 3 places we use it, but fixing that can start by giving an
accurate name to the thing we have.
This was part of the logic to handle EXTERNAL_API_PATH varying.
But also it was already no longer used -- it was only ever passed
into template contexts, as `external_api_uri`, and it'd been
overtaken there by `external_api_uri_subdomain`.
So, update our dev docs to reflect that, and eliminate the variable.
http://zulip.readthedocs.io/en/latest/architecture-overview.html
mentions "Our mobile clients are separate code repositories: Android
and React Native iOS app. " However Zulip Mobile is the official
mobile Zulip client supporting both iOS and Android.
Edited by tabbott to also fix confusion about the desktop.
The Vagrant environment setup tutorial anchor links for the
Troubleshooting & Common Errors section don't work in GitHub flavored
markdown because the ampersand in the heading gets removed and the
resulting space is filled with a hyphen. (See here:
https://gist.github.com/asabaylus/3071099#gistcomment-1593627)
To make anchor links work with ReadTheDocs as well as with GitHub
markdown for the time being, this commit changes ampersand in the
heading to 'and'.
Fixes#7056.
Except in:
- docs/writing-bots-guide.md, because bots are supposed to be Python 2
compatible
- puppet/zulip_ops/files/zulip-ec2-configure-interfaces, because this
script is still on python2.7
- tools/lint
- tools/linter_lib
- tools/lister.py
For the latter two, because they might be yanked away to a separate repo
for general use with other FLOSS projects.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful. For configuration and the core
accept/reject logic, it's better to use units that are readily
understood. Switch to those.
I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses. And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft". So, "guesses" it is.
And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting. So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.
(Thanks to @YJDave for a prototype that the code changes in this
commit are based on.)
This has a ton of exclude rules, for two reasons:
(1) We haven't been particularly systematic about avoiding unnecessary
inline style in the past, so there's a lot of code we need to fix.
(2) There are cases where one wants to dynamically compute style
rules. For the latter category, ideally we'd figure out a way to
exclude these automatically (e.g. checking for mustache tags in the
style tag).
We just learned we should be using the "onlytranslated" mode of
Transifex. Since the command is getting a bit complex (and you need
to remember to run `makemessages` first), it makes sense to have a
tool for it.
This commit updates various places where check_send_message had
been previously recommended to recommend check_send_stream_message
for sending messages to a public stream.
This accumulates several changes in recent commits: decimal point
rather than comma, compact log level, and logger names, the latter
abbreviated `zr` in the case of `zulip.request`.
Collaboration in area label teams is only available to members of the
Zulip organization on GitHub. For non-members the related links are
not working, which can be confusing. Address this by explaining the
links won't work and also that anyone can join.
This is just enough of a quick fix to work with a stock Zulip 1.6
server. We should really also make this robust to arbitrary input
from the remote Zulip server, even though it'll be a little tedious.
Tim Abbott
Yago González
Greg Price
Rishi Gupta
Emilio Schadt
Preston Hansen
Steve Howell
Axel Tietje
Lyla Fischer
Shubham Dhama
Aman Agrawal
Puneeth Chaganti
Anurag Sharma
Rohitt Vashishtha
Balaji2198
sinwar
Akash Nimare
gooca
YJDave
feorlen
cg-cnu
Shivam Gera
cPhost
knovokresch
Vishnu Ks
Alicja Raszkowska
Aditya Bansal
rht
Mukul Agrawal
Umair Khan
XavierCooney
Jeremy Zhou
itstakenalr
Harshit Bansal
David Rosa Tamsen
Tommy Ip
derAnfaenger
Sarah
Mohd Ali Rizwi
KiranS
Eeshan Garg
Brock Whittaker
Rhea Parekh
Jack Zhang
Nikhil Kumar Mishra
Alexis La Porte
Priscilla
Garvit Dewan
Joshua Pan
neiljp (Neil Pilgrim)
Alena Volkova