Commit Graph

2674 Commits

Author SHA1 Message Date
Alex Vandiver 43d63bd5a1 puppet: Always set the RabbitMQ nodename to zulip@localhost.
This is required in order to lock down the RabbitMQ port to only
listen on localhost.  If the nodename is `rabbit@hostname`, in most
circumstances the hostname will resolve to an external IP, which the
rabbitmq port will not be bound to.

Installs which used `rabbit@hostname`, due to RabbitMQ having been
installed before Zulip, would not have functioned if the host or
RabbitMQ service was restarted, as the localhost restrictions in the
RabbitMQ configuration would have made rabbitmqctl (and Zulip cron
jobs that call it) unable to find the rabbitmq server.

The previous commit ensures that configure-rabbitmq is re-run after
the nodename has changed.  However, rabbitmq needs to be stopped
before `rabbitmq-env.conf` is changed; we use an `onlyif` on an `exec`
to print the warning about the node change, and let the subsequent
config change and notify of the service and configure-rabbitmq to
complete the re-configuration.
2022-01-25 01:48:02 +00:00
Alex Vandiver 694c4dfe8f puppet: Admit we leave epmd port 4369 open on all interfaces.
The Erlang `epmd` daemon listens on port 4369, and provides
information (without authentication) about which Erlang processes are
listening on what ports.  This information is not itself a
vulnerability, but may provide information for remote attackers about
what local Erlang services (such as `rabbitmq-server`) are running,
and where.

`epmd` supports an `ERL_EPMD_ADDRESS` environment variable to limit
which interfaces it binds on.  While this environment variable is set
in `/etc/default/rabbitmq-server`, Zulip unfortunately attempts to
start `epmd` using an explicit `exec` block, which ignores those
settings.

Regardless, this lack of `ERL_EPMD_ADDRESS` variable only controls
`epmd`'s startup upon first installation.  Upon reboot, there are two
ways in which `epmd` might be started, neither of which respect
`ERL_EPMD_ADDRESS`:

 - On Focal, an `epmd` service exists and is activated, which uses
   systemd's configuration to choose which interfaces to bind on, and
   thus `ERL_EPMD_ADDRESS` is irrelevant.

 - On Bionic (and Focal, due to a broken dependency from
   `rabbitmq-server` to `epmd@` instead of `epmd`, which may lead to
   the explicit `epmd` service losing a race), `epmd` is started by
   `rabbitmq-server` when it does not detect a running instance.
   Unfortunately, only `/etc/init.d/rabbitmq-server` would respects
   `/etc/default/rabbitmq-server` -- and it defers the actual startup
   to using systemd, which does not pass the environment variable
   down.  Thus, `ERL_EPMD_ADDRESS` is also irrelevant here.

We unfortunately cannot limit `epmd` to only listening on localhost,
due to a number of overlapping bugs and limitations:

 - Manually starting `epmd` with `-address 127.0.0.1` silently fails
   to start on hosts with IPv6 disabled, due to an Erlang bug ([1],
   [2]).

 - The dependencies of the systemd `rabbitmq-server` service can be
   fixed to include the `epmd` service, and systemd can be made to
   bind to `127.0.0.1:4369` and pass that socket to `epmd`, bypassing
   the above bug.  However, the startup of this service is not
   guaranteed, because it races with other sources of `epmd` (see
   below).

 - Any process that runs `rabbitmqctl` results in `epmd` being started
   if one is not currently running; these instances do not respect any
   environment variables as to which addresses to bind on.  This is
   also triggered by `service rabbitmq-server status`, as well as
   various Zulip cron jobs which inspect the rabbitmq queues.  As
   such, it is difficult-to-impossible to ensure that some other
   `epmd` process will not win the race and open the port on all
   interfaces.

Since the only known exposure from leaving port 4369 open is
information that rabbitmq is running on the host, and the complexity
of adjusting this to only bind on localhost is high, we remove the
setting which does not address the problem, and document that the port
is left open, and should be protected via system-level or
network-level firewalls.

[1]: https://bugs.launchpad.net/ubuntu/+source/erlang/+bug/1374109
[2]: https://github.com/erlang/otp/issues/4820
2022-01-25 01:46:51 +00:00
Alya Abbott 669010494e portico: Update contributor count from 700 to 1000.
Note: I did not check whether we have numbers other than 700 that also
need to be updated.
2022-01-24 12:41:49 -08:00
Anders Kaseorg a58a71ef43 Remove Ubuntu 18.04 support.
As a consequence:

• Bump minimum supported Python version to 3.7.
• Move Vagrant environment to Debian 10, which has Python 3.7.
• Move CI frontend tests to Debian 10.
• Move production build test to Debian 10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 17:26:14 -08:00
Alex Vandiver be1c4c2bd8 docs: Mention Camo does not use a local Smokescreen in the proxies docs.
This documents the new behaviour in d328d3dd4d.
2022-01-21 15:57:27 -08:00
Eeshan Garg 9e1fd26125 docs: Make general improvements to our billing docs.
With a few wording tweaks from tabbott to the "Upgrading a Zulip
organization" section.
2022-01-21 14:21:02 -08:00
rht 42f46a78e9 docs: Fix grammar problems found by LanguageTool.
With tweaks to security-model.md by tabbott to expand the SSO acronym.

Ignored, but still needs discussion on whether we should exclude this
rule:

```
The word ‘install’ is not a noun.
  ✗ ...ble to connect to the client during the install process:  So you'll need to shut down a...
                                               ^^^^^^^
  ✓ ...ble to connect to the client during the installation process:  So you'll need to shut down a...
  A_INSTALL: a/the + install

The word ‘install’ is not a noun.
  ✗ ...detected at install time will cause the install to abort. If you already have PostgreSQ...
                                               ^^^^^^^
  ✓ ...detected at install time will cause the installation to abort. If you already have PostgreSQ...
  A_INSTALL: a/the + install
```
2022-01-21 14:02:14 -08:00
Alya Abbott ca311e83c8 docs: Fix typos in GSoC guide. 2022-01-21 13:38:30 -08:00
Alya Abbott 19154f81c0
docs: Clarify purpose of zulip-announce. 2022-01-19 15:34:24 -08:00
Eeshan Garg fd303e3b1b docs: Update the release checklist for our PyPI packages. 2022-01-19 13:59:13 -08:00
Alex Vandiver 5f237cb34e puppet: Document that upgrades from Git require 3GB.
The step of rebuilding static assets using webpack requires more than
2G of RAM.
2022-01-19 12:36:44 -08:00
Abhijeet Prasad Bodas 12eb52221b docs: Rename `contributing/gsoc-ideas` -> `contributing/gsoc`.
This page contains a lot of other material related to GSoC than
just project ideas.
We would also want to add a redirect from the old URL to the new
one from the RTD admin page.
2022-01-19 11:35:33 -08:00
Alya Abbott 3659d95092 developer docs: Update GSoC documentation. 2022-01-18 21:16:18 -08:00
Rishabh-792 177931a23d doc: Fix typos in accessibility doc.
Hyphenated open source to open-source.

Capitalized aXe to Axe.
2022-01-11 15:41:08 -08:00
Rishabh-792 6cd8e088e9 doc: Fix typo in zulipbot-usage doc.
Fix a spelling mistake in the zulipbot doc.
2022-01-11 15:40:06 -08:00
Rishabh-792 1ec018d237 doc: Fix typos in code reviewing doc.
Made some spelling and grammatical changes.
2022-01-11 15:40:05 -08:00
Alex Vandiver d328d3dd4d puppet: Allow routing camo requests through an outgoing proxy.
Because Camo includes logic to deny access to private subnets, routing
its requests through Smokescreen is generally not necessary.  However,
it may be necessary if Zulip has configured a non-Smokescreen exit
proxy.

Default Camo to using the proxy only if it is not Smokescreen, with a
new `proxy.enable_for_camo` setting to override this behaviour if need
be.  Note that that setting is in `zulip.conf` on the host with Camo
installed -- not the Zulip frontend host, if they are different.

Fixes: #20550.
2022-01-07 12:08:10 -08:00
Alex Vandiver 2c5fc1827c puppet: Standardize what values are bools, and what true is.
For `no_serve_uploads`, `http_only`, which previously specified
"non-empty" to enable, this tightens what values are true.  For
`pgroonga` and `queue_workers_multiprocess`, this broadens the
possible values from `enabled`, and `true` respectively.
2022-01-07 12:08:10 -08:00
Anders Kaseorg 1696144df7 docs: Consistently hyphenate “self-host” and “self-service”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-05 16:21:35 -08:00
Alex Vandiver 6218ed91c2 puppet: Use lazy-apps and uwsgi control sockets for rolling reloads.
Restarting the uwsgi processes by way of supervisor opens a window
during which nginx 502's all responses.  uwsgi has a configuration
called "chain reloading" which allows for rolling restart of the uwsgi
processes, such that only one process at once in unavailable; see
uwsgi documentation ([1]).

The tradeoff is that this requires that the uwsgi processes load the
libraries after forking, rather than before ("lazy apps"); in theory
this can lead to larger memory footprints, since they are not shared.
In practice, as Django defers much of the loading, this is not as much
of an issue.  In a very basic test of memory consumption (measured by
total memory - free - caches - buffers; 6 uwsgi workers), both
immediately after restarting Django, and after requesting `/` 60 times
with 6 concurrent requests:

                      |  Non-lazy  |  Lazy app  | Difference
    ------------------+------------+------------+-------------
    Fresh             |  2,827,216 |  2,870,480 |   +43,264
    After 60 requests |  3,332,284 |  3,409,608 |   +77,324
    ..................|............|............|.............
    Difference        |   +505,068 |   +539,128 |   +34,060

That is, "lazy app" loading increased the footprint pre-requests by
43MB, and after 60 requests grew the memory footprint by 539MB, as
opposed to non-lazy loading, which grew it by 505MB.  Using wsgi "lazy
app" loading does increase the memory footprint, but not by a large
percentage.

The other effect is that processes may be served by either old or new
code during the restart window.  This may cause transient failures
when new frontend code talks to old backend code.

Enable chain-reloading during graceful, puppetless restarts, but only
if enabled via a zulip.conf configuration flag.

Fixes #2559.

[1]: https://uwsgi-docs.readthedocs.io/en/latest/articles/TheArtOfGracefulReloading.html#chain-reloading-lazy-apps
2022-01-05 14:48:52 -08:00
BIKI DAS 42dd58cffe
docs: Fix a few typos in documentation. 2021-12-28 09:36:59 -08:00
BIKI DAS c1134a8bda
docs: Fix "should should" typo. 2021-12-28 09:19:04 -08:00
Anders Kaseorg 1d3520db12 webhooks: Remove space from UptimeRobot.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-22 14:05:17 -08:00
Anders Kaseorg 68c99511a2 webhooks: Fix TeamCity capitalization.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-22 14:05:17 -08:00
Anders Kaseorg 65868b09eb webhooks: Add missing space in Review Board.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-22 14:05:17 -08:00
Anders Kaseorg c02c053ec3 webhooks: Fix Mailchimp capitalization.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-22 14:05:17 -08:00
Anders Kaseorg cd8a01587b webhooks: Fix Jotform capitalization.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-22 14:05:17 -08:00
Anders Kaseorg 3ca2f8ca1e webhooks: Fix Clubhouse capitalization.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-22 14:05:17 -08:00
Anders Kaseorg 517ddbc9e6 setup-advanced: Remove misleading python3 symlink suggestion.
One should never have to manually symlink things in /usr/bin,
especially with -f.  That should be managed by the system package
manager.  Indeed, on CentOS 7 and 8, one can simply install the
python3 package and get a working /usr/bin/python3.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-12-16 17:30:04 -08:00
Alya Abbott aaf1258de2 developer docs: Tweak ToS for push notifications wording. 2021-12-14 14:13:34 -08:00
Tim Abbott ee77c6365a portico: Use /help/ style pages for displaying policies.
This replaces the TERMS_OF_SERVICE and PRIVACY_POLICY settings with
just a POLICIES_DIRECTORY setting, in order to support settings (like
Zulip Cloud) where there's more policies than just those two.

With minor changes by Eeshan Garg.
2021-12-10 17:56:12 -08:00
Alex Vandiver 01e8f752a8 puppet: Use certbot package timer, not our own cron job.
The certbot package installs its own systemd timer (and cron job,
which disabled itself if systemd is enabled) which updates
certificates.  This process races with the cron job which Zulip
installs -- the only difference being that Zulip respects the
`certbot.auto_renew` setting, and that it passes the deploy hook.
This means that occasionally nginx would not be reloaded, when the
systemd timer caught the expiration first.

Remove the custom cron job and `certbot-maybe-renew` script, and
reconfigure certbot to always reload nginx after deploying, using
certbot directory hooks.

Since `certbot.auto_renew` can't have an effect, remove the setting.
In turn, this removes the need for `--no-zulip-conf` to
`setup-certbot`.  `--deploy-hook` is similarly removed, as running
deploy hooks to restart nginx is now the default; pass
`--no-directory-hooks` in standalone mode to not attempt to reload
nginx.  The other property of `--deploy-hook`, of skipping symlinking
into place, is given its own flog.
2021-12-09 13:47:33 -08:00
Eeshan Garg 3bab91079f external links: Migrate the rest of /developer-community links.
We recently changed /developer-community to /development-community.
Now that this change is in production, we can also migrate the
external links in our ReadTheDocs documentation.
2021-12-09 12:14:26 -08:00
Alex Vandiver cb2d0ff32b postgresql: Support replication on PostgreSQL >= 11, document.
PostgreSQL 11 and below used a configuration file names
`recovery.conf` to manage replicas and standbys; support for this was
removed in PostgreSQL 12[1], and the configuration parameters were
moved into the main `postgresql.conf`.

Add `zulip.conf` settings for the primary server hostname and
replication username, so that the complete `postgresql.conf`
configuration on PostgreSQL 14 can continue to be managed, even when
replication is enabled.  For consistency, also begin writing out the
`recovery.conf` for PostgreSQL 11 and below.

In PostgreSQL 12 configuration and later, the `wal_level =
hot_standby` setting is removed, as `hot_standby` is equivalent to
`replica`, which is the default value[2].  Similarly, the
`hot_standby = on` setting is also the default[3].

Documentation is added for these features, and the commentary on the
"Export and Import" page referencing files under `puppet/zulip_ops/`
is removed, as those files no longer have any replication-specific
configuration.

[1]: https://www.postgresql.org/docs/current/recovery-config.html
[2]: https://www.postgresql.org/docs/12/runtime-config-wal.html#GUC-WAL-LEVEL
[3]: https://www.postgresql.org/docs/12/runtime-config-replication.html#GUC-HOT-STANDBY
2021-12-03 16:32:41 -08:00
Emilio López baea14ee57 docs: Clarify use of `loadbalancer.ips` when using a reverse proxy.
When Zulip is run behind one or more reverse proxies, you must
configure `loadbalancer.ips` so that Zulip respects the client IP
addresses found in the `X-Forwarded-For` header. This is not
immediately clear from the documentation, so this commit makes it more
clear and augments the existing examples to showcase this need.

Fixes: #19073
2021-12-03 13:59:31 -08:00
Alex Vandiver ab8be84b36 docs: Secret reading is done using RawConfigParser, not ConfigParser.
ConfigParser makes `%` signs require escaping, which is why it is not
used in Zulip, particularly for secrets.
2021-12-02 15:25:04 -08:00
Alex Vandiver 54d037f24a version: Update version and changelog after 4.8 release. 2021-12-01 23:42:11 +00:00
AEsping f6c4f17900 dev docs: Update Jinja translation tag link.
Updates the link to Jinja i18n extension for auto-translation.
2021-11-30 14:36:29 -08:00
AEsping 828313b54a dev docs: Update Jinja translation tag link.
Updates the link to Jinja i18n extension for auto-translation.
2021-11-30 14:36:29 -08:00
AEsping 704c9609ee dev docs: Update Tig link.
Updates the link to the Tig git visualizer.
2021-11-30 14:36:29 -08:00
AEsping 11f2575c31 dev docs: Update "Solo" link.
Fixes the link to "El adveribo <<solo>> y los pronombres
demonstrativos, sin tilde."
2021-11-30 14:36:29 -08:00
AEsping 510b8867a6 dev docs: Update Neil Green link in the reading list.
Fixes the link to the Neil Green presentation on TypeScript
vs Coffee Script vs ES6.

This is a change from slides to a video becasue the slides are
no longer available.
2021-11-30 14:36:29 -08:00
AEsping 55f9178506 dev docs: Update Black link.
Updates the link to the editior integration for Black.
2021-11-30 14:36:29 -08:00
AEsping 5410009a88 prod docs: Update BBB configuration link.
Updates the Big Blue Button customization link for
extracting shared secrets.
2021-11-30 14:36:29 -08:00
Mateusz Mandera 8c1a6f4bba docs: Suggest updating settings.py in OIDC instructions.
OIDC config features a get_secret call (so it requires adding an import)
as well as having a bunch of its instructions in the form of comments on
the various keys of the config dict - thus users should really update
settings.py to fetch all of that.
2021-11-29 15:52:52 -08:00
Alex Vandiver 0ae375e0f9 ci: Test upgrades from the latest minor release. 2021-11-25 08:00:34 -08:00
AEsping 6ad1c5c8ed docs:: Update GSoC application tips.
- Add missing link for GitHub.
- Fix broken links to Matt Ringel's blog post.
- Add link to Julia Evans blog post.
- Add section heading for "Questions Are Important."
- Rearrange some content to fit with new section heading.

With additional tweaks from tabbott:
* Avoid linking to chat.zulip.org not via our documentation.
* Avoid the CZO abbreviation.
2021-11-23 16:05:33 -08:00
Alex Vandiver b982222e03 camo: Replace with go-camo implementation.
The upstream of the `camo` repository[1] has been unmaintained for
several years, and is now archived by the owner.  Additionally, it has
a number of limitations:
 - It is installed as a sysinit service, which does not run under
   Docker
 - It does not prevent access to internal IPs, like 127.0.0.1
 - It does not respect standard `HTTP_proxy` environment variables,
   making it unable to use Smokescreen to prevent the prior flaw
 - It occasionally just crashes, and thus must have a cron job to
   restart it.

Swap camo out for the drop-in replacement go-camo[2], which has the
same external API, requiring not changes to Django code, but is more
maintained.  Additionally, it resolves all of the above complaints.

go-camo is not configured to use Smokescreen as a proxy, because its
own private-IP filtering prevents using a proxy which lies within that
IP space.  It is also unclear if the addition of Smokescreen would
provide any additional protection over the existing IP address
restrictions in go-camo.

go-camo has a subset of the security headers that our nginx reverse
proxy sets, and which camo set; provide the missing headers with `-H`
to ensure that go-camo, if exposed from behind some other non-nginx
load-balancer, still provides the necessary security headers.

Fixes #18351 by moving to supervisor.
Fixes zulip/docker-zulip#298 also by moving to supervisor.

[1] https://github.com/atmos/camo
[2] https://github.com/cactus/go-camo
2021-11-19 15:58:26 -08:00
Alex Vandiver c33562f0a8 puppet: Default to installing smokescreen on application frontends.
This is an additional security hardening step, to make Zulip default
to preventing SSRF attacks.  The overhead of running Smokescreen is
minimal, and there is no reason to force deployments to take
additional steps in order to secure themselves against SSRF attacks.

Deployments which already have a different external proxy configured
will not gain a local Smokescreen installation, and running without
Smokescreen is supported by explicitly unsetting the `host` or `port`
values in `/etc/zulip/zulip.conf`.
2021-11-19 15:29:28 -08:00
Sahil Batra 5024ba98cd misc: Rename "dark mode" to "dark theme".
Fixes part of #20228.
2021-11-18 16:57:11 -08:00
Ankur c578f186b6
docs: Change chat.zulip.org links to zulip.com/developer-community.
This helps increase the probability that folks read the guidelines for how the
chat.zulip.org community works and what streams to use before arriving there.

Fixes #19827.
2021-11-17 22:25:55 -08:00
Tim Abbott c64eebcc9e docs: Mention server_name in existing server nginx docs.
As discussed in the issue, it'd be a helpful hint for folks doing this
to mention this detail.

Fixes #16705.
2021-11-17 16:53:11 -08:00
Tim Abbott cc11743047 docs: Advertise docker-zulip on rqeuirements page.
We also remove coverage of 64-bit, since it's no longer a relevant
detail in 2021.

Fixes #18534.
2021-11-17 16:48:51 -08:00
Aman Agrawal fc42568248 docs: Separate recommend and vagrant setup sections for dev setup.
This will possibly direct more users towards WSL setup on Windows.
2021-11-15 10:58:08 -08:00
Alex Vandiver 544e8c569e install: Switch default to PostgreSQL 14. 2021-11-08 18:21:46 -08:00
YashRE42 40444cf415 docs: Update all links to Django docs to point to version /3.2/.
Previously, our docs had links to various versions of the Django docs,
eg https://docs.djangoproject.com/en/1.10/topics/migrations/ and
https://docs.djangoproject.com/en/2.0/ref/signals/#post-save, opening
a link to a doc with an outdated Django version would show a warning
"This document is for an insecure version of Django that is no longer
supported. Please upgrade to a newer release!".

This commit uses a search with the regex
"docs.djangoproject.com/en/([0-9].[0-9]*)/" and replaces all matches
inside the /docs/ folder with "docs.djangoproject.com/en/3.2/".

All the new links in this commit have been generated by the above
replace and each link has then been manually checked to ensure that
(1) the page still exists and has not been moved to a new location
(and it has been found that no page has been moved like this), (2)
that the anchor that we're linking to has not been changed (and it has
been found that this happened once, for https://docs.djangoproject.com
/en/1.8/ref/django-admin/#runserver-port-or-address-port, where
/#runserver-port-or-address-port was changed to /#runserver).
2021-11-08 18:06:16 -08:00
YashRE42 87dd2a15c2 docs: Re-add accidentally deleted link definitions in advanced setup.
In commit f6c78a35a4 we accidentally
deleted these link definitions, probably thinking that the end of the
markdown file would be the same as the end of the rendered doc. This
broke the links `[cloning your fork of the Zulip
repository][zulip-rtd-git-cloning]` and `[connecting the Zulip
upstream repository][zulip-rtd-git-connect]`.

This commit fixes things by adding back the definitions.
2021-11-08 18:06:16 -08:00
Vishnu KS 6c06858e02 billing: Migrate to Stripe hosted checkout page. 2021-11-05 17:23:10 -07:00
Vishnu KS 1a1b9b28ff corporate: Store the Stripe API version. 2021-11-05 17:23:10 -07:00
Anders Kaseorg 6fa86385aa docs: Enable sphinx_rtd_theme as an extension.
This has no effect at present, but it’s documented as necessary to
enable localization of theme strings in translated output, so maybe
it’ll be relevant some day.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-11-04 11:55:15 -07:00
Anders Kaseorg 03a7d0c053 docs: Remove html_theme_path override.
This block has been obsolete since at least sphinx-rtd-theme 0.2.5.
Removing it fixes the heading permalink icon in a local build to be
consistent with the one shown on Read the Docs, and has no other
effect.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-11-04 11:55:15 -07:00
Anders Kaseorg 8d4568140c docs: Enable collapse_navigation for local builds.
This makes local builds significantly faster, while leaving the fancy
navigation enabled on Read the Docs where it’s important.

https://sphinx-rtd-theme.readthedocs.io/en/stable/configuring.html#confval-collapse_navigation

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-11-04 11:55:15 -07:00
Anders Kaseorg 177dde15eb docs: Clean Sphinx configuration file.
Delete all the boilerplate comments and unused options generated by
the ancient version of Sphinx that originally generated this file,
leaving a file that one can realistically read.  Leave some links for
those who want to read about all the options that exist.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-11-04 11:55:15 -07:00
Lauryn Menard dd5cad549c documentation: Expand documentation processes in new feature tutorial.
Expands the developer tutorial 'Writing a new application feature' to
include more detail about the documentation aspects of adding a new
feature. Adds references to specific files that will be impacted and
highlights API changes as well as writing `/help` articles.
2021-11-02 14:03:34 -07:00
AEsping 6963876e22 docs: Update instructions for Windows 10 installation.
This commit includes the following changes.
- Adds the definition of the WSL acronym.
- Adds information for changing BIOS settings
in order to enable machine virtulization.
- Fixes a broken link to Microsoft WSL installation instructions.
- Adds a reminder to create a new SSH key before connecting to
GitHub.
- Removes the step to install Ubuntu.  This step is now
included in the standard installation.
- Reminds the user to launch Ubuntu as and administrator.
- Switches the text editor in the example to nano from vim.
Nano is included with the wsl installation, and is easier for
most people to use than vim.
- Adds a separate step to fork the Zulip/Zulip repository.
- Adds the bash command to open VS Code and
reminds the user to install the relevant extensions.

With various formatting tweaks by tabbott.
2021-11-02 13:47:11 -07:00
Tim Abbott 61c0825036 docs: Extend Certbot troubleshooting documentation.
This should help folks who have problems with Certbot renewal; we had
a couple reported this week which I think were both caused by firewall
issues.
2021-11-01 18:11:31 -07:00
Mateusz Mandera 3d731de3f1 docs: Add disabling of Force POST Binding to Keyclock SLO instructions.
This needs to be disabled, because python3-saml only supports the
Redirect binding. This step was forgotten in the original writing of
this doc.
2021-11-01 11:02:51 -07:00
Mateusz Mandera 4105ccdb17 saml: Implement IdP-initated logout for Keycloak.
Fixes #13948.
2021-10-27 13:13:55 -07:00
Jonny Tran ddf4053a73 composebox_typeahead: Remove redundant options from the `/` typeahead.
Removes the `/day` and `/night` options from the typeahead menu while
still allowing the commands to be used. Typing `/day` and `/night`
will now suggest `/light` and `/dark`, respectively. Also changes the
`Dark mode` and `Light mode` popups that appear after using the
corresponding command.

Fixes #18318.
2021-10-21 10:50:27 -07:00
YashRE42 f6c78a35a4 docs: Remove AWS Cloud9 setup instructions.
Due to the fact that it's not possible to run the development
environment on a t2.micro (1 GiB RAM + 1 vCPU), which is what is
available from the free tier, the fact that signing up require a
credit/debit card and can take up to 24 hours, and that it is quite
easy to unintentionally exceed the free tier resources when expanding
or upgrading, it is no longer feasible to develop on cloud9. As such,
we should not recommend it in out setup docs.
2021-10-18 10:38:00 -07:00
YashRE42 c8353a1820 minor: Add missing word to "sending messages" doc. 2021-10-18 10:38:00 -07:00
YashRE42 94167c6db9 docs: Update wsl install guide link.
The previous link "/wsl/wsl2-install" leads to a 404 page which
recommends "/wsl/install". This commit updates the link to
"/wsl/install".

The previous link has been giving a redirect since at least May 23,
2020.
2021-10-18 10:38:00 -07:00
Eeshan Garg d312432ea8 docs: Update links to zulip-announce mailing list.
The links we have now redirect to "My groups" and not to our
Google group. Also, the RSS feed is no longer supported by Google,
so we should no longer link to it.

Fixes #19560.
2021-10-14 16:51:36 -07:00
Tim Abbott a87bbe22be docs: Document how to test the polling email gateway configuration.
We don't recommend this configuration, so it's rarely used, but it's
worth a few minutes to have nice documentation.

Fixes #19565.
2021-10-14 14:50:52 -07:00
Mateusz Mandera 16321327f9 docs: Improve documentation on syncing UserProfile bool flags via ldap.
It's better to explicitly list the possibilities. Also, the
recommendation regarding is_active should be changed to a strict
"Don't", as Subscription.is_user_active is a denormalized field and
flipping a user's is_active will cause inconsistent state by leaving
Subscriptions unupdated. Given that similar things can be introduced in
the future for any other flag not officially supported by having a
setter, the recommendation should "Don't" in general.
2021-10-06 17:38:56 -07:00
Tim Abbott f01ee5f17a lint: Fix line wrapping of new schema migrations doc. 2021-10-04 22:16:55 -07:00
Alex Vandiver 36f92d6843 docs: Document what to do when releases need migrations. 2021-10-04 22:10:24 -07:00
Alex Vandiver 3922b171a8 version: Update version and changelog after 4.7 release. 2021-10-04 21:52:52 +00:00
Tim Abbott 67fdbbe5fd docs: Inline some upgrade instructions.
It feels like the "Same as" content was unnecessarily requiring the
user to bounce around in these cases.

(I've left the "Same as" text for the Ubuntu ones, where it's two
steps in a row to follow).
2021-10-01 11:08:14 -07:00
Gaurav Pandey 77d1c4361b docs: Document upgrade steps from buster to bullseye.
Fixes #17863.
2021-10-01 11:08:14 -07:00
Anders Kaseorg 2f993f1a79 install-node: Stop using NVM.
NVM doesn’t check hashes or signatures and really just adds
complexity we don’t need.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-24 06:58:32 -07:00
Tim Abbott 10c47b5d6c version: Update version and changelog after 4.6 release. 2021-09-23 16:19:57 -07:00
Anders Kaseorg 6367384510 docs: Recommend Docker for all macOS users.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-17 21:18:38 -07:00
Tim Abbott ea6a40661c migrations: Add progress output to confirmation backfill.
This makes it a lot easier to estimate how long this migration will
take to run. Additionally, document this migration in the upgrade notes.
2021-09-17 11:23:32 -07:00
Mateusz Mandera 8ad7520180 ldap: Support alternative attrs to userAccountControl.
Fixes #17456.

The main tricky part has to do with what values the attribute should
have. LDAP defines a Boolean as
Boolean = "TRUE" / "FALSE"
so ideally we'd always see exactly those values. However,
although the issue is now marked as resolved, the discussion in
https://pagure.io/freeipa/issue/1259 shows how this may not always be
respected - meaning it makes sense for us to be more liberal in
interpreting these values.
2021-09-16 13:29:13 -07:00
Mateusz Mandera b32450f98e docs: Remove mentions of some ldap features being added in 2.0.
2.0 is old enough that explicitly mentioning when these features were
implemented isn't particularly useful and adds clutter.
2021-09-16 13:29:13 -07:00
Eeshan Garg 3b1cb0b25a docs: Indicate latest Zulip version in installation and upgrade docs.
With copy-editing from tabbott, and also a migration to use
LATEST_RELEASE_VERSION, which will be correct even on the /latest/
paths.

Fixes #19695.
2021-09-10 14:50:48 -07:00
Tim Abbott 908e1e6d19 docs: Run prettier on authentication-methods.md.
The new sections of this file were rebased past the addition of
running prettier on these files.
2021-09-10 13:23:11 -07:00
Philipp Seßner a8797b4d9f help: Document how to configure Keycloak as the SAML IdP. 2021-09-10 13:04:04 -07:00
Anders Kaseorg 5360bd76ac help: Add newline after admonition header for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-10 10:39:25 -07:00
Gaurav Pandey 502697d239 docs: Add documentation for bullseye support.
The support for bullseye was added in #17951
but it was not documented as bullseye was
frozen and did not have proper configuration
files, hence wasn't documented.

Since now bullseye is released as a stable
version, it's support can be documented.
2021-09-09 11:05:16 -07:00
Anders Kaseorg 1fc801a43f docs: Add missing space in “feature set”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 15:31:28 -07:00
Anders Kaseorg 85fec56697 docs: Update links for zulip-mobile branch rename.
GitHub redirects these, but we should use the canonical URLs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 15:30:37 -07:00
Anders Kaseorg a4dbc1edd4 docs: Format Markdown with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg 35c1c8d41b docs: Apply sentence single-spacing from Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg 915884bff7 docs: Apply bullet style changes from Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg 6145fdf678 docs: Rewrap to avoid line breaks in inline code spans.
This works around https://github.com/prettier/prettier/issues/11372.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg a6e01b35fc docs: Fix `pip compile` typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg f4d2d199e2 docs: Display `main` branch name as inline code.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-06 12:56:35 -07:00
Anders Kaseorg 646c04eff2 Rename default branch to ‘main’.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-06 12:56:35 -07:00
Tim Abbott 1c3517a5de docs: Document ReadTheDocs preview builds. 2021-09-06 12:40:36 -07:00
Anders Kaseorg 1ce12191aa docs: Update links for other repository branch renames.
GitHub redirects these, but we should use the canonical URLs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 15:28:01 -07:00
Anders Kaseorg 0dcb84a2e8 docs: Follow Sphinx 4 rename of master_doc to root_doc.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 14:24:16 -07:00
Alex Vandiver 7d7d727865 tools: Switch to download.zulip.com from www.zulip.org. 2021-08-26 17:21:31 -07:00
Alex Vandiver 710a92f48c tools: Add a tool to upload release tarballs to S3. 2021-08-26 17:21:31 -07:00
Alex Vandiver faf71eea41 upgrade-postgresql: Do not remove other supervisor configs.
We previously used `zulip-puppet-apply` with a custom config file,
with an updated PostgreSQL version but more limited set of
`puppet_classes`, to pre-create the basic settings for the new cluster
before running `pg_upgradecluster`.

Unfortunately, the supervisor config uses `purge => true` to remove
all SUPERVISOR configuration files that are not included in the puppet
configuration; this leads to it removing all other supervisor
processes during the upgrade, only to add them back and start them
during the second `zulip-puppet-apply`.

It also leads to `process-fts-updates` not being started after the
upgrade completes; this is the one supervisor config file which was
not removed and re-added, and thus the one that is not re-started due
to having been re-added.  This was not detected in CI because CI added
a `start-server` command which was not in the upgrade documentation.

Set a custom facter fact that prevents the `purge` behaviour of the
supervisor configuration.  We want to preserve that behaviour in
general, and using `zulip-puppet-apply` continues to be the best way
to pre-set-up the PostgreSQL configuration -- but we wish to avoid
that behaviour when we know we are applying a subset of the puppet
classes.

Since supervisor configs are no longer removed and re-added, this
requires an explicit start-server step in the instructions after the
upgrades complete.  This brings the documentation into alignment with
what CI is testing.
2021-08-24 19:00:58 -07:00
Anders Kaseorg fdb7ec8c9e docs: Avoid [GitHub] as an internal Markdown link reference name.
To avoid confusing the linter later when Prettier lowercases these.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 0e4a30daad reading-list: Inline links.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 32da7e3eb2 docs: Adjust colon fence for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 7b3d4ff1de docs: Escape asterisks for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 0a3e022376 docs: Adjust list item indentation for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 4bfffc9f74 docs: Fix list item indentation mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 962f14995e docs: Fix “sinternet” typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg fe3db63381 docs: Add missing blockquote.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg b53e676860 docs: Convert reST index pages to Markdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 06:22:42 -07:00
Anders Kaseorg b29b6f6526 docs: Add syntax highlighting languages to code blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 06:14:47 -07:00
Anders Kaseorg 672358c5da docs: Migrate from recommonmark to MyST-Parser.
Recommonmark is no longer maintained, and MyST-Parser is much more
complete.

https://myst-parser.readthedocs.io/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 8f2827a65d docs: Remove trailing newlines from code blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg cb61ea69c6 docs: Fix misaligned Markdown source indentation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 25c6d5c99c docs: Fix code span syntax in embedded reST block.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg c3646ec67f docs: Move authentication-methods#ldap anchor to appropriate heading.
Commit 30eaed0378 (#15001) incorrectly
inserted a different section between the anchor and the heading.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Mateusz Mandera fdbde59b07 rate_limit: Add management command to reset auth rate limit.
The auth attempt rate limit is quite low (on purpose), so this can be a
common scenario where a user asks their admin to reset the limit instead
of waiting. We should provide a tool for administrators to handle such
requests without fiddling around with code in manage.py shell.
2021-08-19 00:45:17 -07:00
Alex Vandiver ec354ee013 changelog: Add 2.1.8 release notes. 2021-08-12 01:06:15 +00:00
Alex Vandiver e94b6afb00 nagios: Remove broken check_email_deliverer_* checks and related code.
These checks suffer from a couple notable problems:
 - They are only enabled on staging hosts -- where they should never
   be run.  Since ef6d0ec5ca, these supervisor processes are only
   run on one host, and never on the staging host.
 - They run as the `nagios` user, which does not have appropriate
   permissions, and thus the checks always fail.  Specifically,
   `nagios` does not have permissions to run `supervisorctl`, since
   the socket is owned by the `zulip` user, and mode 0700; and the
   `nagios` user does not have permission to access Zulip secrets to
   run `./manage.py print_email_delivery_backlog`.

Rather than rewrite these checks to run on a cron as zulip, and check
those file contents as the nagios user, drop these checks -- they can
be rewritten at a later point, or replaced with Prometheus alerting,
and currently serve only to cause always-failing Nagios checks, which
normalizes alert failures.

Leave the files installed if they currently exist, rather than
cluttering puppet with `ensure => absent`; they do no harm if they are
left installed.
2021-08-03 16:07:13 -07:00
Tim Abbott f84703bc48 test-backend: Default to running all tests.
This switches the default of the previous --nonfatal-errors option,
and also moves to use the `-x | --stop` names used by the nose test
framework.
2021-07-30 16:18:42 -07:00
Tim Abbott 665f479f4d docs: Update changelog since 4.0 release. 2021-07-29 14:54:17 -07:00
Tim Abbott e4667c2af6 docs: Add developer documentation for Python support. 2021-07-28 11:23:27 -07:00
sahil839 9ef8727a4a docs: Fix new_feature_tutorial to replace add_emoji_by_admins_only.
This commit replaces add_emoji_by_admins_only with
add_custom_emoji_policy in new_feature_tutorial.md
as the old boolean setting is replaced by a new
integer setting.
2021-07-27 16:41:23 -07:00
Suyash Vardhan Mathur ad8d9f2133 docs: Document the new APIDocs template system.
Recently, the need for individual Markdown templates for
every endpoint's OpenAPI page was removed, as they are now
auto-generated from OpenAPI data. Further, as a part of this
migration, several new fields and Markdown extensions were added.
This commit updates the documentation to reflect the changes that
have occured as a result of the migration.

With various edits by tabbott to clarify or simplify the documentation.
2021-07-26 18:38:57 -07:00
PIG208 c7210bfc75 docs: Update client.md for the request notes change.
`request.client` is no longer valid since the ZulipRequestNotes change.
This update the documentation to reflect that. And it also makes it
recommend `check_send_webhook_message` in favor of
`check_send_stream_message`.
2021-07-26 08:40:53 -07:00
Tim Abbott 454a900487 docs: Replace chat-zulip-org page with /developer-community/. 2021-07-25 22:17:57 -07:00
Tim Abbott 361112a08f version: Update version and changelog after 4.5 release. 2021-07-25 16:05:22 -07:00
Mateusz Mandera 388932bcc4 docs: Move docs on SOCIAL_AUTH_OIDC_FULL_NAME_VALIDATED to ReadTheDocs. 2021-07-24 15:16:13 -07:00
Mateusz Mandera 9bf84a35ef docs: Mention auto_signup OIDC option in ReadTheDocs documentation. 2021-07-24 15:13:47 -07:00
Mateusz Mandera 0d6b1cd08b docs: Tweak phrasing about OIDC supporting only a single provider.
Strictly speaking, this sentence is talking about the IdP configuration,
while the backend is just GenericOpenIdConnectBackend, so the new
phrasing is more correct.
2021-07-24 15:12:06 -07:00
Alex Vandiver 34833aa795 version: Update version and changelog after 4.4 release. 2021-07-22 17:12:36 -07:00
Alex Vandiver d78723b6e8 ci: Update outdated comments, documentation and gitignore.
Use of `Dockerfile.template` and generated `tools/ci/images/` was
removed in 16067bc4fc.
2021-07-22 14:09:01 -07:00
manavdesai27 572cef9a0f provision: Add support for Fedora 34. 2021-07-20 12:10:41 -07:00
Alex Vandiver 91282ab490 reindex-textual-data: Provide a tool to reindex all text indices.
The script is added to upgrade steps for 20.04 and Buster because
those are the upgrades that cross glibc 2.28, which is most
problematic.  It will also be called out in the upgrade notes, to
catch those that have already done that upgrade.
2021-07-19 16:34:23 -07:00
Alex Vandiver 528e5adaab smokescreen: Default to only listening on 127.0.0.1.
This prevents Smokescreen from acting as an open proxy.

Fixes #19214.
2021-07-14 15:40:26 -07:00
Anders Kaseorg 47897c76a2 scripts: Use curl -f (--fail).
This makes curl exit with nonzero status on HTTP 4xx/5xx errors.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-13 16:47:49 -07:00
Anders Kaseorg 77169408f4 docs: Remove do-release-upgrade -d option.
It has not been necessary since Ubuntu 20.04.1 was released in August.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-13 11:02:44 -07:00
Suyash Vardhan Mathur 981e4f8946
openapi: Render all responses of an operation.
Previously, one needed to specifying all the HTTP status
codes that we want to render along with the operation,
but the primary use case just needs the responses of
all the status codes, and not just one.

This commit modifies the Markdown extension to render
all the responses of all status codes of a specified
operation in a loop.
2021-07-13 08:33:43 -07:00
Anders Kaseorg 93f62b999e nagios: Replace check_website_response with standard check_http plugin.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-09 16:47:03 -07:00
Tim Abbott 1905bb8f1a docs: Document forked change password command. 2021-07-09 12:37:55 -07:00
Mateusz Mandera 85cbdc8904 rate_limit: Add rate limiting of ZulipRemoteServer. 2021-07-08 15:55:02 -07:00
Mateusz Mandera 3b4f8cc85b saml: Add setting to skip the "continue to registration" page.
It's a smoother Just-In-Time provisioning process to allow
creating the account and getting signed in on the first login by the
user.
2021-07-08 15:21:20 -07:00
Steve Howell 8ffe22fc15 slash commands: Remove /settings command.
It's easy enough for power keyboard users to just use "g"
to get to settings, and it's certainly easier for
mouse users to just go to the menu.
2021-07-08 10:47:52 -04:00
PIG208 8b9011dff8 json_error: Completely remove json_error.
This completes the migration from `return json_error` to
`raise JsonableError`.
2021-07-06 15:34:33 -07:00
Eeshan Garg ef6582edb8 readthedocs: Add proper backlink to the Zulip homepage.
Now that we are starting to link this pages from the landing page's
top navigation, it makes sense to have proper backlinks to the
homepage so that there is some continuity when the user clicks on
a link that takes them to a ReadTheDocs page from the main website.
2021-07-06 10:59:59 -07:00
Anders Kaseorg 0ae94c6051 mypy: Move configuration to pyproject.toml.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 17:01:32 -07:00
Anders Kaseorg 19ee45ccc4 docs: Empty development-only page list for 4.0 release.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 16:59:25 -07:00
Anders Kaseorg ee8724e436 docs: Correct Big Blue Button to BigBlueButton.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 16:59:00 -07:00
Abhijeet Prasad Bodas 6bbfddb6c3 docs: Split widgets and slash command developer pages. 2021-07-01 08:06:46 -04:00
PIG208 a254fa24ee docs: Use `JsonableError` instead of `json_error`. 2021-06-30 16:26:54 -07:00
Tim Abbott 322a9b0346 docs: Define the term linter.
I heard with a new contributor that they were really confused by this
page, and I believe the key problem was that we didn't define the
somewhat technical term "linter".
2021-06-30 15:24:18 -07:00
Steve Howell dcb058ca8b docs: Remove BETA label for widget docs. 2021-06-28 21:59:22 -07:00
Anders Kaseorg 0ba9114c22 install-yarn: Rewrite Yarn installer.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-27 16:58:17 -07:00
Anders Kaseorg 91bfebca7d install: Replace wget with curl.
curl uses Happy Eyeballs to avoid long timeouts on systems with broken
IPv6.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-25 09:05:07 -07:00
Abhijeet Prasad Bodas 678526881b docs: Update developer documentation for notification_data.py.
We extracted a bunch of code into the common notification_data.py
module, with a data class managing a lot of the logic; this updates
the documentation to cover that.
2021-06-25 08:57:56 -07:00
Anders Kaseorg 128eb8cd8e docs: Document build-docker-images.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-24 15:39:57 -07:00
Dinesh b92d9fb77c docs: Mention about test_example.py in testing-docs. 2021-06-20 07:16:35 -04:00
Tim Abbott 054be8dd2e docs: Document new copied Django function. 2021-06-17 11:32:43 -07:00
Sean Yuan 33793a3a07 docs: Add translation policy on API error messages.
Record Zulip's translation policy on API error messages.
2021-06-14 17:22:41 -07:00
Alex Vandiver 108e7ee8af docs: Update example supervisorctl status. 2021-06-14 17:12:59 -07:00
akshatdalton 803e199063 docs: Update doc on how to mock HTTP responses.
We prefer to use respones module rather than a
custom system to mock the HTTP responses.

Rewritten by tabbott to provide more context/explanation.
2021-06-09 12:23:57 -07:00
Anders Kaseorg e0769e39da docs: Document how ShellCheck and shfmt are installed.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-09 11:01:42 -07:00
Mateusz Mandera c54b48452d saml: Support syncing custom profile fields with SAML attributes.
Fixes #17277.

The main limitation of this implementation is that the sync happens if
the user authing already exists. This means that a new user going
through the sign up flow will not have their custom fields synced upon
finishing it. The fields will get synced on their consecutive log in via
SAML in the future. This can be addressed in the future by moving the
syncing code further down the codepaths to login_or_register_remote_user
and plumbing the data through to the user creation process.

We detail that limitation in the documentation.
2021-06-08 14:47:33 -07:00
Tim Abbott 16700fc82c docs: Remove inaccurate LDAP coming soon notice.
We implemented this feature in the series ending with the
documentation in 2abe325147.
2021-06-08 14:29:47 -07:00
Tim Abbott bb83aab860 docs: Correct widgets doc discussion of slash commands. 2021-06-04 09:23:51 -07:00
Alex Vandiver 161645c83c docs: Link to install-existing-server from production/deployment. 2021-06-02 17:16:14 -07:00
Alex Vandiver 03dd44e0ad docs: Remove link to 16.04, which can be confusing.
The instructions do not just apply to 16.04; the block below describes
the settings, which are correct for all relevant Ubuntu versions.
2021-06-02 16:58:24 -07:00
Tim Abbott 84c7c8cc0d version: Update version and changelog after 4.3 release. 2021-06-02 12:55:50 -07:00
Abhijeet Prasad Bodas e11648f22e docs: Fix typo in subsystems/notifications.md. 2021-06-01 15:26:49 -07:00
Alex Vandiver 1771f61f96 docs: Add a missing close paren. 2021-06-01 14:57:42 -07:00
Alex Vandiver 663d3b780d docs: Update path to nginx.conf, as it is now a template.
Also provide the right expansion for the one embedded variable
currently in the template.
2021-06-01 14:57:42 -07:00
Mateusz Mandera e1b8315f5e docs: Give an example command for generating SAML key/cert pair. 2021-06-01 08:29:05 -07:00
Mateusz Mandera b31ef13cf1 docs: Mention that SAML private key cannot be encrypted.
The underlying xmlsec library doesn't give a very useful error when
trying to load an encrypted private key which can lead to confusion.
2021-06-01 08:29:05 -07:00
Mateusz Mandera e17758f8ad auth: Implement a generic OpenID Connect backend.
Fixes #11939.
2021-05-28 09:43:06 -07:00
Alex Vandiver 87a109e3e0 puppet: Pull in pinned puppet modules.
Using puppet modules from the puppet forge judiciously will allow us
to simplify the configuration somewhat; this specifically pulls in the
stdlib module, which we were already using parts of.
2021-05-27 21:14:48 -07:00
Gilbert Bishop-White e2654b7154 docs: Add instructions for Apple Silicon Macs. 2021-05-27 16:01:42 -07:00
Tim Abbott 8960e64443 docs: Provide more detailed instructions for requesting API key.
Some users had trouble getting an API key that was resolved by doing
this.

Fixes #18524.
2021-05-27 12:09:40 -07:00
Tim Abbott 0294ce76b4 docs: Mention TODO/compatibility comments for events migrations. 2021-05-26 15:21:48 -07:00
Abhijeet Prasad Bodas ddd123f133 message send: Rename `push_notify_user_ids` -> `online_push_user_ids`.
The old name `push_notify_user_ids` was misleading, because
it does not contain user ids which should be notified for
the current message, but rather user ids who have the online
push notifications setting enabled.

When the Tornado server is restarted during an upgrade, if
server has old events with the `push_notify_user_ids` fields,
the server will throw error after this rename. Hence, we need
to explicitly handle such cases while processing the event.
2021-05-26 15:13:08 -07:00
Alex Vandiver 1ab14f8d15 docs: Remove a file which no longer exists from THIRDPARTY.
It was removed in 771e03cfa7.
2021-05-25 21:00:26 -07:00
Adam Birds 3c28fa033d docs: Update documentation for new database user and dbname flags.
Updated the install documentation to include the explanation of the
two new install options `--postgresql-database-name` and
`--postgresql-database-user`.
2021-05-25 13:56:05 -07:00
Tim Abbott 47689feb33 docs: Extend documentation on Tornado schema changes. 2021-05-24 13:56:58 -07:00
Aman Agrawal d1c385f425 docs: Clarify how to set GIPHY API key in settings. 2021-05-18 17:32:55 -07:00
Anders Kaseorg e015f3ed7d docs: Correct “webapp” to “web app”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-14 13:15:38 -07:00
Anders Kaseorg e3c570401e docs: Document password hashing change from PBKDF2 to Argon2.
This changed in commit 483a351d44
(#3410).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-14 13:07:51 -07:00
Tim Abbott ed04966f79 version: Update version and changelog following 4.2 release. 2021-05-13 22:09:48 -07:00
Tim Abbott 6d4e7aa307 version: Update version and changelog following 4.1 release. 2021-05-13 18:58:11 -07:00
Tim Abbott bf4f6544e7 version: Update version following 4.0 release. 2021-05-13 15:48:15 -07:00
Tim Abbott 825192a051 Release Zulip Server 4.0. 2021-05-13 15:37:37 -07:00
Tim Abbott 4fd9a971ed lint: Allow changelog link to latest changelog. 2021-05-13 12:36:56 -07:00
Tim Abbott 7486b646d8 docs: Update missing details in release checklist. 2021-05-13 12:36:56 -07:00
Alex Vandiver 653c67a2b6 docs: Explicitly list possible outgoing HTTP connections. 2021-05-13 11:06:30 -07:00
Tim Abbott 6777660e02 deployment: Document smokescreen default and configuration. 2021-05-13 11:05:47 -07:00
Tim Abbott 49e076e01c changelog: Fix link to release lifecycle. 2021-05-12 22:51:15 -07:00
Tim Abbott df2e8fd047 docs: Update changelog in preparation for 4.0 release.
We drop the 4.0-rc1 label, since the changelog now has things not
present in that release candidate.
2021-05-12 22:38:25 -07:00
Alex Vandiver a9ced1d228 smokescreen: Capitalize Smokescreen in documentation. 2021-05-12 12:32:39 -07:00
Alex Vandiver 0f1611286d management: Rename the deliver_email command to deliver_scheduled_email.
This makes it parallel with deliver_scheduled_messages, and clarifies
that it is not used for simply sending outgoing emails (e.g. the
`email_senders` queue).

This also renames the supervisor job to match.
2021-05-11 13:07:29 -07:00
Matthew Wegner c43f883699
docs: Fix typo in export docs.
"as part of a high evailability environment" => "as part of a high availability environment".
2021-05-10 17:33:21 -07:00
Mateusz Mandera 8261f7e801 commands: Add delete_user management command and document it. 2021-05-10 15:38:14 -07:00
Siddharth Asthana 6ee7d5c983 docs: Improve documentation for changing a realm's subdomain.
Add a single line for self-hosting case to point to the ReadTheDocs
section of management command to change the realm's subdomain.

Fixes: #17857
2021-05-10 12:33:43 -07:00
Anders Kaseorg 544bbd5398 docs: Fix capitalization mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-10 09:57:26 -07:00
Tim Abbott 129b5c6779 docs: Document our choice of tooltip library.
This page has very little documentation on UI widgets in general, but
adding this is probably net helpful.

Fixes #1398.
2021-05-09 21:54:51 -07:00
Tim Abbott 9a5ad4163e i18n: Rewrite text explaining default language panel.
* Explain what the percentages mean.
* Adjust old verbiage from before we had many translations.
2021-05-09 19:02:09 -07:00
Mateusz Mandera 8bc614abf6 docs: Mention quirk of mixing ALTER TABLE and RunPython in migrations. 2021-05-09 09:38:08 -07:00
Mateusz Mandera ddcb919e4d docs: Mention new AddIndexConcurrently operation in migration doc. 2021-05-09 09:37:28 -07:00
Adam Birds 5fa8cf042d docs: Make it easier to locate docs for how to build integration.
Make it easier to locate docs for how to build an integration by adding
a link to the docs at the top of the docs on how to document an
integration.

Tweaked by tabbott to adjust the whole first paragraph and make the
link more contextual.
2021-05-08 07:43:39 -07:00
Tim Abbott d4e2abb53a docs: Clean up documentation writing page. 2021-05-08 07:41:40 -07:00
Adam Birds 3508ec01c9 docs: Add link to integrations developer documentation. 2021-05-08 07:33:34 -07:00
Tim Abbott 0044c8f85f docs: Remove recommendations of json_validator with check_string.
See #18035 for background.
2021-05-07 15:12:18 -07:00
Anders Kaseorg 405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00
Tim Abbott 17d2b28085 docs: Add two new paragraphs to testing philosophy. 2021-05-06 17:46:44 -07:00
Alex Vandiver 0663094920 docs: Document Sentry in logging docs. 2021-05-05 23:04:12 -07:00
Alex Vandiver b26ef55d38 Remove old "conversion" documentation. 2021-05-05 23:03:26 -07:00
Alex Vandiver bf1a6d18ef docs: Remove out-of-date "users" documentation. 2021-05-05 23:03:26 -07:00
Alex Vandiver 045e8aa2f0 docs: Remove old "custom apps" documentation.
This is significantly out of date; https://chat.zulip.org/api/ serves
as documentation of the current state of affairs.
2021-05-05 23:03:26 -07:00
Tim Abbott 35a5cd1e65 docs: Add new summary section to the release lifecycle page. 2021-05-05 18:24:17 -07:00
Tim Abbott 07a2e91688 docs: Update changelog heading for new release lifecycle page.
Also update the release checklist to make sure we don't forget to
update the top-heading section when we publish a new major release.
2021-05-05 17:45:11 -07:00
Tim Abbott d002035856 docs: Clarify how changelog is updated.
This should help avoid anyone being confused that Zulip may not be
actively developed when viewing our ReadTheDocs.
2021-05-05 17:20:19 -07:00
Alya Abbott 279f7b1c8d user docs: Create separate "Code blocks" user documentation page.
* Move the extended documentation of code blocks to a separate page.
* Merge "code playgrounds" documentation to be a section of that page.
* Document copy widget on code blocks.
* This commit changes how we refer to "```python" type syntax for code
  blocks. Instead of being called a syntax highlighting label, this is
  now referred to as a "language tag", since it serves both syntax
  highlighting and playgrounds.
* Remap all the links.
* Advertise this new page in various places that previously did not have a link.
2021-05-05 16:11:21 -07:00
Abhijeet Prasad Bodas 80ccfc79df docs: Fix outdated pre-ES6 function examples. 2021-05-05 09:34:39 -07:00
Tim Abbott d7d79b57b7 docs: Replace legacy roadmap.md with release lifecycle page.
This new pages accomplishes several interrelated things:
* Documents that Zulip Cloud runs master and how that works.
* Documents policies on how long client apps are expected to support
  old releases in our compatibility matrix.
* Removes the 3-years-stale roadmap article.
* Provides a central place to talk about different versions in Zulip.
* Provides a better place to link to from our "you need to upgrade" nag.

This content is not intended to be final, but should be finalized in
the next week or so.

Fixes #18322.
2021-05-05 09:08:43 -07:00
Tim Abbott eee430e795 docs: Delete gsod-ideas.md.
This page is documentation for our participation in GSoD, which was a
few years ago. We can resurrect it if we participate in the program
again, so there's no need to keep it as a live page.
2021-05-04 15:21:49 -07:00
Mateusz Mandera 684908bdce docs: Update changelog the mention user may get logged out.
This is due to the mechanism described in
https://docs.djangoproject.com/en/3.2/topics/auth/default/#session-invalidation-on-password-change
combined with the changes introduced in
76ae6ccf85
2021-05-04 14:56:17 -07:00
Tim Abbott d5559e1e7b docs: Fix link to roles-and-permissions. 2021-05-03 17:44:36 -07:00
Tim Abbott 4fd0725678 version: Update for 4.0-rc1 release. 2021-05-03 17:37:01 -07:00
Tim Abbott a1f543f605 docs: Update changelog with changes present in 4.0-rc1. 2021-05-03 17:27:22 -07:00
Mateusz Mandera 537186f474 docs: Remove mention of SessionMiddleware from django-upgrades.md.
We no longer fork SessionMiddleware since
dce6b4a40f.
2021-05-02 13:15:38 +02:00
Wesley Aptekar-Cassels 6b7a3fb74a markdown: Rewrite all external images to use Camo.
Requesting external images is a privacy risk, so route all external
images through Camo.

Tweaked by tabbott for better test coverage, more comments, and to fix
bugs.
2021-04-30 10:36:16 -07:00
Anders Kaseorg fc243854a8 docs: Note that Iterable is wrong for values iterated multiple times.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-29 16:06:17 -07:00
Tim Abbott 03e155f38a docs: Add production documentation for custom management commands. 2021-04-29 12:36:13 -07:00
strifel 02c5676a67 ldap: Add advanced LDAP realm access control.
This allows access to be more configurable than just setting one
attribute.  This can be configured by setting the setting
AUTH_LDAP_ADVANCED_REALM_ACCESS_CONTROL.
2021-04-28 09:08:13 -07:00
Aman Agrawal f71b591f95 Revert "docs: Temporarily remove Giphy help center link."
This reverts commit a94b9cc9e2.
2021-04-28 07:07:36 -07:00
Alex Vandiver 4c88da8ed9 scripts: Tool to find the diff to an original settings.py prod template.
This hits the unauthenticated Github API to get the list of tags,
which is rate-limited to 60 requests per hour.  This means that the
tool can only be run 60 times per hour before it starts to exit with
errors, but that seems like a reasonable limit for the moment.
2021-04-27 21:50:33 -07:00
Adam Birds ceb4b239c4 docs: Update install docs to document the install options correctly.
Update `docs/production/install.md` and
`docs/production/deployment.md` to document the install flags that can
be used as part of the installer more clearly.

Fixes #18122.
2021-04-27 19:44:05 -07:00
Alex Vandiver bc9d20eaa2 docs: Describe the process of upgrading settings.py files.
Fixes #17782.
2021-04-27 18:01:28 -07:00
Alex Vandiver ae2c377d13 postgresql: Switch to defaulting to PostgreSQL 13. 2021-04-27 16:55:04 -07:00
Alex Vandiver d50d46525c docs: Document the `test-install` tooling. 2021-04-27 00:55:50 -07:00
Tim Abbott a615dcb83e docs: Fix double-period typo in settings.md. 2021-04-26 17:07:41 -07:00
Anders Kaseorg 6060d0d364 docs: Add missing space to compound verbs “log in”, “set up”, etc.
Noun: backup, checkout, cleanup, login, logout, setup, shutdown, signup,
timeout.

Verb: back up, check out, clean up, log in, log out, set up, shut
down, sign up, time out.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg e3f2ffa681 docs: Capitalize “Markdown” consistently.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Anders Kaseorg 178736c8eb docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Mateusz Mandera 1b5a7a2337 docs: Clarify what the certificate is supposed to be in the saml config.
Closes #17955.

Users were sometimes confused about what this certificate is supposed to
be.
2021-04-23 15:16:58 -07:00
Gaurav Pandey fa235e60ff login: Add show password feature to login page.
The show password feature is a functionality to
toggle the visibility of the password fields in forms
so that one can check if they have entered the correct
password or not. We implement this using an eye icon
toggling which converts input field type from password
to text and vice-versa.
Fixes part of #17301.
2021-04-23 15:01:37 -07:00
Alex Vandiver 476524c0c1 scripts: Add a script to stop the server.
Using `supervisorctl stop all` to stop the server is not terribly
discoverable, and may stop services which are not part of Zulip
proper.

Add an explicit tool which only stops the relevant services.  It also
more carefully controls the order in which services are stopped to
minimize lost requests, and maximally quiesce the server.

Locations which may be stopping _older_ versions of Zulip (without
this script) are left with using `supervisorctl stop all`.

Fixes #14959.
2021-04-21 10:24:08 -07:00
Mateusz Mandera 977a2f7fa0 emails: Rename "missed message email" to "message notification email". 2021-04-21 10:10:54 -07:00