Commit Graph

2441 Commits

Author SHA1 Message Date
Tim Abbott ea6a40661c migrations: Add progress output to confirmation backfill.
This makes it a lot easier to estimate how long this migration will
take to run. Additionally, document this migration in the upgrade notes.
2021-09-17 11:23:32 -07:00
Mateusz Mandera 8ad7520180 ldap: Support alternative attrs to userAccountControl.
Fixes #17456.

The main tricky part has to do with what values the attribute should
have. LDAP defines a Boolean as
Boolean = "TRUE" / "FALSE"
so ideally we'd always see exactly those values. However,
although the issue is now marked as resolved, the discussion in
https://pagure.io/freeipa/issue/1259 shows how this may not always be
respected - meaning it makes sense for us to be more liberal in
interpreting these values.
2021-09-16 13:29:13 -07:00
Mateusz Mandera b32450f98e docs: Remove mentions of some ldap features being added in 2.0.
2.0 is old enough that explicitly mentioning when these features were
implemented isn't particularly useful and adds clutter.
2021-09-16 13:29:13 -07:00
Eeshan Garg 3b1cb0b25a docs: Indicate latest Zulip version in installation and upgrade docs.
With copy-editing from tabbott, and also a migration to use
LATEST_RELEASE_VERSION, which will be correct even on the /latest/
paths.

Fixes #19695.
2021-09-10 14:50:48 -07:00
Tim Abbott 908e1e6d19 docs: Run prettier on authentication-methods.md.
The new sections of this file were rebased past the addition of
running prettier on these files.
2021-09-10 13:23:11 -07:00
Philipp Seßner a8797b4d9f help: Document how to configure Keycloak as the SAML IdP. 2021-09-10 13:04:04 -07:00
Anders Kaseorg 5360bd76ac help: Add newline after admonition header for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-10 10:39:25 -07:00
Gaurav Pandey 502697d239 docs: Add documentation for bullseye support.
The support for bullseye was added in #17951
but it was not documented as bullseye was
frozen and did not have proper configuration
files, hence wasn't documented.

Since now bullseye is released as a stable
version, it's support can be documented.
2021-09-09 11:05:16 -07:00
Anders Kaseorg 1fc801a43f docs: Add missing space in “feature set”.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 15:31:28 -07:00
Anders Kaseorg 85fec56697 docs: Update links for zulip-mobile branch rename.
GitHub redirects these, but we should use the canonical URLs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 15:30:37 -07:00
Anders Kaseorg a4dbc1edd4 docs: Format Markdown with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg 35c1c8d41b docs: Apply sentence single-spacing from Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg 915884bff7 docs: Apply bullet style changes from Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg 6145fdf678 docs: Rewrap to avoid line breaks in inline code spans.
This works around https://github.com/prettier/prettier/issues/11372.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg a6e01b35fc docs: Fix `pip compile` typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-08 12:06:24 -07:00
Anders Kaseorg f4d2d199e2 docs: Display `main` branch name as inline code.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-06 12:56:35 -07:00
Anders Kaseorg 646c04eff2 Rename default branch to ‘main’.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-09-06 12:56:35 -07:00
Tim Abbott 1c3517a5de docs: Document ReadTheDocs preview builds. 2021-09-06 12:40:36 -07:00
Anders Kaseorg 1ce12191aa docs: Update links for other repository branch renames.
GitHub redirects these, but we should use the canonical URLs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 15:28:01 -07:00
Anders Kaseorg 0dcb84a2e8 docs: Follow Sphinx 4 rename of master_doc to root_doc.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 14:24:16 -07:00
Alex Vandiver 7d7d727865 tools: Switch to download.zulip.com from www.zulip.org. 2021-08-26 17:21:31 -07:00
Alex Vandiver 710a92f48c tools: Add a tool to upload release tarballs to S3. 2021-08-26 17:21:31 -07:00
Alex Vandiver faf71eea41 upgrade-postgresql: Do not remove other supervisor configs.
We previously used `zulip-puppet-apply` with a custom config file,
with an updated PostgreSQL version but more limited set of
`puppet_classes`, to pre-create the basic settings for the new cluster
before running `pg_upgradecluster`.

Unfortunately, the supervisor config uses `purge => true` to remove
all SUPERVISOR configuration files that are not included in the puppet
configuration; this leads to it removing all other supervisor
processes during the upgrade, only to add them back and start them
during the second `zulip-puppet-apply`.

It also leads to `process-fts-updates` not being started after the
upgrade completes; this is the one supervisor config file which was
not removed and re-added, and thus the one that is not re-started due
to having been re-added.  This was not detected in CI because CI added
a `start-server` command which was not in the upgrade documentation.

Set a custom facter fact that prevents the `purge` behaviour of the
supervisor configuration.  We want to preserve that behaviour in
general, and using `zulip-puppet-apply` continues to be the best way
to pre-set-up the PostgreSQL configuration -- but we wish to avoid
that behaviour when we know we are applying a subset of the puppet
classes.

Since supervisor configs are no longer removed and re-added, this
requires an explicit start-server step in the instructions after the
upgrades complete.  This brings the documentation into alignment with
what CI is testing.
2021-08-24 19:00:58 -07:00
Anders Kaseorg fdb7ec8c9e docs: Avoid [GitHub] as an internal Markdown link reference name.
To avoid confusing the linter later when Prettier lowercases these.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 0e4a30daad reading-list: Inline links.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 32da7e3eb2 docs: Adjust colon fence for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 7b3d4ff1de docs: Escape asterisks for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 0a3e022376 docs: Adjust list item indentation for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 4bfffc9f74 docs: Fix list item indentation mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 962f14995e docs: Fix “sinternet” typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg fe3db63381 docs: Add missing blockquote.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg b53e676860 docs: Convert reST index pages to Markdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 06:22:42 -07:00
Anders Kaseorg b29b6f6526 docs: Add syntax highlighting languages to code blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 06:14:47 -07:00
Anders Kaseorg 672358c5da docs: Migrate from recommonmark to MyST-Parser.
Recommonmark is no longer maintained, and MyST-Parser is much more
complete.

https://myst-parser.readthedocs.io/

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 8f2827a65d docs: Remove trailing newlines from code blocks.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg cb61ea69c6 docs: Fix misaligned Markdown source indentation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg 25c6d5c99c docs: Fix code span syntax in embedded reST block.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Anders Kaseorg c3646ec67f docs: Move authentication-methods#ldap anchor to appropriate heading.
Commit 30eaed0378 (#15001) incorrectly
inserted a different section between the anchor and the heading.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-19 00:53:32 -07:00
Mateusz Mandera fdbde59b07 rate_limit: Add management command to reset auth rate limit.
The auth attempt rate limit is quite low (on purpose), so this can be a
common scenario where a user asks their admin to reset the limit instead
of waiting. We should provide a tool for administrators to handle such
requests without fiddling around with code in manage.py shell.
2021-08-19 00:45:17 -07:00
Alex Vandiver ec354ee013 changelog: Add 2.1.8 release notes. 2021-08-12 01:06:15 +00:00
Alex Vandiver e94b6afb00 nagios: Remove broken check_email_deliverer_* checks and related code.
These checks suffer from a couple notable problems:
 - They are only enabled on staging hosts -- where they should never
   be run.  Since ef6d0ec5ca, these supervisor processes are only
   run on one host, and never on the staging host.
 - They run as the `nagios` user, which does not have appropriate
   permissions, and thus the checks always fail.  Specifically,
   `nagios` does not have permissions to run `supervisorctl`, since
   the socket is owned by the `zulip` user, and mode 0700; and the
   `nagios` user does not have permission to access Zulip secrets to
   run `./manage.py print_email_delivery_backlog`.

Rather than rewrite these checks to run on a cron as zulip, and check
those file contents as the nagios user, drop these checks -- they can
be rewritten at a later point, or replaced with Prometheus alerting,
and currently serve only to cause always-failing Nagios checks, which
normalizes alert failures.

Leave the files installed if they currently exist, rather than
cluttering puppet with `ensure => absent`; they do no harm if they are
left installed.
2021-08-03 16:07:13 -07:00
Tim Abbott f84703bc48 test-backend: Default to running all tests.
This switches the default of the previous --nonfatal-errors option,
and also moves to use the `-x | --stop` names used by the nose test
framework.
2021-07-30 16:18:42 -07:00
Tim Abbott 665f479f4d docs: Update changelog since 4.0 release. 2021-07-29 14:54:17 -07:00
Tim Abbott e4667c2af6 docs: Add developer documentation for Python support. 2021-07-28 11:23:27 -07:00
sahil839 9ef8727a4a docs: Fix new_feature_tutorial to replace add_emoji_by_admins_only.
This commit replaces add_emoji_by_admins_only with
add_custom_emoji_policy in new_feature_tutorial.md
as the old boolean setting is replaced by a new
integer setting.
2021-07-27 16:41:23 -07:00
Suyash Vardhan Mathur ad8d9f2133 docs: Document the new APIDocs template system.
Recently, the need for individual Markdown templates for
every endpoint's OpenAPI page was removed, as they are now
auto-generated from OpenAPI data. Further, as a part of this
migration, several new fields and Markdown extensions were added.
This commit updates the documentation to reflect the changes that
have occured as a result of the migration.

With various edits by tabbott to clarify or simplify the documentation.
2021-07-26 18:38:57 -07:00
PIG208 c7210bfc75 docs: Update client.md for the request notes change.
`request.client` is no longer valid since the ZulipRequestNotes change.
This update the documentation to reflect that. And it also makes it
recommend `check_send_webhook_message` in favor of
`check_send_stream_message`.
2021-07-26 08:40:53 -07:00
Tim Abbott 454a900487 docs: Replace chat-zulip-org page with /developer-community/. 2021-07-25 22:17:57 -07:00
Tim Abbott 361112a08f version: Update version and changelog after 4.5 release. 2021-07-25 16:05:22 -07:00
Mateusz Mandera 388932bcc4 docs: Move docs on SOCIAL_AUTH_OIDC_FULL_NAME_VALIDATED to ReadTheDocs. 2021-07-24 15:16:13 -07:00
Mateusz Mandera 9bf84a35ef docs: Mention auto_signup OIDC option in ReadTheDocs documentation. 2021-07-24 15:13:47 -07:00
Mateusz Mandera 0d6b1cd08b docs: Tweak phrasing about OIDC supporting only a single provider.
Strictly speaking, this sentence is talking about the IdP configuration,
while the backend is just GenericOpenIdConnectBackend, so the new
phrasing is more correct.
2021-07-24 15:12:06 -07:00
Alex Vandiver 34833aa795 version: Update version and changelog after 4.4 release. 2021-07-22 17:12:36 -07:00
Alex Vandiver d78723b6e8 ci: Update outdated comments, documentation and gitignore.
Use of `Dockerfile.template` and generated `tools/ci/images/` was
removed in 16067bc4fc.
2021-07-22 14:09:01 -07:00
manavdesai27 572cef9a0f provision: Add support for Fedora 34. 2021-07-20 12:10:41 -07:00
Alex Vandiver 91282ab490 reindex-textual-data: Provide a tool to reindex all text indices.
The script is added to upgrade steps for 20.04 and Buster because
those are the upgrades that cross glibc 2.28, which is most
problematic.  It will also be called out in the upgrade notes, to
catch those that have already done that upgrade.
2021-07-19 16:34:23 -07:00
Alex Vandiver 528e5adaab smokescreen: Default to only listening on 127.0.0.1.
This prevents Smokescreen from acting as an open proxy.

Fixes #19214.
2021-07-14 15:40:26 -07:00
Anders Kaseorg 47897c76a2 scripts: Use curl -f (--fail).
This makes curl exit with nonzero status on HTTP 4xx/5xx errors.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-13 16:47:49 -07:00
Anders Kaseorg 77169408f4 docs: Remove do-release-upgrade -d option.
It has not been necessary since Ubuntu 20.04.1 was released in August.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-13 11:02:44 -07:00
Suyash Vardhan Mathur 981e4f8946
openapi: Render all responses of an operation.
Previously, one needed to specifying all the HTTP status
codes that we want to render along with the operation,
but the primary use case just needs the responses of
all the status codes, and not just one.

This commit modifies the Markdown extension to render
all the responses of all status codes of a specified
operation in a loop.
2021-07-13 08:33:43 -07:00
Anders Kaseorg 93f62b999e nagios: Replace check_website_response with standard check_http plugin.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-09 16:47:03 -07:00
Tim Abbott 1905bb8f1a docs: Document forked change password command. 2021-07-09 12:37:55 -07:00
Mateusz Mandera 85cbdc8904 rate_limit: Add rate limiting of ZulipRemoteServer. 2021-07-08 15:55:02 -07:00
Mateusz Mandera 3b4f8cc85b saml: Add setting to skip the "continue to registration" page.
It's a smoother Just-In-Time provisioning process to allow
creating the account and getting signed in on the first login by the
user.
2021-07-08 15:21:20 -07:00
Steve Howell 8ffe22fc15 slash commands: Remove /settings command.
It's easy enough for power keyboard users to just use "g"
to get to settings, and it's certainly easier for
mouse users to just go to the menu.
2021-07-08 10:47:52 -04:00
PIG208 8b9011dff8 json_error: Completely remove json_error.
This completes the migration from `return json_error` to
`raise JsonableError`.
2021-07-06 15:34:33 -07:00
Eeshan Garg ef6582edb8 readthedocs: Add proper backlink to the Zulip homepage.
Now that we are starting to link this pages from the landing page's
top navigation, it makes sense to have proper backlinks to the
homepage so that there is some continuity when the user clicks on
a link that takes them to a ReadTheDocs page from the main website.
2021-07-06 10:59:59 -07:00
Anders Kaseorg 0ae94c6051 mypy: Move configuration to pyproject.toml.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 17:01:32 -07:00
Anders Kaseorg 19ee45ccc4 docs: Empty development-only page list for 4.0 release.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 16:59:25 -07:00
Anders Kaseorg ee8724e436 docs: Correct Big Blue Button to BigBlueButton.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-07-05 16:59:00 -07:00
Abhijeet Prasad Bodas 6bbfddb6c3 docs: Split widgets and slash command developer pages. 2021-07-01 08:06:46 -04:00
PIG208 a254fa24ee docs: Use `JsonableError` instead of `json_error`. 2021-06-30 16:26:54 -07:00
Tim Abbott 322a9b0346 docs: Define the term linter.
I heard with a new contributor that they were really confused by this
page, and I believe the key problem was that we didn't define the
somewhat technical term "linter".
2021-06-30 15:24:18 -07:00
Steve Howell dcb058ca8b docs: Remove BETA label for widget docs. 2021-06-28 21:59:22 -07:00
Anders Kaseorg 0ba9114c22 install-yarn: Rewrite Yarn installer.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-27 16:58:17 -07:00
Anders Kaseorg 91bfebca7d install: Replace wget with curl.
curl uses Happy Eyeballs to avoid long timeouts on systems with broken
IPv6.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-25 09:05:07 -07:00
Abhijeet Prasad Bodas 678526881b docs: Update developer documentation for notification_data.py.
We extracted a bunch of code into the common notification_data.py
module, with a data class managing a lot of the logic; this updates
the documentation to cover that.
2021-06-25 08:57:56 -07:00
Anders Kaseorg 128eb8cd8e docs: Document build-docker-images.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-24 15:39:57 -07:00
Dinesh b92d9fb77c docs: Mention about test_example.py in testing-docs. 2021-06-20 07:16:35 -04:00
Tim Abbott 054be8dd2e docs: Document new copied Django function. 2021-06-17 11:32:43 -07:00
Sean Yuan 33793a3a07 docs: Add translation policy on API error messages.
Record Zulip's translation policy on API error messages.
2021-06-14 17:22:41 -07:00
Alex Vandiver 108e7ee8af docs: Update example supervisorctl status. 2021-06-14 17:12:59 -07:00
akshatdalton 803e199063 docs: Update doc on how to mock HTTP responses.
We prefer to use respones module rather than a
custom system to mock the HTTP responses.

Rewritten by tabbott to provide more context/explanation.
2021-06-09 12:23:57 -07:00
Anders Kaseorg e0769e39da docs: Document how ShellCheck and shfmt are installed.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-06-09 11:01:42 -07:00
Mateusz Mandera c54b48452d saml: Support syncing custom profile fields with SAML attributes.
Fixes #17277.

The main limitation of this implementation is that the sync happens if
the user authing already exists. This means that a new user going
through the sign up flow will not have their custom fields synced upon
finishing it. The fields will get synced on their consecutive log in via
SAML in the future. This can be addressed in the future by moving the
syncing code further down the codepaths to login_or_register_remote_user
and plumbing the data through to the user creation process.

We detail that limitation in the documentation.
2021-06-08 14:47:33 -07:00
Tim Abbott 16700fc82c docs: Remove inaccurate LDAP coming soon notice.
We implemented this feature in the series ending with the
documentation in 2abe325147.
2021-06-08 14:29:47 -07:00
Tim Abbott bb83aab860 docs: Correct widgets doc discussion of slash commands. 2021-06-04 09:23:51 -07:00
Alex Vandiver 161645c83c docs: Link to install-existing-server from production/deployment. 2021-06-02 17:16:14 -07:00
Alex Vandiver 03dd44e0ad docs: Remove link to 16.04, which can be confusing.
The instructions do not just apply to 16.04; the block below describes
the settings, which are correct for all relevant Ubuntu versions.
2021-06-02 16:58:24 -07:00
Tim Abbott 84c7c8cc0d version: Update version and changelog after 4.3 release. 2021-06-02 12:55:50 -07:00
Abhijeet Prasad Bodas e11648f22e docs: Fix typo in subsystems/notifications.md. 2021-06-01 15:26:49 -07:00
Alex Vandiver 1771f61f96 docs: Add a missing close paren. 2021-06-01 14:57:42 -07:00
Alex Vandiver 663d3b780d docs: Update path to nginx.conf, as it is now a template.
Also provide the right expansion for the one embedded variable
currently in the template.
2021-06-01 14:57:42 -07:00
Mateusz Mandera e1b8315f5e docs: Give an example command for generating SAML key/cert pair. 2021-06-01 08:29:05 -07:00
Mateusz Mandera b31ef13cf1 docs: Mention that SAML private key cannot be encrypted.
The underlying xmlsec library doesn't give a very useful error when
trying to load an encrypted private key which can lead to confusion.
2021-06-01 08:29:05 -07:00
Mateusz Mandera e17758f8ad auth: Implement a generic OpenID Connect backend.
Fixes #11939.
2021-05-28 09:43:06 -07:00
Alex Vandiver 87a109e3e0 puppet: Pull in pinned puppet modules.
Using puppet modules from the puppet forge judiciously will allow us
to simplify the configuration somewhat; this specifically pulls in the
stdlib module, which we were already using parts of.
2021-05-27 21:14:48 -07:00
Gilbert Bishop-White e2654b7154 docs: Add instructions for Apple Silicon Macs. 2021-05-27 16:01:42 -07:00
Tim Abbott 8960e64443 docs: Provide more detailed instructions for requesting API key.
Some users had trouble getting an API key that was resolved by doing
this.

Fixes #18524.
2021-05-27 12:09:40 -07:00
Tim Abbott 0294ce76b4 docs: Mention TODO/compatibility comments for events migrations. 2021-05-26 15:21:48 -07:00