Commit Graph

44489 Commits

Author SHA1 Message Date
Anders Kaseorg 3cb66d59ac install: Remove /dev/null redirect for zulip-puppet-apply.
The usual output from this command looks like

Notice: Compiled catalog for localhost in environment production in 2.33 seconds
Notice: /Stage[main]/Zulip::Apt_repository/Exec[setup_apt_repo]/returns: current_value 'notrun', should be ['0'] (noop)
Notice: Class[Zulip::Apt_repository]: Would have triggered 'refresh' from 1 event
Notice: Stage[main]: Would have triggered 'refresh' from 1 event
Notice: Applied catalog in 1.20 seconds

which doesn’t seem abnormally alarming, and hiding it makes failures
harder to diagnose.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 16:30:53 -07:00
Anders Kaseorg 2ec5f613e2 docs: Clarify HackerOne program type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 16:30:01 -07:00
Anders Kaseorg 1ce12191aa docs: Update links for other repository branch renames.
GitHub redirects these, but we should use the canonical URLs.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 15:28:01 -07:00
Anders Kaseorg 0dcb84a2e8 docs: Follow Sphinx 4 rename of master_doc to root_doc.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 14:24:16 -07:00
Anders Kaseorg 817146c28b python: Upgrade SQLAlchemy from 1.3.24 to 1.4.23.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-31 06:47:39 -07:00
rht 6bfebd5558 emoji: Enable Japanese characters in the symbol section.
This was originally meant to fix the emoji mapping conflict during a
Slack import. In Slack, 🎉 and ㊗️ have different
symbols, but they both map to 🎉 in Zulip prior to this commit.
㊗️ now refers to the Japanese character version, as is
observed in Matrix and Slack.

I expand the fix to include all other Japanese characters.  Matrix.org
and Slack already have those characters in their symbol section, and so
this is to reach feature parity.

See the discussion thread in https://chat.zulip.org/#narrow/stream/9-issues/topic/duplicate.20emoji.20in.20data.20import
2021-08-31 06:42:36 -07:00
Mateusz Mandera 78297efefd ldap: Use a workaround instead of forking django-auth-ldap.
Till now, we've been forking django-auth-ldap at
https://github.com/zulip/django-auth-ldap to put the
LDAPReverseEmailSearch feature in it, hoping to get it merged
upstream in https://github.com/django-auth-ldap/django-auth-ldap/pull/150

The efforts to get it merged have stalled for now however and we don't
want to be on the fork forever, so this commit puts the email search
feature as a clumsy workaround inside our codebase and switches to using
the latest upstream release instead of the fork.
2021-08-28 23:11:09 -07:00
Alex Vandiver 6c15df68e8 downloads: Serve desktop downloads from desktop-download.zulip.com.
This makes them work for sites which block github.com.
2021-08-28 23:08:45 -07:00
Alex Vandiver 1806e0f45e puppet: Remove zulip.org configuration. 2021-08-26 17:21:31 -07:00
Alex Vandiver 7d7d727865 tools: Switch to download.zulip.com from www.zulip.org. 2021-08-26 17:21:31 -07:00
Alex Vandiver 710a92f48c tools: Add a tool to upload release tarballs to S3. 2021-08-26 17:21:31 -07:00
Alex Vandiver fe25517295 email_notifications: Handle empty rendered_messages.
The transforms called from `build_message_payload` use
`lxml.html.fromstring` to parse (and stringify, and re-parse) the HTML
generated by Markdown.  However, this function fails if it is passed
an empty document.  "empty" is broader than just the empty string; it
also includes any document made entirely out of control characters,
spaces, unpaired surrogates, U+FFFE, or U+FFFF, and so forth.  These
documents would fail to parse, and raise a ParserError.

Using `lxml.html.fragment_fromstring` handles these cases, but does by
wrapping the contents in a <div> every time it is called.  As such,
replacing each `fromstring` with `fragment_fromstring` would nest
another layer of `<div>`.

Instead of each of the helper functions re-parsing, modifying, and
stringifying the HTML, parse it once with `fragment_fromstring` and
pass around the parsed document to each helper, which modifies it
in-place.  This adds one outer `<div>`, requiring minor changes to
tests and the prepend-sender functions.

The modification to add the sender is left using BeautifulSoup, as
that sort of transform is much less readable, and more fiddly, in raw
lxml.

Partial fix for #19559.
2021-08-25 15:50:55 -07:00
Alex Vandiver 42e3c4e6ec email_notifications: Rename append_sender_to_message to prepend.
The sender goes on the beginning, which is prepending to the message.
2021-08-25 15:50:55 -07:00
Alex Vandiver 27881babab puppet: Increase prometheus storage, from the default 15d. 2021-08-24 23:40:43 -07:00
Alex Vandiver faf71eea41 upgrade-postgresql: Do not remove other supervisor configs.
We previously used `zulip-puppet-apply` with a custom config file,
with an updated PostgreSQL version but more limited set of
`puppet_classes`, to pre-create the basic settings for the new cluster
before running `pg_upgradecluster`.

Unfortunately, the supervisor config uses `purge => true` to remove
all SUPERVISOR configuration files that are not included in the puppet
configuration; this leads to it removing all other supervisor
processes during the upgrade, only to add them back and start them
during the second `zulip-puppet-apply`.

It also leads to `process-fts-updates` not being started after the
upgrade completes; this is the one supervisor config file which was
not removed and re-added, and thus the one that is not re-started due
to having been re-added.  This was not detected in CI because CI added
a `start-server` command which was not in the upgrade documentation.

Set a custom facter fact that prevents the `purge` behaviour of the
supervisor configuration.  We want to preserve that behaviour in
general, and using `zulip-puppet-apply` continues to be the best way
to pre-set-up the PostgreSQL configuration -- but we wish to avoid
that behaviour when we know we are applying a subset of the puppet
classes.

Since supervisor configs are no longer removed and re-added, this
requires an explicit start-server step in the instructions after the
upgrades complete.  This brings the documentation into alignment with
what CI is testing.
2021-08-24 19:00:58 -07:00
Alex Vandiver e46e862f2b puppet: Add a bare-bones zulipbot profile.
This sets up the firewalls appropriate for zulipbot, but does not
automate any of the configuration of zulipbot itself.
2021-08-24 16:05:58 -07:00
Alex Vandiver 5857dcd9b4 puppet: Configure ip6tables in parallel to ipv4.
Previously, IPv6 firewalls were left at the default all-open.

Configure IPv6 equivalently to IPv4.
2021-08-24 16:05:46 -07:00
Alex Vandiver 845509a9ec puppet: Be explicit that existing iptables are only ipv4. 2021-08-24 16:05:46 -07:00
Anders Kaseorg 019bd848ca requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-24 07:51:48 -07:00
Anders Kaseorg e3a3519913 semgrep: Replace pattern-where-python with metavariable-regex.
pattern-where-python is deprecated and doesn’t work in 0.62.0.  This
should be faster, too.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-24 07:51:48 -07:00
Anders Kaseorg 3e78de4ce8 sync_ldap_user_data: Log all exceptions.
This is a roundabout way to appease a semgrep complaint about
‘error_msg = error_msg % (string_id,)’ while also improving the code.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-24 07:51:48 -07:00
Anders Kaseorg 9406c21340 fetch-contributor-data: Normalize names to NFC.
Fixes an (admittedly picky) HTML validator warning on
https://zulip.com/team/.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 16:19:26 -07:00
Anders Kaseorg a9b1ac288e fetch-contributor-data: Organize imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 16:19:26 -07:00
Anders Kaseorg 7170fb01f1 icons: Set font-display: block for icon font.
It’s not helpful for the browser to substitute another font for the
icon font while it’s loading.

This suppresses a warning from the Lighthouse performance analyzer.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 15:59:54 -07:00
Anders Kaseorg cea9fe5f13 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 14:30:02 -07:00
Anders Kaseorg ad480a8b54 install-shfmt: Upgrade shfmt from 3.3.0 to 3.3.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 12:34:05 -07:00
Anders Kaseorg 7b2e585213 install-yarn: Upgrade Yarn from 1.22.10 to 1.22.11.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 12:33:27 -07:00
Anders Kaseorg ebb8e9109c install-node: Upgrade Node.js from 14.17.3 to 14.17.5.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-23 12:29:04 -07:00
Mateusz Mandera 7ef1a024db management: Rename clear_auth_rate_limit_history command. 2021-08-23 11:52:35 -07:00
Sahil Batra 9cc4290140 settings: Rename "Send test notification" to "Test desktop notification". 2021-08-21 07:48:38 -07:00
Eeshan Garg 8697d98652 webhooks/clubhouse: Ignore label removals for story batch updates.
9ac55a8cf6 introduced support for
batch updates to stories. However, that commit didn't skip label
removals, as we already do in non-batch story payloads. This led
to an exception for batch story update payloads where labels were
removed but none were added.
2021-08-20 23:27:14 -07:00
Anders Kaseorg 7df2be0965 prettier: Exclude backend-processed Markdown files.
Our backend processor is not yet sufficiently CommonMark compliant to
accept Prettier formatted Markdown files.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 08fb51483b prettier: Disable embedded language formatting for Markdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 676fc93e1f editorconfig: Restore indent_size = 2 for Markdown.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg fdb7ec8c9e docs: Avoid [GitHub] as an internal Markdown link reference name.
To avoid confusing the linter later when Prettier lowercases these.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 0e4a30daad reading-list: Inline links.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 32da7e3eb2 docs: Adjust colon fence for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 7b3d4ff1de docs: Escape asterisks for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 0a3e022376 docs: Adjust list item indentation for Prettier compatibility.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 4bfffc9f74 docs: Fix list item indentation mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg 962f14995e docs: Fix “sinternet” typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Anders Kaseorg fe3db63381 docs: Add missing blockquote.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-08-20 23:14:37 -07:00
Vishnu KS 5c8d1b7ca3 oneclick: Upgrade the base image to ubuntu-20-04-x64. 2021-08-20 22:53:39 -07:00
Vishnu KS bf1f7e4571 oneclick: Create subdomain on oneclick.zulip.dev for test droplets. 2021-08-20 22:53:39 -07:00
Sahil Batra fef825d56f settings: Rename class of notification settings elements.
We rename class of notification settings except checkboxes
by prefixing them with 'setting_' for clarity.

We do not change class of checkboxes because settings_checkbox
is used by other templates also and if we only change class
of those using notification_settings_checkboxes then live
update code will break and will need to add separate condition
for differentiating between which partial template is used.
2021-08-20 07:51:23 -07:00
Sahil Batra d05730a199 settings: Rename classes for display settings elements.
We rename the class of display setting elements by prefixing
them with 'setting_' for clarity.
2021-08-20 07:51:11 -07:00
Sahil Batra 9c17e7019d settings: Add 'user_' prefix to presence_enabled setting element.
This commit adds 'user_' prefix to presence_enabled setting
element such that we can have unique ids when we add a realm-level
setting for this.
2021-08-20 07:50:51 -07:00
Sahil Batra 02ce8531cc notification: Rename notification-sound audio and child elements.
This is a prep commit for adding UI for realm-level defaults
of user settings.
This commit renames notification-sound-audio element to
user-notification-sound-audio because we will be adding similar
element for realm-level default of user settings also.

We also change the child elements to have current id as their
class for the same reason.
2021-08-20 07:50:51 -07:00
Sahil Batra 7b79980be9 settings: Add prefix to checkboxes used in notification_settings.
This commit adds prefix paramter with value "user_" to the
settings_checkbox and notification_settings_checkboxes references
in notification_settings.hbs such that ids are unique when we use
notification_settings.hbs for realm-level settings UI.
2021-08-20 07:50:51 -07:00
Sahil Batra 32a43985d7 settings: Create user_notification_settings.hbs template.
This commit creates a new template user_notification_settings.hbs
for user notification settings and notification_settings.hbs will
be used as a common template for user-level and realm-level
settings.
2021-08-20 07:50:51 -07:00