Commit Graph

33410 Commits

Author SHA1 Message Date
Chris Bobbe 04e6c3bd18 docs/version-control: Set commit message line limit to 70.
https://github.com/zulip/zulip-mobile/pull/3404#issuecomment-477411875

A discussion followed at

https://chat.zulip.org/#narrow/stream/3-backend/topic/commit.20line.20length

and that was agreed on for the commit message body, but noted that a
length of 76 was acceptable for the summary because of the
single-line constraint and the fact that it ends in a period, making
it clear where it ends.
2019-09-16 15:02:02 -07:00
Wyatt Hoodes d2c3ddc765 test_events: Use stdout_suppressed in export test. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes e37c91d195 test_realm_export: Use stdout_suppressed in tests.
From a testing perspective, calling `export_realm_wrapper`
prints unnecessary data to stdout.
2019-09-13 11:54:14 -07:00
Wyatt Hoodes e3595e4b3f test_slack_message_conversion: Use subTest for fixtures test. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes dbaf6ac7e7 test_push_notifications: Remove fixtures print statement. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes 4a64a487d3 bugdown: Remove print statement from fixtures test. 2019-09-13 11:54:14 -07:00
Wyatt Hoodes 698b0f1b91 test_management_commands: Use subTest with help tests.
We remove the print statement, and instead use
subTest to improve debuggability.
2019-09-13 11:54:14 -07:00
Anders Kaseorg 2ec946ad4d postgres-init-db: Require an Enter press in confirmation prompt.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-12 16:04:55 -07:00
Anders Kaseorg 78b018989e export: Refuse to overwrite an existing directory or tarball.
Previously, incorrectly passing an existing directory to the
`manage.py export --output` option would remove its contents without
warning.  Abort instead.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-12 16:04:55 -07:00
Tim Abbott fcf5936341 version: Update version and changelog for Zulip 2.0.5 release. 2019-09-11 16:36:19 -07:00
Anders Kaseorg 780ecb672b CVE-2019-16216: Fix MIME type validation.
* Whitelist a small number of image/ types to be served as
  non-attachments.
* Serve the file using the type that we validated rather than relying
  on an independent guess to match.

This issue can lead to a stored XSS security vulnerability for older
browsers that don't support Content-Security-Policy.

It primarily affects servers using Zulip's local file uploads backend
for servers running Ubuntu 16.04 Xenial or newer; the legacy local
file upload backend for (now EOL) Ubuntu 14.04 Trusty was not affected
and it has limited impact for the S3 upload backend (which uses an
unprivileged S3 bucket domain to serve files).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-11 15:46:36 -07:00
Mateusz Mandera a1fa0b011e docs: Fix typo in production/email.md. 2019-09-10 16:10:12 -07:00
Anders Kaseorg 6bab61a0d6 styles: Remove overrides for KaTeX line-height and white-space.
Commit ba66dfe977 incorrectly inflated
the specificity level of these rules by moving them inside
.rendered_markdown “entirely for readability”.  KaTeX has its own
rules that work better, so just delete ours.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-10 16:03:20 -07:00
Thomas Ashish Cherian ea17673885 docs: Include su in email docs restart-server command.
It's likely the user was root as this point in the instructions.
2019-09-10 16:00:34 -07:00
Anders Kaseorg 17192ed46b Revert "provision: Give concrete NFS error message on older OSX versions."
This reverts commit 073ecaac66 (#9365).

This exception handler was overly broad in catching all `OSError`s,
and it made debugging harder by hiding the actual exception.
Furthermore, we no longer use NFS (#12963), and we’re now getting
reports of Windows users running into this message.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-09 14:47:51 -07:00
Thomas Ip 574c35c0b8 markdown: Render ordered lists using <ol> markup.
This brings us in line, and also allows us to style these more like
unordered lists, which is visually more appealing.

On the backend, we now use the default list blockprocessor + sane list
extension of python-markdown to get proper list markup; on the
frontend, we mostly return to upstream's code as they have followed
CommonMark on this issue.

Using <ol> here necessarily removes the behaviour of not renumbering
on lists written like 3, 4, 7; hopefully users will be OK with the
change.

Fixes #12822.
2019-09-08 16:42:20 -07:00
Wyatt Hoodes 4a176714e1 actions: Remove leftover print statement. 2019-09-08 15:36:30 -07:00
Tim Abbott 9c21fb75cf requirements: Upgrade twisted to 19.7.0. 2019-09-08 09:42:32 -07:00
Tim Abbott 7e75f987df ldap: Fix logging of warning for deactivated users.
Also cleans up the interface between the management command and the
LDAP backends code to not guess/recompute under what circumstances
what should be logged.

Co-authored-by: mateuszmandera <mateusz.mandera@protonmail.com>
2019-09-08 09:35:23 -07:00
Tim Abbott d1a2784d52 ldap: Fix attempting to sync data for deactivated users.
The order of operations for our LDAP synchronization code wasn't
correct: We would run the code to sync avatars (etc.) even for
deactivated users.

Thanks to niels for the report.

Co-authored-by: mateuszmandera <mateusz.mandera@protonmail.com>
2019-09-08 09:35:23 -07:00
Anders Kaseorg 7af04690b9 requirements: Use PyPI fork of line_profiler supporting Python 3.7.
Also move it to dev.in.

Other notes for posterity: this should have been installed with a
pinned commit hash, and could have been installed directly from the
upstream Git repository, even on Python 3.7, as long as Cython was
installed as well.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-08 09:34:55 -07:00
Tim Abbott 94251a9341 docs: Add a full article on modifying Zulip.
We have lots of documentation for Zulip developers; but previously
didn't have a nice top-level page for Zulip server administrators to
learn how to manage patches to Zulip.
2019-09-06 13:52:56 -07:00
Rishi Gupta 81f06ab375 readme: Remove Travis badge.
Travis hasn't been our primary CI system for a long time, and flakes,
and the badges look duplicate with the CircleCI one, so we should just
remove it.

See discussion at
https://chat.zulip.org/#narrow/stream/101-design/topic/github.20badges.
2019-09-06 13:14:14 -07:00
Rishi Gupta 34dee2d953 readme: Advertise digital ocean droplet and docker install. 2019-09-06 13:14:01 -07:00
Mateusz Mandera 2ce2024bd7 ldap: Fix unintended user deactivation in case of connection failure.
Fixes #13130.

django_auth_ldap doesn't give any other way of detecting that LDAPError
happened other than catching the signal it emits - so we have to
register a receiver. In the receiver we just raise our own Exception
which will properly propagate without being silenced by
django_auth_ldap. This will stop execution before the user gets
deactivated.
2019-09-05 11:59:20 -07:00
Tim Abbott 7ca65b2bb5 css: Remove buggy stream settings media CSS for narrow windows.
This caused weird behavior in the relevant band of window widths, and
removing it works considerably better.

There's still bad behavior in handling situations where the stream
name is too long and thus this wraps, but we should address that
as a follow-up.
2019-09-05 11:57:27 -07:00
Tim Abbott fd5f9be14f stream settings: Use fa-circle-o for link to /help/. 2019-09-05 11:48:32 -07:00
Tim Abbott 94c51676fe stream settings: Use <label> tags for section labels.
Previously, these were configured as divs.
2019-09-05 11:48:32 -07:00
Tim Abbott 8b5df88596 css: Make label CSS usable outside settings model.
We intend to use it in the "manage streams" modal as well.
2019-09-05 11:48:32 -07:00
Mateusz Mandera bf7f4f3f1b stream settings: Replace email address hint popup with link to docs.
Fixes #13134 as the last commit in the series for this issue.
Solves the "The (?) should just be a target=_blank link to
/help/message-a-stream-by-email." part of the issue.
As a result, a bunch code managing the email hint popup can be deleted,
together with a node test for that.
2019-09-05 11:48:32 -07:00
Mateusz Mandera dfd6771237 stream settings: Use the .show-sender version of email address.
As requested in #13134.
2019-09-05 11:48:32 -07:00
Mateusz Mandera 307d8d8758 test_email_mirror: Better document the various address formats. 2019-09-05 11:48:32 -07:00
Vishnu KS 30440cf466 mattermost: Fix test flake in test_convert_channel_data.
So the reason 38f8cf612c seems
to be flaking is because the value of harry id switches between
1 and 2 in Xenial while in Bionic it would be fixed at 2. The
reason behind this is that Bionic ships with Python3.6 which
preserves dict insert order while Python3.5 that ships with Xenial
dont preserve the order. In initialize_stream_membership_dicts
we iterate user_data_map dict and the order in which the iteration
happens affects the ID of the users.
2019-09-05 11:20:33 -07:00
Puneeth Chaganti df134be235 webhooks/papertrail: Requests from Papertrail are not JSON requests.
Papertrail sends requests with the content type
`application/x-www-form-urlencoded`, with the payload parameter holding the
JSON body. This commit fixes the papertrail integration to use the payload
parameter in the request's POST data instead of trying to parse the
request's entire body as JSON.

Papertrail documentation here:
https://help.papertrailapp.com/kb/how-it-works/web-hooks#encoding
2019-09-04 13:16:17 -07:00
Rohitt Vashishtha b3df3f2e22 docs/botserver: Document running custom bot modules.
This documents the feature added in
https://github.com/zulip/python-zulip-api/pull/523.
2019-09-03 13:28:01 -07:00
Wyatt Hoodes fc2574fdfd test_mattermost_importer: Add test coverage.
This is a rebase from a recent revert. A test
was failing non-deterministically relating to
`convert_channel_data` and use of an incorrect
`realm_id`.
2019-09-03 12:36:41 -07:00
Mateusz Mandera a7ec1a089c errors: Fix infinite recursion during memcached outages.
We have a very useful piece of code, _RateLimitFilter, which is
designed to avoid sending us a billion error emails in the event that
a Zulip production server is down in a way that throws the same
exception a lot. The code uses memcached to ensure we send each
traceback roughly once per Zulip server per 10 minutes (or if
memcached is unavailable, at most 1/process/10 minutes, since we use
memcached to coordinate between processes)

However, if memcached is down, there is a logging.error call internal
to the Django/memcached setup that happens inside the cache.set() call,
and those aren't caught by the `except Exception` block around it.

This ends up resulting in infinite recursion, eventually leading to
Fatal Python error: Cannot recover from stack overflow., since this
handler is configured to run for logging.error in addition to
logging.exception.

We fix this using a thread-local variable to detect whether we are
being called recursively.

This change should prevent some nasty failure modes we've had in the
past where memcached being down resulted in infinite recursion
(resulting in extra resources being consumed by our error
notifications code, and most importantly, the error notifications not
being sent).

Fixes #12595.
2019-09-03 11:44:39 -07:00
Anders Kaseorg 177eb2e02e webpack: Move CSS minification to optimization stage.
Otherwise Bootstrap doesn’t get minified, and also the minification
state is incorrectly reflected in the webpack cache.

The Terser plugin is used by default; we need to include it explicitly
to avoid removing it.

Switch from cssnano to clean-css because it’s noticeably faster.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 21:58:13 -07:00
Anders Kaseorg ad116cac4c postcss: Add postcss-calc plugin.
As foreshadowed by commit d0634181b5
(#13115).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 21:48:05 -07:00
Anders Kaseorg 2bdb115581 styles: Remove stray semicolon from input.user_status block.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 21:45:43 -07:00
Anders Kaseorg 9b0401b76d package.json: Use caret ranges for more deduplication opportunities.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 19:30:09 -07:00
Anders Kaseorg e57ede9421 yarn.lock: Share duplicate packages with yarn-deduplicate.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 19:30:09 -07:00
Anders Kaseorg e5ab9834c5 logos: Golf harder with svgo 1.3.0.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-02 19:30:09 -07:00
Tim Abbott 70c513a640 analytics: Fix logging for errors connecting to push bouncer.
There's no reason for this to be a category of error that emails the
server administrator, since there's a good chance that fixing it will
need to be done in the Zulip codebase, not administrator action.
2019-09-02 18:47:10 -07:00
Anders Kaseorg d0634181b5 styles: Fix left sidebar indentation for PostCSS migration.
cssnano reduces this to a constant in a production build.  (We could
add postcss-calc if we wanted this reduced in development.)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-30 15:06:29 -07:00
Mateusz Mandera d70e1bcdb7 settings: Add FAKE_EMAIL_DOMAIN setting.
Fixes #9401.

This adds a FAKE_EMAIL_DOMAIN setting, which should be used if
EXTERNAL_HOST is not a valid domain, and something else is needed to
form bot and dummy user emails (if email visibility is turned off).
It defaults to EXTERNAL_HOST.

get_fake_email_domain() should be used to get this value. It validates
that it's correctly set - that it can be used to form valid emails.

If it's not set correctly, an exception is raised. This is the right
approach, because it's undesirable to have the server seemingly
peacefully operating with that setting misconfigured, as that could
mask some hidden sneaky bugs due to UserProfiles with invalid emails,
which would blow up the moment some code that does validate the emails
is called.
2019-08-30 14:59:00 -07:00
Anders Kaseorg 4de3bbeafa styles: Finish removing manual antialiasing configuration.
Followup from commit ddb965110f.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-30 14:51:52 -07:00
Anders Kaseorg abbd8a7f45 styles: Remove most vendor-prefixed CSS attributes.
Many of them are now automatically generated by autoprefixer, while
others are unnecessary based on .browserslistrc, and some were just
wrong (the linear-gradient based checkerboard pattern in lightbox has
been broken in Firefox for a while).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-30 14:51:52 -07:00
Tim Abbott 62c9ea7cf9 linkifiers: Fix problems with capture groups called "name".
Apparently, due to poor naming of the outer capture group we use to
separate the actual match from the surrounding whitespace (etc.) we
use to determine if the syntax is a possible linkifier start/end, if
you created a linkifier using "name" as the capture group, we'd try to
compile a pattern with two capture groups called "name", which would
500, preventing anyone from accessing the organization.
2019-08-30 09:36:14 -07:00
Anders Kaseorg 096ef1445f parse_os_release: Use /etc/os-release always; remove DISTRIB_FAMILY.
To replace DISTRIB_FAMILY, there’s now an os_families function using
the standard ID and ID_LIKE information in /etc/os-release.

Fixes #13070; fixes #13071.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-29 17:30:20 -07:00