Commit Graph

33801 Commits

Author SHA1 Message Date
Anders Kaseorg f15cfc00ef test-documentation: Invoke sphinx-build via make instead of directly.
This lets us pick up the -W option from docs/Makefile.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-02 18:39:33 -08:00
Anders Kaseorg fdaca911e4 docs: Replace Makefile based on output of newer sphinx-quickstart.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-02 18:39:33 -08:00
Anders Kaseorg 7d5450917a test_messages: Fix list ordering flake in test_wildcard_mention.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-02 18:33:40 -08:00
Mateusz Mandera b5307b1b5a changelog: Document LDAP settings changes in upgrade notes.
Text tweaked by tabbott.
2019-12-02 17:55:23 -08:00
Mateusz Mandera 74dd21c8fa register: Allow creating non-ldap users via social backends.
In configurations that use the ldap authentication backend and a social
backend, make it possible to create non-ldap users via the social backend.
2019-12-02 17:44:11 -08:00
Mateusz Mandera fcc91ae370 ldap: Disallow creating non-ldap accounts matching LDAP_APPEND_DOMAIN.
In configurations with LDAP_APPEND_DOMAIN, we don't want people creating
non-ldap accounts with emails matching the ldap domain.
So in the registration flow, if the email isn't found in LDAP, but
matches LDAP_APPEND_DOMAIN, we stop, rather than proceeding with account
creation. In case of emails not matching LDAP_APPEND_DOMAIN, we will
still continue to make a normal, non-ldap account.
2019-12-02 17:44:11 -08:00
Mateusz Mandera 82674b9b83 register: Improve handling of non-ldap users in LDAPPopulator configs.
The problem was that, for example, given a configuration of social
backend + LDAPPopulator, if a user that's not in ldap was being
registered, the Full Name field in the registration form would be
empty instead of getting prefilled with the name provided by the
social backend.

This fixes it - first we try to get the name from ldap. If that
succeeds, a form is created pre-filled with that name.  Otherwise, we
proceed to attempt to pre-fill with other means.

This also has a nice side effect of reorganizing most of the logic to
be more parallel between LDAP and other sources of name data.
2019-12-02 17:36:53 -08:00
David Rosa 1596da9450 docs: Remove build warning.
Temporary workaround to remove multiple
"UserWarning: Container node skipped: type=document"
caused by upstream bug readthedocs/recommonmark#177.

Addresses #13263.
2019-12-02 17:29:55 -08:00
Rohitt Vashishtha 68e93d2435 update-message: Use MentionData in the update_message_backend code.
This is a performance optimization, since we can avoid doing work
related to wildcard mentions in the common case that the message can't
have any.  We also add a unit test for adding wildcard mentions in a
message edit.
2019-12-02 12:12:35 -08:00
Rohitt Vashishtha bb42539b3f do_send_messages: Populate possible_wildcard_mentions from MentionData.
Fixes #13430.
2019-12-02 12:12:35 -08:00
Rohitt Vashishtha 9174c636ce bugdown: Store if message has wildcards in MentionData.
We also switch the underlying exctact_mention_text method to use
a regular for loop, as well as make the related methods return
tuples of (names, is_wildcard). This abstraction is hidden from the
MentionData callers behind mention_data.message_has_wildcards().

Concerns #13430.
2019-12-02 12:12:35 -08:00
Anders Kaseorg fb9e2b68fd docs: Remove broken link in changelog.
This section was removed by commit
b036fa897e.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-02 12:06:24 -08:00
David Rosa 1be4e10a2d docs: Explain link sharing in /api/upload-file.
Rewrittten by tabbott to clearly explain the security model, and add a
code example.
2019-12-02 12:01:46 -08:00
Tim Abbott b036fa897e docs: Move unattended-upgrades docs to troubleshooting guide.
This also rewrites the text to better explain what's happening.  It's
likely further polish would be valuable, but that's true for the whole
"Troubleshooting" page.

This block of text was misplaced when we split the long
maintain-secure-update; article; we want it to be easy to find by
folks who are looking into error emails Zulip is sending.
2019-12-02 11:37:16 -08:00
David Rosa 62d555115a docs: Improve formatting of important notes.
Improves formatting using reST and Sphinx admonition directives
so that important notes stand out when viewed in ReadTheDocs.
2019-12-02 10:37:49 -08:00
joaomcarvalho cd2c68c778 stream settings: Fix initialization of main toggler state.
The "Stream settings" UI was always intended to be initialized in the
"Subscribed" tab when opened not through navigation that explicitly
aims to via "All streams".  We had implemented that through how the UI
is rendered as well as the internal state tracking variable
`subscribed_only`, which was initialized to `true`.

The bug was that we didn't reset that to `true` when re-opening
"Stream settings" via a code path that calls `setup_page` (e.g. via
the menus on the left sidebar).

Ths fixes a bug where the stream-list in the stream settings would
list all streams but would show the 'Subscribed' label after
navigating to "All streams", closing "Manage streams", and then
reopening it.

Fixes #13297.
2019-12-02 09:59:13 -08:00
Vishnu Ks a26b379a14 support: Send confirmation email on realm activation. 2019-12-02 09:51:45 -08:00
Tim Abbott 8b55a310f1 typing: Fix invalid typing notifications for stream messages.
In e42c3f7418, we made the assumption
that compose_pm_pill.get_recipient() would return no users for stream
messages.  It turns out, due to the confusing name of
compose_state.recipient (which we just renamed to
compose_state.private_message_recipient), this assumption was wrong.

As a result, when composing a stream message using the reply hotkeys,
we'd end up sending typing notiifcations to the person who sent the
message we're replying to as though a PM was being composed.

We fix this by avoiding passing an (expected to be unused) value for
private_message_recipient to compose_state.start.
2019-12-02 09:31:16 -08:00
Tim Abbott ea7c6d395f compose_state: Rename compost_state.recipient to be about PMs only.
The compose_state.recipient field was only actually the recipient for
the message if it was a private_message_recipient (in the sense of
other code); we store the stream in compose_state.stream instead.

As a result, the name was quite confusing, resulting in the
possibility of problematic correctness bugs where code assumes this
field has a valid value for stream messages.  Fix this by changing it
to compose_state.private_message_recipient for clarity.
2019-12-02 08:53:55 -08:00
Anders Kaseorg 65270419b9 docs: Document dev environment issues that Windows users are hitting.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-01 23:05:22 -08:00
Anders Kaseorg a7d36495c9 docs: Remove discussion of NFS-related issues.
We’re not using NFS anymore (#12963).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-01 23:05:22 -08:00
Tim Abbott 38d2d23ad3 Release Zulip server 2.1.0-rc1. 2019-11-22 15:33:36 -08:00
Tim Abbott c367e55892 i18n: Update translation data from Transifex. 2019-11-22 15:33:36 -08:00
Mohit Gupta 452e226ea2 narrow: Fix to show last message in narrow when narrow allows.
Fixes commit id 648a60baf6. When
allow_use_first_unread_when_narrowing() is false last message of
narrow is shown in view.

Comments rewritten by tabbott to explain in detail what's happening.
2019-11-22 12:31:43 -08:00
Anders Kaseorg c81f967a1f docs: Use ZULIP_VERSION.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-22 12:26:54 -08:00
Anders Kaseorg 35e2013199 tsconfig: Fix typescript-eslint memory usage disaster.
typescript-eslint was using hundreds of megabytes to track `*.js`
files that it shouldn’t be checking.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-22 11:38:25 -08:00
Tim Abbott bf117fed96 api docs: Fix link to client libraries.
This should have been caught by CI.
2019-11-22 11:08:44 -08:00
Tim Abbott c9a3e4e872 docs: Reorganize and refocus section on management commands. 2019-11-22 10:48:37 -08:00
Tim Abbott 5a7b5f1337 docs: Rework text for scalability and monitoring sections.
This text is very old and hadn't been edited in a long time, in large
part because it was buried within old docs.  This change cleans it up
to give accurate and better-organized information.
2019-11-22 10:22:07 -08:00
David Rosa 87a2831b83 docs: Split maintain-secure-upgrade into dedicated docs.
* Moves "Management commands" to a top-level section.
* Moves "Scalability" as a subsection at the bottom of "Requirements".
* Moves "Monitoring" as a subsections at the bottom of "Troubleshooting".
* Replaces "API and your Zulip URL" with a link to REST API docs.  This
  documentation text has been irrelevant for some time.
* Removes maintain-secure-upgrade from the TOC but the file remains to
  avoid breaking old links from release blog posts and emails.
2019-11-22 10:21:20 -08:00
David Rosa 1c45e4ac11 docs: Add top level page for language bindings.
Lists API client bindings/libraries/wrappers and makes a distinction
between official core projects and community projects.

Tweaked by tabbott to sort the community and outdated libraries, and
restructure how we talk about other projects.

Fixes #2093.
2019-11-22 04:33:18 -08:00
Tim Abbott 263ac0eb45 pm_conversations: Initialize using server data.
This simple change switches us to take advantage of the
server-maintained data for the pm_conversations system we implemented
originally for mobile use.

This should make it a lot more convenient to find historical private
message conversations, since one can effectively scroll infinitely
into the history.

We'll need to do some profiling of the backend after this is deployed
in production; it's possible we'll need to add some database indexes,
denormalization, or other optimizations to avoid making loading the
Zulip app significantly slower.

Fixes #12502.
2019-11-21 17:01:41 -08:00
Tim Abbott 93b83b28a7 pm_conversations: Refactor to sort by message ID.
message_id, rather than timestamps, is our standard way to sort by
time.  And this refactor is important because we're about to start
using data from the server to populate this data structure.
2019-11-21 17:01:41 -08:00
Tim Abbott 89ff62dafa topic_list: Limit number of unread topics shown at once.
This avoids a stream having potentially near-infinite height when
opened in a stream with a large number of unread topics; the benefit
is that you can easily access the next stream.

We show an unread count next to "more topics" to make it hard to miss
that there might be more, older topics with unread messages.

With CSS work by Anders Kaseorg.

Fixes #13087.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-21 13:12:33 -08:00
Tim Abbott 1dfbeaf05b version: Update LATEST_RELEASE_VERSION for 2.0.7. 2019-11-21 12:52:00 -08:00
Mateusz Mandera 67b6179df2 ldap: Fix error while updating a user registered in multiple realms.
Previously, the LDAP code for syncing user data was not
multiple-realm-aware, resulting in errors trying to sync data for an
LDAP user present in multiple realms.

Tweaked by tabbott to add some extended comments.

Fixes #11520.
2019-11-21 11:13:31 -08:00
Tim Abbott 68d9ed8c4c docs: Update changelog for Zulip 2.0.7 release. 2019-11-21 10:36:09 -08:00
Anders Kaseorg bae5e40f25 version: Set ZULIP_VERSION = "2.1.dev+git".
Since we don’t support downgrading from master to any 2.0.x release,
we shouldn’t set a ZULIP_VERSION that might lead someone to mistake
any such downgrade for an upgrade.  ZULIP_VERSION should always be at
least a minor version ahead of LATEST_RELEASE_VERSION, except on the
release branch.

`.dev` is a decreasing suffix that sorts before `alpha`, `beta`, `rc`
according to PEP 440/`packaging.version.Version`.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-21 10:33:55 -08:00
Mateusz Mandera 06c2161f7e auth: Use zxcvbn to ensure password strength on server side.
For a long time, we've been only doing the zxcvbn password strength
checks on the browser, which is helpful, but means users could through
hackery (or a bug in the frontend validation code) manage to set a
too-weak password.  We fix this by running our password strength
validation on the backend as well, using python-zxcvbn.

In theory, a bug in python-zxcvbn could result in it producing a
different opinion than the frontend version; if so, it'd be a pretty
bad bug in the library, and hopefully we'd hear about it from users,
report upstream, and get it fixed that way. Alternatively, we can
switch to shelling out to node like we do for KaTeX.

Fixes #6880.
2019-11-21 10:23:37 -08:00
Mateusz Mandera 0c2cc41d2e CVE-2019-18933: Fix insecure account creation via social authentication.
A bug in Zulip's new user signup process meant that users who
registered their account using social authentication (e.g. GitHub or
Google SSO) in an organization that also allows password
authentication could have their personal API key stolen by an
unprivileged attacker, allowing nearly full access to the user's
account.

Zulip versions between 1.7.0 and 2.0.6 were affected.

This commit fixes the original bug and also contains a database
migration to fix any users with corrupt `password` fields in the
database as a result of the bug.

Out of an abundance of caution (and to protect the users of any
installations that delay applying this commit), the migration also
resets the API keys of any users where Zulip's logs cannot prove the
user's API key was not previously stolen via this bug.  Resetting
those API keys will be inconvenient for users:

* Users of the Zulip mobile and terminal apps whose API keys are reset
  will be logged out and need to login again.
* Users using their personal API keys for any other reason will need
  to re-fetch their personal API key.

We discovered this bug internally and don't believe it was disclosed
prior to our publishing it through this commit.  Because the algorithm
for determining which users might have been affected is very
conservative, many users who were never at risk will have their API
keys reset by this migration.

To avoid this on self-hosted installations that have always used
e.g. LDAP authentication, we skip resetting API keys on installations
that don't have password authentication enabled.  System
administrators on installations that used to have email authentication
enabled, but no longer do, should temporarily enable EmailAuthBackend
before applying this migration.

The migration also records which users had their passwords or API keys
reset in the usual RealmAuditLog table.
2019-11-21 10:23:37 -08:00
Anders Kaseorg 16ea89ad89 js: Automatically convert var to let and const in remaining files.
This commit was automatically generated by `tools/lint --only=eslint
--fix`, except for the `.eslintrc.json` change itself.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 23:04:01 -08:00
Jack Tiggleman 1682d75ea8 message_edit: Add message edit local echo.
Updates the message editing process to do a local 'echo'.

On slow connections, now there is visual confirmation of the edit,
similar to when sending messages.  The contains_backend_only_syntax
logic and check are the same as there.

We showing "(SAVING)" until the edit is completed, and on successful
edit, the word "(EDITED)" appears.  There's likely useful future work
to do on making the animation experience nicer.

Substantially rewritten by tabbott to better handle corner cases and
communicate more clearly about what's happening.

Fixes: #3530.
2019-11-20 17:40:19 -08:00
Tim Abbott f0fd812cc5 tornado: Add transitional code for sender_delivery_email.
This issue was introduced in 54e357e154.
2019-11-20 17:31:11 -08:00
Anders Kaseorg a681ca6cf5 queue: Update error callback signatures for Pika 1.1.
The expected signatures for these callbacks seem to have changed
somewhere in https://github.com/pika/pika/pull/1002.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 17:23:48 -08:00
Tim Abbott bf1386405c settings_notifications: Fix linter issue. 2019-11-20 17:16:43 -08:00
Tim Abbott 55a262d47d message_edit: Move save lower in the file. 2019-11-20 17:06:08 -08:00
Tim Abbott 124f5d12a4 message_edit: Adjust API of edit_locally.
This makes it more extensible for future use of locally echoing edits
to fully sent messages.
2019-11-20 17:06:08 -08:00
Vinit Singh 19234f8705 sidebar: Move the buddy list tooltip content logic to JS.
Moved the logic from static/templates/buddy_list_tooltip_content.hbs to
the get_title_data function to simplify the template.

Fixes #13426.
2019-11-20 17:04:31 -08:00
Tim Abbott 1fe4f795af settings: Add notification settings checkboxes for wildcard mentions.
This change makes it possible for users to control the notification
settings for wildcard mentions as a separate control from PMs and
direct @-mentions.
2019-11-20 16:58:46 -08:00
Tim Abbott 67efed0b64 node tests: Reformat some notification settings lists.
This just cleans up line-wrapping.
2019-11-20 15:44:44 -08:00