Commit Graph

10581 Commits

Author SHA1 Message Date
Tim Abbott 263ac0eb45 pm_conversations: Initialize using server data.
This simple change switches us to take advantage of the
server-maintained data for the pm_conversations system we implemented
originally for mobile use.

This should make it a lot more convenient to find historical private
message conversations, since one can effectively scroll infinitely
into the history.

We'll need to do some profiling of the backend after this is deployed
in production; it's possible we'll need to add some database indexes,
denormalization, or other optimizations to avoid making loading the
Zulip app significantly slower.

Fixes #12502.
2019-11-21 17:01:41 -08:00
Mateusz Mandera 67b6179df2 ldap: Fix error while updating a user registered in multiple realms.
Previously, the LDAP code for syncing user data was not
multiple-realm-aware, resulting in errors trying to sync data for an
LDAP user present in multiple realms.

Tweaked by tabbott to add some extended comments.

Fixes #11520.
2019-11-21 11:13:31 -08:00
Mateusz Mandera 06c2161f7e auth: Use zxcvbn to ensure password strength on server side.
For a long time, we've been only doing the zxcvbn password strength
checks on the browser, which is helpful, but means users could through
hackery (or a bug in the frontend validation code) manage to set a
too-weak password.  We fix this by running our password strength
validation on the backend as well, using python-zxcvbn.

In theory, a bug in python-zxcvbn could result in it producing a
different opinion than the frontend version; if so, it'd be a pretty
bad bug in the library, and hopefully we'd hear about it from users,
report upstream, and get it fixed that way. Alternatively, we can
switch to shelling out to node like we do for KaTeX.

Fixes #6880.
2019-11-21 10:23:37 -08:00
Mateusz Mandera 0c2cc41d2e CVE-2019-18933: Fix insecure account creation via social authentication.
A bug in Zulip's new user signup process meant that users who
registered their account using social authentication (e.g. GitHub or
Google SSO) in an organization that also allows password
authentication could have their personal API key stolen by an
unprivileged attacker, allowing nearly full access to the user's
account.

Zulip versions between 1.7.0 and 2.0.6 were affected.

This commit fixes the original bug and also contains a database
migration to fix any users with corrupt `password` fields in the
database as a result of the bug.

Out of an abundance of caution (and to protect the users of any
installations that delay applying this commit), the migration also
resets the API keys of any users where Zulip's logs cannot prove the
user's API key was not previously stolen via this bug.  Resetting
those API keys will be inconvenient for users:

* Users of the Zulip mobile and terminal apps whose API keys are reset
  will be logged out and need to login again.
* Users using their personal API keys for any other reason will need
  to re-fetch their personal API key.

We discovered this bug internally and don't believe it was disclosed
prior to our publishing it through this commit.  Because the algorithm
for determining which users might have been affected is very
conservative, many users who were never at risk will have their API
keys reset by this migration.

To avoid this on self-hosted installations that have always used
e.g. LDAP authentication, we skip resetting API keys on installations
that don't have password authentication enabled.  System
administrators on installations that used to have email authentication
enabled, but no longer do, should temporarily enable EmailAuthBackend
before applying this migration.

The migration also records which users had their passwords or API keys
reset in the usual RealmAuditLog table.
2019-11-21 10:23:37 -08:00
Tim Abbott f0fd812cc5 tornado: Add transitional code for sender_delivery_email.
This issue was introduced in 54e357e154.
2019-11-20 17:31:11 -08:00
Anders Kaseorg a681ca6cf5 queue: Update error callback signatures for Pika 1.1.
The expected signatures for these callbacks seem to have changed
somewhere in https://github.com/pika/pika/pull/1002.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 17:23:48 -08:00
Tim Abbott 1fe4f795af settings: Add notification settings checkboxes for wildcard mentions.
This change makes it possible for users to control the notification
settings for wildcard mentions as a separate control from PMs and
direct @-mentions.
2019-11-20 16:58:46 -08:00
Mateusz Mandera c3e83a0e6b push_notifications: Update link to google's upstream API docs.
Due to the migration to Firebase, the old link now gives 404. We replace
that with a working link to the migrated legacy API.
2019-11-20 10:50:24 -08:00
Hashir Sarwar cc56147c44 actions: Removed redundant assignment of message['mention_data']. 2019-11-20 10:49:01 -08:00
Tim Abbott 6407d0b1f9 push_notifications: Clear PushDeviceToken on API key change.
This includes adding a new endpoint to the push notification bouncer
interface, and code to call it appropriately after resetting a user's
personal API key.

When we add support for a user having multiple API keys, we may need
to add an additional key here to support removing keys associated with
just one client.
2019-11-19 15:37:43 -08:00
Anders Kaseorg 6717daf4a6 test_events: Fix apparent typo.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-19 15:34:05 -08:00
Tim Abbott 0338e8a6f6 send_custom_email: Add support for specifying reply-to. 2019-11-18 17:34:01 -08:00
Tim Abbott e266fb4343 management: Add command to send a custom email to users.
This allows us to email sets of users on a server with a nicely
formatted email similar to our onboarding emails, built off of a
Markdown template.

The code was based on send_password_reset_email, but it doesn't
replace that use case, since one cannot include special values like
password reset tokens in these emails.
2019-11-18 15:35:54 -08:00
Matheus Melo 39bd565941 test_realm: Automate tests for invalid integer values in Realm.
Fixes #13362.
2019-11-18 15:21:11 -08:00
Matheus Melo a890652bfd realm: Use check_int_in validator for enum format views.
This lets us remove some ugly semi-duplicated code.

Modified by tabbott to include some additional endpoints in the
changes.
2019-11-18 15:21:00 -08:00
Matheus Melo 678c3a89d0 validators: Add check_int_in validator.
This is a useful helper for our enum format fields where we want to
only allow a fixed list of integer values.
2019-11-18 15:14:56 -08:00
Matheus Melo 31558cb8b9 decorator: Extract require_user_group_edit_permission.
We move the check that the user is a member or admin inot this
decorator.

This name better communicates that this may do other checks beyond
just verifying the policy.
2019-11-18 15:13:29 -08:00
Matheus Melo 21ed834101 decorator: Extract OrganizationAdministratorRequired common exception.
This eliminates significant code duplication of error messages for
situations where an organization administrator is required.
2019-11-18 15:10:56 -08:00
Vishnu Ks 08103544cc tests: Remove upload-file from curl test exclude_list. 2019-11-18 12:23:38 -08:00
Vishnu Ks 76c610c953 tests: Remove upload-custom-emoji from curl test exclude_list. 2019-11-18 12:23:38 -08:00
Vishnu Ks 4c5e9e8eb0 tests: Add support for patching requestBody in curl example test. 2019-11-18 12:23:38 -08:00
Vishnu Ks 742de73bde tests: Remove update-notification-settings from curl test exclude_list. 2019-11-18 12:23:38 -08:00
Vishnu Ks c4525cceee tests: Remove get-user-groups from curl test exclude_list. 2019-11-18 12:23:38 -08:00
Vishnu Ks 3468764415 tests: Remove remove-linkifiers from curl test exclude_list. 2019-11-18 12:23:38 -08:00
Vishnu Ks 258089bdc3 tests: Remove add-linkifiers from curl test exclude_list. 2019-11-18 12:23:38 -08:00
Vishnu Ks ec53b8c574 tests: Unescape the html generated in curl example test.
Curl examples generated in test_generated_curl_examples_for_success
used to be html escaped. This commit removes the escaping in the test
since curl examples are not html escaped when run from terminal.
2019-11-18 12:23:38 -08:00
Vishnu Ks a478e861a2 tests: Remove create-user from curl test exclude_list.
The API test client uses an admin client since
0f64fe530667fd3b96a434842b124075dea84300
2019-11-18 12:23:38 -08:00
Tim Abbott 2e6e187c16 integrations: Deduplicate gogs and gitea integrations. 2019-11-18 12:08:09 -08:00
Jan Koscisz b88192d5bb integrations: Add Gitea integration.
Gitea integration adopted from Gogs integration with minor
adjusting. More events are now handled.

Fixes #13346
2019-11-18 11:55:24 -08:00
Jan Koscisz ace1503cb6 integrations: Add missing event handling to Gogs.
Issues event where not handled by Gogs integrations.
This added them to supported events.
2019-11-18 11:54:44 -08:00
Tim Abbott 4e421ebe12 scripts: Move inline-email-css from tools to scripts.
We'll be soon documenting a production workflow that involves using
it, and that means it needs to live under scripts/ (since tools/ isn't
present in release tarballs).
2019-11-15 17:39:42 -08:00
Tim Abbott f34a40581f actions: Fix a few references to user_profile.email.
The only functional change here is for what is sent to the signups
queue worker, which at present isn't used in production anywhere.
2019-11-15 17:31:41 -08:00
Tim Abbott 6618cec9db logging: Switch various logging code paths to use user IDs.
This fixes EMAIL_ADDRESS_VISIBILITY_ADMINS support as well as being
more reliable/stable over time.
2019-11-15 17:24:01 -08:00
Tim Abbott 5edc5ae1fb create_test_bot: Use delivery_email for login.
The `email` field won't actually work if
EMAIL_ADDRESS_VISIBILITY_ADMINS is enabled.
2019-11-15 17:20:23 -08:00
Tim Abbott 0e317a8773 registration: Use delivery_email in logging code path.
This is essentially an assertion failure code path, so it doesn't
really matter, but it seems best to use the value that's the cause of
the problem here.
2019-11-15 17:18:19 -08:00
Tim Abbott caf4c7af8f get_user_by_id: Use delivery_email when finding cross-realm bots.
When actually looking for bots, it doesn't matter, but this feels more
consistent with the other lookup code.
2019-11-15 17:17:05 -08:00
Tim Abbott b85c9b0810 tornado: Use delivery_email in logging.
Eventually, we'll want to replace emails with user IDs here entirely,
but until we make that happen, we should at least use the same email
address present in our other logging.

I think we won't miss updating these in a future migration thanks to
mypy types.
2019-11-15 17:16:05 -08:00
Tim Abbott 7935b3cbd7 archive: Fix calculation of gravatar URL.
This is mostly dead code right now, but in case we revive it, I found
this error in an audit.
2019-11-15 17:14:44 -08:00
Tim Abbott 993ed9c2b1 tornado: Remove stale user_profile_email field.
Since years ago, this field hasn't been used for anything other than
some logging that would be better off logging the user ID anyway.

It existed in the first place simply because we weren't passing the
user_profile_id to Tornado at all.
2019-11-15 17:07:52 -08:00
Tim Abbott df90533eef home: Switch invalid pointer logging to by user ID. 2019-11-15 17:07:52 -08:00
Tim Abbott 093e9394dd find_account: Use delivery_email when searching emails.
Previously, this code wouldn't work with
EMAIL_ADDRESS_VISIBILITY_ADMINS.
2019-11-15 17:07:52 -08:00
Tim Abbott 3bc7ba1767 digest: Switch from emails to user IDs for logging.
This is better practice.
2019-11-15 17:07:52 -08:00
Tim Abbott bb64b0fa4d queue processors: Switch SignupWorker to logging user IDs.
This is a better setup than logging emails, especially with
EMAIL_ADDRESS_VISIBILITY_ADMINS.
2019-11-15 17:07:24 -08:00
Tim Abbott b14b18b76f soft_deactivation: Remove 'email' from logging.
The value wasn't correct with EMAIL_ADDRESS_VISIBILITY_ADMINS, and in
any case we really just need the user ID.
2019-11-15 17:06:51 -08:00
Tim Abbott 8b45756dd2 onboarding: Fix welcome emails not using delivery_email.
For organizations with EMAIL_ADDRESS_VISIBILITY_ADMINS, we were using
the wrong email address in the notice telling the user how to login in
the future.
2019-11-15 16:49:28 -08:00
Tim Abbott e124837cdc management commands: Fix incorrect use of user_profile.email.
All of these management commands should be interacting with
.delivery_email; this results in buggy behavior with
EMAIL_ADDRESS_VISIBILITY_ADMINS.
2019-11-15 16:43:51 -08:00
Vishnu Ks 0632d26e5e tests: Remove update-stream from curl test exclude_list. 2019-11-15 15:53:31 -08:00
Vishnu Ks be49dd4f30 tests: Remove delete-user-group from curl test exclude_list. 2019-11-15 15:53:31 -08:00
Vishnu KS ee2d20ff3c tests: Remove update-user-group from curl test exclude_list. 2019-11-15 15:53:31 -08:00
Vishnu Ks 4b212efba6 tests: Remove create-user-group from curl test exclude_list. 2019-11-15 15:53:31 -08:00