Commit Graph

2380 Commits

Author SHA1 Message Date
Anders Kaseorg 56aadf6503 tests: Fix CI apocalypse perpetrated by previous commit.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-26 21:25:25 -07:00
Steve Howell 697b00dd6e default streams: Change remove api to use stream_id.
This is a full-stack change:

    - server
    - JS code
    - templates

It's all pretty simple--just use stream_id instead
of stream_name.

I am 99% sure we don't document this API nor use it
in mobile, so it should be a safe change.
2020-03-25 17:11:25 -07:00
Graham Bleaney 2fe9d85a5f redirects: Refactor redirect code to use central helper function.
This commit introduces two new functions in 'url_encoding.py' which
centralize two common patterns for constructing redirect URLs. It
also migrates the files using those patterns to use the new
functions.
2020-03-25 16:39:17 -07:00
Graham Bleaney 3e602a9bd4 integrations: Harden development views against path traversal.
This commit hardens the code in the developer integrations views by
ensuring that file paths can only be constructed with valid
integration names.
2020-03-25 16:39:17 -07:00
Tim Abbott d9bb6d0081 compatibility: Add more strict desktop app blocking.
This allows us to block use of the desktop app with insecure versions
(we simply fail to load the Zulip webapp at all, instead rendering an
error page).

For now we block only versions that are known to be both insecure and
not auto-updating, but we can easily adjust these parameters in the
future.
2020-03-24 20:33:11 -07:00
Tim Abbott 85c9ffd91c message: Validate propagate_mode parameters.
This improves the error handling for invalid values of the
propagate_mode parameter to our message editing endpoints.
Previously, invalid values would just work like change_one rather than
doing nothing.
2020-03-24 12:36:45 -07:00
Dinesh 5888d7c0f5 auth: Change how config error URLs are configured.
URLs for config errors were configured seperately for each error
which is better handled by having error name as argument in URL.
A new view `config_error_view` is added containing context for
each error that returns `config_error` page with the relevant
context.
Also fixed tests and some views in `auth.py` to be consistent with
changes.
2020-03-22 17:15:18 -07:00
Steve Howell a041d9e4aa minor: Clean up lstrip() for help article titles.
Saying `foo.lstrip('# ')` does more than just remove
a '# ' prefix.  It removes any combination of '#' and
spaces.

We now make the intention slightly more clear.

We would strip these as you'd expect:

    # foo
    ## foo
    ### foo

but for this we now only strip the first "#":

    # # # # # foo
2020-03-22 11:32:29 -07:00
Steve Howell edf1b1e5e8 minor: Fix buggy lstrip() call in integrations dev panel.
Thanks to @minusworld for catching this--see #14264, which
points out that lstrip() doesn't do what your intuition
might tell you it does.

Now we properly remove the "HTTP_" prefix.

It's not clear to me why we need these prefixes for Django
purposes in the fixtures, but I didn't want to go down
the rabbit hole of fixing those.

To test:

    got to http://YOUR-DEV_SERVER/devtools/integrations/
    select "bitbucket3" for the integration.
    select "diagnostics_ping.json" for the fixture.
    see "X_EVENT_KEY" in "Custom HTTP Headers"

Fixes #14264
2020-03-22 11:32:29 -07:00
Steve Howell d71111f3dc presence api: Use email to look up presence.
We don't want to use delivery_email to look up
presence on email-restricted realms.
2020-03-19 14:32:18 -07:00
Mateusz Mandera f5e95c4fc1 requirements: Bump python-social-auth version.
We had a bunch of ugly hacks to monkey patch things due to upstream
being temporarily unmaintained and not merging PRs. Now the project is
active again and the fixes have been merged and included in the latest
version - so we clean up all that code.
2020-03-18 12:14:31 -07:00
Tim Abbott b064559652 zephyr: Add strict assertion about username format.
This ensures that even if it were possible to create an MIT Kerberos
account with a malicious username and/or hack webathena to pretend
that's the case, one couldn't do anything malicious.

This security improvement only impacts a single installation of Zulip
where Zephyr mirroring is in use that has already had the fix applied,
so there's no reason to do a security notice for it.

Found by Graham Bleaney using pysa.
2020-03-17 05:37:25 -07:00
Mateusz Mandera 2000608a9e report_error: Fix inaccurate docstring.
do_report_error isn't actually below.
2020-03-09 13:54:58 -07:00
Mateusz Mandera 0255ca9b6a middleware: Log user.id/realm.string_id instead of _email. 2020-03-09 13:54:58 -07:00
akashaviator 9c63976da5 api: Refactor get_members_backend in zerver/views/users.py.
This refactors get_members_backend to return user data of a single
user in the form of a dictionary (earlier being a list with a single
dictionary).

This also refactors it to return the data with an appropriate key
(inside a dictionary), "user" or "members", according to the type of
data being returned.

Tweaked by tabbott to use somewhat less opaque code and simple OpenAPI
descriptions.
2020-03-08 18:43:30 -07:00
Tim Abbott 2c75b39078 templates: Delete show_debug feature.
As far as I know, this hasn't been used in at least 5 years, and I'm
not sure there's a real use case for it with the current app.
2020-03-08 18:34:59 -07:00
Tim Abbott 53cc00c21c messages: Ban the sender property when not mirroring.
The "sender" property in `send_message_backend` is meant to only do
something when doing Zephyr mirroring (or similar).  We should help
clients behave correctly by banning this property in requests that are
not specifically requesting mirroring behavior.

This commit requires changes to a number of tests that incorrectly
passed this parameter or didn't use the right setup for mirroring.
2020-03-08 14:09:32 -07:00
Tim Abbott 9230213bde settings: Add EMAIL_ADDRESS_VISIBILITY_NOBODY.
This extends our email address visibility settings to deny access to
user email addresses even to organization administrators.

At the moment, they can of course change the setting (which leaves an
audit trail), but in the future only organization owners will be able
to change that setting.

While we're at this, we rewrite the settings_data.js test to cover all
the cases in a more consistent way.

Fixes #14111.
2020-03-06 16:34:08 -08:00
Steve Howell f2b8eef21a refactor: Avoid hacky use of ValidationError.code.
We were using `code` to pass around messages.

The `code` field is designed to be a code, not
a human-readable message.

It's possible that we don't actually need two
flavors of messages for these type of validations,
but I didn't want to change that yet.

We **definitely** don't need to put two types of
message in the exception, so I fix that.  Instead,
I just have the caller ask what level of detail
it needs.

I added a non-verbose message for the case of
system bots.

I removed the non-translated version of the message
for deactivated accounts, which didn't have test
coverage and is slightly more prone to leaking
email info that we don't want to leak.
2020-03-06 11:53:22 -08:00
Steve Howell 62fb3ad801 refactor: Move validate_email_not_already_in_realm.
We move this to email_validation.py.
2020-03-06 11:53:22 -08:00
Steve Howell ad85e286de user settings: Inline call to validate_email.
We are trying to elminate the version of
`validate_email` that lives in `actions.py`.

Inlining it barely increases the code size, and
it removes some noise related the three-item
tuple that `check_incoming_email` returns.
2020-03-06 11:53:22 -08:00
Steve Howell 4f5b07a7e6 refactor: Extract zerver/lib/email_validation.py. 2020-03-06 11:53:22 -08:00
Steve Howell 57f1aa722c refactor: Rename validate_email_for_realm.
Now called:

    validate_email_not_already_in_realm

We have a separate validation function that
makes sure that the email fits into a realm's
domain scheme, and we want to avoid naming
confusion here.
2020-03-06 11:53:22 -08:00
Mateusz Mandera 6e4dcc714e auth: Fix bug in backend configuration checks in start_social_login.
We shouldn't check if *every* backend in the backends list is
configured, but only the one the user is attempting to log in with.
2020-03-01 22:30:38 -08:00
Mateusz Mandera 2d55c7e0d5 home: Don't assume user agent header is set for insecure_desktop_app.
The header may not be set - this leads to CI failures on 2.1.x branch,
but in any case is a real bug.
2020-02-28 16:51:23 -08:00
Tim Abbott d79a7a8c35 panels: Show a banner for users with legacy desktop apps.
Users who are using ZulipDesktop or haven't managed to auto-update to
ZulipElectron should be strongly encouraged to upgrade.

We'll likely want to move to something even stricter that blocks
loading the app at all, but this is a good start.
2020-02-28 01:54:46 -08:00
Mateusz Mandera efb3065158 social_auth: Take user to find_account if invalid subdomain is given.
This allows to also clean up some code that's not really useful.
2020-02-27 17:27:55 -08:00
Anders Kaseorg 1cdab5ae61 emoji: Resolve emoji sprite sheets and stylesheets through Webpack.
This gives them cache-compatible URLs, and also avoids some extra
copies of the sprite sheet images.

Comments on the Octopus emoji added by tabbott.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 14:43:46 -08:00
Steve Howell 29dd7d2267 minor: Fix comment in send_message_backend.
The `send_message_backend` function no longer
calls `recipient_for_emails`.
2020-02-24 15:32:29 -08:00
Mateusz Mandera 2343f80d72 auth: Remove redundant get_subdomain(request) call in remote_user_sso.
subdomain = get_subdomain(request) is already called earlier in the
function.
2020-02-24 12:39:48 -08:00
Mateusz Mandera f2903e9c70 auth: Refactor - convert remote_username to email in remote_user_sso.
So far the conversion was in a very random place -
register_remote_user(). All other codepaths that use
login_or_register_remote_user() call it with the user's email address.
Making remote_user_sso convert remote_username to the email address
before calling login_or_register_remote_user makes this usage consistent
across the board.
2020-02-24 12:39:48 -08:00
Mateusz Mandera 98ae2fb940 auth: Remove redundant realm argument to finish_desktop_flow.
finish_desktop_flow is called with the assumption that the request
successfully proved control over the user_profile and generates a
special link to log into the user_profile account. There's no reason to
pass the realm param, as user_profile.realm can be assumed.
2020-02-24 12:39:48 -08:00
Dinesh 1308544a70 auth: Remove `if` blocks to redirect to config error page.
In `auth.py` there are three `if` blocks for different backends
to redirect to config error page with similar code. It is better
handled with common code using `get_attr()` function on
constructed setting names.
2020-02-24 12:19:45 -08:00
vsvipul 020a263a67 auth: Create a new page hop for desktop auth.
Create a new page for desktop auth flow, in which
users can select one from going to the app or
continue the flow in the browser.

Co-authored-by: Mateusz Mandera <mateusz.mandera@protonmail.com>
2020-02-20 11:59:55 -08:00
Vishnu KS e1a7716578 emails: Translate from_name of account security emails. 2020-02-18 17:45:33 -08:00
Mateusz Mandera 6a0b68bc7f models: Delete get_stream_recipient function and its uses.
With recipient being now a Stream field, there's no more use for
this helper function.
2020-02-18 10:49:14 -08:00
rht 41e3db81be dependencies: Upgrade to Django 2.2.10.
Django 2.2.x is the next LTS release after Django 1.11.x; I expect
we'll be on it for a while, as Django 3.x won't have an LTS release
series out for a while.

Because of upstream API changes in Django, this commit includes
several changes beyond requirements and:

* urls: django.urls.resolvers.RegexURLPattern has been replaced by
  django.urls.resolvers.URLPattern; affects OpenAPI code and related
  features which re-parse Django's internals.
  https://code.djangoproject.com/ticket/28593
* test_runner: Change number to suffix. Django changed the name in this
  ticket: https://code.djangoproject.com/ticket/28578
* Delete now-unnecessary SameSite cookie code (it's now the default).
* forms: urlsafe_base64_encode returns string in Django 2.2.
  https://docs.djangoproject.com/en/2.2/ref/utils/#django.utils.http.urlsafe_base64_encode
* upload: Django's File.size property replaces _get_size().
  https://docs.djangoproject.com/en/2.2/_modules/django/core/files/base/
* process_queue: Migrate to new autoreload API.
* test_messages: Add an extra query caused by .refresh_from_db() losing
  the .select_related() on the Realm object.
* session: Sync SessionHostDomainMiddleware with Django 2.2.

There's a lot more we can do to take advantage of the new release;
this is tracked in #11341.

Many changes by Tim Abbott, Umair Waheed, and Mateusz Mandera squashed
are squashed into this commit.

Fixes #10835.
2020-02-13 16:27:26 -08:00
Mateusz Mandera bde495db87 registration: Add support for mobile and desktop flows.
This makes it possible to create a Zulip account from the mobile or
desktop apps and have the end result be that the user is logged in on
their mobile device.

We may need small changes in the desktop and/or mobile apps to support
this.

Closes #10859.
2020-02-12 11:22:16 -08:00
Mateusz Mandera 2475adbf8a messages_for_topic: Use stream.recipient_id for more efficient query. 2020-02-11 17:39:43 -08:00
Dinesh 4304d5f8db auth: Add support for GitLab authentication.
With some tweaks by tabbott to the documentation and comments.

Fixes #13694.
2020-02-11 13:54:17 -08:00
Tim Abbott fcac3a4342 recipients: Rename extract_recipients to extract_private_recipients.
Recent changes mean this function is now only used for private
messages.
2020-02-11 12:28:14 -08:00
Steve Howell 1b6578cafd messages: Fix bug with commas in stream names.
We now validate streams with a separate
function from PM recipients.

It's confusing enough all the ways you can
encode a stream or encode the PM recipients,
but trying to do it all in one function was
hard to reason about and led to at least one
bug.

In particular, there was a bug where streams
with commas in them would get split.  Now
we just don't ever split on commas inside
of `extract_stream_indicator`.

Fixes #13836
2020-02-11 12:20:54 -08:00
akashaviator 1ae5964ab8 api: Add an api endpoint for GET /users/{id}
This adds a new API endpoint for querying basic data on a single other
user in the organization, reusing the existing infrastructure (and
view function!) for getting data on all users in an organization.

Fixes #12277.
2020-02-07 10:36:31 -08:00
Tim Abbott 79e5dd1374 users: Rename get_raw_user_data user parameter to acting_user.
This is for improved clarity as we extend this function to take
multiple user objects.
2020-02-07 10:36:31 -08:00
Steve Howell bf3baa14ac presence: Rename get_status_dict_by_user(). 2020-02-06 17:16:22 -08:00
Steve Howell 36fba1076f presence: Move get_status_dict_by_user. 2020-02-06 17:16:22 -08:00
Steve Howell 703338dfa3 presence: Extract lib/presence.py.
This will make more sense when we pull some
code out of the model.
2020-02-06 17:16:22 -08:00
Steve Howell a5093be867 presence: Rename get_status_list.
The word "status" is vague, and this isn't
actually returning a list, so we now name it
get_presence_response.

I originally was gonna rename this to
get_presence_dict, but there's a function
called get_status_dict that returns a subset
of the response, so I think it's a bit more
clear that this is the bigger dict that
actually gets sent back.
2020-02-06 17:16:22 -08:00
Steve Howell 8a1fb2dcd6 presence: Calculate server_timestamp slightly earlier.
We want to err on the side of server_timestamp being
old, since we may eventually use this to make responses
just include incremental changes, and we don't want a
time window (however small) when we miss presence rows.
The clients will be able to deal with duplicate data
to the extent that the time windows are overlapping.

Also, extracting the other local var here
(for `presences`) will set up a subsequent commit
where we re-format the data for clients with
slim_presence=True.
2020-02-06 17:16:22 -08:00
Tim Abbott b5cc1a3afe home: Finish removing assumption that user_profile isn't None. 2020-02-05 16:16:33 -08:00