Commit Graph

1828 Commits

Author SHA1 Message Date
Aditya Bansal 8c0a5c69f3 csp: Dynamically generate and add nonce to script tags. 2018-04-24 06:13:21 +05:30
Tim Abbott 1703e23980 templates: Move all core app templates into a subdirectory.
This should make it easier to find the templates that are actually
part of the core webapp, instead of having them all mixed together
with the portico pages.
2018-04-23 16:46:37 -07:00
Vishnu Ks c9e932a7ce settings: Add support for Hangouts as the video chat provider.
The only thing that's annoying about this feature is that you need to
be a paying G Suite customer to use it.
2018-04-23 09:39:47 -07:00
Tim Abbott c88163eea8 auth: Make "Continue to registration" actually register you.
The main change here is to send a proper confirmation link to the
frontend in the `confirm_continue_registration` code path even if the
user didn't request signup, so that we don't need to re-authenticate
the user's control over their email address in that flow.

This also lets us delete some now-unnecessary code: The
`invalid_email` case is now handled by HomepageForm.is_valid(), which
has nice error handling, so we no longer need logic in the context
computation or template for `confirm_continue_registration` for the
corner case where the user somehow has an invalid email address
authenticated.

We split one GitHub auth backend test to now cover both corner cases
(invalid email for realm, and valid email for realm), and rewrite the
Google auth test for this code path as well.

Fixes #5895.
2018-04-22 19:55:05 -07:00
Tim Abbott 2dcec3704c auth: Introduce confirmation_link local variable.
This is just a prepartory refactor in maybe_send_to_registration.
2018-04-22 16:24:43 -07:00
Tim Abbott 8026b4f9db auth: Refactor login_or_register_remote_user interface.
By moving all of the logic related to the is_signup flag into
maybe_send_to_registration, we make the login_or_register_remote_user
function quite clean and readable.

The next step is to make maybe_send_to_registration less of a
disaster.
2018-04-22 16:24:43 -07:00
Tim Abbott 64023fc563 auth: Fix incorrect use of get_realm_from_request.
The code in maybe_send_to_registration incorrectly used the
`get_realm_from_request` function to fetch the subdomain.  This usage
was incorrect in a way that should have been irrelevant, because that
function only differs if there's a logged-in user, and in this code
path, a user is never logged in (it's the code path for logged-out
users trying to sign up).

This this bug could confuse unit tests that might run with a logged-in
client session.  This made it possible for several of our GitHub auth
tests to have a totally invalid subdomain value (the root domain).

Fixing that bug in the tests, in turn, let us delete a code path in
the GitHub auth backend logic in `backends.py` that is impossible in
production, and had just been left around for these broken tests.
2018-04-22 16:24:43 -07:00
Tim Abbott b36298efda mypy: Fix a missing type annotation in auth code. 2018-04-22 16:24:43 -07:00
Tim Abbott 00c9f45821 auth: Remove dead invalid_subdomain code path.
This code path has actually been dead for a while (since
`invalid_subdomain` gets set to True only when `user_profile` is
`None`).  We might want to re-introduce it later, but for now, we
eliminate it and the artificial test that provided it with test
coverage.
2018-04-22 14:44:06 -07:00
Tim Abbott 65025e8327 auth: Add return_data for RemoteUserBackend.
This is done mainly because this backend has the simplest code path
for calling login_or_register_remote_user, more than because we expect
this case to come up.  It'll make it easier to write unit tests for
the `invalid_subdomain` corner case.
2018-04-22 14:44:06 -07:00
Tim Abbott f806526551 auth: Remove unused request.verified_email variable.
This has never been used since it was introduced in the very first
version of our Google auth code in April 2013.
2018-04-22 14:44:06 -07:00
Shubham Dhama 26d2ffa821 populate_db: Add new user of "guest" type.
The purpose of this user is to act as a guest.
(This is a preliminary step in adding the guest type of user
and is a part of #8385.)
2018-04-20 16:20:00 -07:00
Umair Khan 0420b89468 profile: Add hint attribute to custom profile fields.
This is the model for #8876.
2018-04-19 11:32:35 -07:00
Aditya Bansal 6c1a50da76 csp_reports: Add endpoint to handle logging of reports sent by clients. 2018-04-11 23:01:13 +05:30
Vishwesh Jainkuniya 383c62fb03 dev_login: Identify each user's realm when listing them.
This is a mobile-specific endpoint used for logging into a dev server.
On mobile without this realm_uri it's impossible to send a login request
to the corresponding realm on the dev server and proceed further; we can
only guess, which doesn't work for using multiple realms.

Also rename the endpoint to reflect the additional data.

Testing Plan:
Sent a request to the endpoint, and inspected the result.

[greg: renamed function to match, squashed renames with data change,
 and adjusted commit message.]
2018-04-10 17:03:36 -07:00
Vishnu Ks e92838a31f registration: Catch email validation error and show error message. 2018-04-06 15:18:32 -07:00
Tim Abbott ad861c5fae messages: Improve comment on need_messages. 2018-04-06 08:57:46 -07:00
Tim Abbott b0b134cb4c help: Clean up settings_html and subscriptions_html.
After some thinking, I don't think there's any actual value to doing
the ../ style relative links here, whereas there is actual harm from
the links being slightly broken in the current model.  We fix this by
just using /#settings as the URL.

Fixes #8978.
2018-04-05 14:48:26 -07:00
Steve Howell 36844418e9 bug fix: Respect include_history for certain queries.
For certain queries where both include_history and
use_first_unread_anchor are set to True, we were excluding
historical rows.  Now we only use the use_first_unread_anchor
flag to filter rows that we use to find the anchor, without
having it filter the actual search results.

The bug went unreported for a long time, because it only
affected mobile users who had newly subscribed to streams.

Note that we make a small change to the test called
test_use_first_unread_anchor_with_muted_topics, which has
a very scary comment about being "arcane" and "be
absolutely sure you know what you're doing."  I think it's
fine.

Also, the new test code would fail before this fix, so it
should help prevent future regressions.

Fixes #8958
2018-04-05 17:16:41 -04:00
Steve Howell b64117d872 refactor: Build query in find_first_unread_anchor().
This is a bit more than a pure refactor, because we duplicate a
chunk of code to calculate a query inside of
find_first_unread_anchor(), so we're doing a bit more work
than before.

We need this refactoring to start decoupling find_first_unread_anchor
from get_messages_backend for the case where include_history is
True.  This will happen in a subsequent commit.

The only test that changes here is a direct test on
find_first_unread_anchor().  All other tests pass without
modification, and we have decent coverage on get_messages_backend.
2018-04-05 17:16:41 -04:00
Steve Howell 345d44b5f1 Extract get_base_query_for_search(). 2018-04-05 17:16:41 -04:00
Steve Howell 59a9b69c25 Simplify search code for keyword searches.
We use an array now to build up the list of search operands and
then consolidate the special search handling after the loop (which
means setting the flag, putting two more columns in the query, and
using ' '.join to build the string).
2018-04-05 13:27:31 -07:00
Steve Howell d521906fb6 search refactor: Extract add_narrow_conditions().
This code was basically pulled from two near-to-each-other
sections of get_messages_backend, and it does an early
return for narrow-is-None.
2018-04-05 13:27:31 -07:00
Steve Howell 3ac660d972 minor: Show `narrow` for UnicodeDecodeError.
We have a debugging statement for some obscure errors we get
when narrows have search terms.  We now show all the narrow
operators.  This isn't really to improve debugging; it's more
to make it easier in the next commit to extract a function
that would make search_term have to be passed back in a tuple.
But it shouldn't hurt debugging either.
2018-04-05 13:27:31 -07:00
Tim Abbott 98889608a2 help: Fix structure of markdown context logic.
Refactoring in this file had resulted in the logic for
html_settings_link being duplicated and extra logic being needed to
ensure these variables were set where they were needed.

This fixes subscriptions_html not being rendered properly in the /help
and /api pages, in addition to removing duplicate code.
2018-04-05 12:22:41 -07:00
Steve Howell 4cba679d38 Move code for find_first_unread_anchor().
This is a pure refactoring and just pulls the function out
to the top level of the module.  (The prior commit extracted
it inside a larger function to make a nicer diff.)
2018-04-05 09:55:54 -07:00
Steve Howell d8a95c6517 Extract find_first_unread_anchor().
This is a pure refactoring.
2018-04-05 09:55:54 -07:00
Tim Abbott 228f41e916 messages: Pass UserProfile to is_public_stream_by_name and rename.
The new name can_access_stream_history_by_name gets to the point of
what this function actually does.  And passing in a user object lets
us define what this does based on the user subscribed.
2018-04-04 15:13:11 -07:00
Tim Abbott 5e82d750c5 get_messages: Refactor ok_to_include_history to accept a UserProfile.
If we make history accessible to some stream subscribers of private
streams, we'll need the UserProfile to be available here.
2018-04-04 15:06:53 -07:00
Tim Abbott 758d7b9146 bots: Clean up editing bots impacting non-bot users.
This fixes a bug where the endpoint for editing bot users would allow
an organization administrator to edit the full name of a bot user.

A combination of this an another recently fixed bug made it possible
for this process to set a `bot_owner` for a non-bot user; so we also
include a migration to fix that for any users that might have had our
model invariants corrupted in that way.
2018-04-02 18:58:26 -07:00
Tim Abbott 938c4cee08 settings: Add option to control Jitsi video chat integration.
Fixes #8922.
2018-04-02 16:55:16 -07:00
Shubham Dhama b7aae89029 org settings: Add realm level default time format setting. 2018-03-31 16:23:26 -07:00
Tim Abbott 656f882a44 bots: Eliminate NEW_USER_BOT.
This bot was basically a duplicate of NOTIFICATION_BOT for some
specific corner cases, and didn't add much value.  It's better to just
eliminate it, which also removes some ugly corner cases around what
happens if the user account doesn't exist.
2018-03-29 12:01:21 -07:00
Tarun Kumar 86eaa553a6 user-groups: Add backend enforcing for new modification settings.
Add function in user-groups.py for getting member ids
for a group.
Update view to enforce checks for modifying user-groups.
Only admins and user group members can modify user-groups.
2018-03-23 14:44:47 -07:00
Tim Abbott fcd265304c message editing: Fix buggy timestamp check.
This somehow got messed up in a final refactoring before merging
91197fa4f1.
2018-03-22 16:26:18 -07:00
Sarah 91197fa4f1 org settings: Add logic for applying allow_community_topic_editing.
Applies the logic to allow community members to edit topics
of others' messages if this setting is True. Otherwise,
only administrators can update the topic of others' messages.

This logic includes a 24-hour time limit for community topic editing.
2018-03-22 16:13:36 -07:00
Sarah f5c2fb8438 org settings: Create backend api for allow_community_topic_editing.
Adds the code for updating the allow_community_topic_editing
setting.
2018-03-22 16:02:27 -07:00
Jack Weatherilt 8535625341 parse_user_agent: Assert user agent is not None.
This commit asserts that parse_user_agent never returns None. The
RegEx will match any string, so that `match` is never None. This
brings test coverage of lib/user_agent.py to 100%. Changes were also
made in test/test_decorators.py and views/compatibility.py to reflect
that parse_user_agent cannot return None.

Improves: #7089.
Fixes: #8779.
2018-03-22 14:29:29 -07:00
Aditya Bansal 2a2df0ef5e auth: Make redirects to next work for REMOTE_USER based Apache SSO.
It's possible that this won't work with some versions of the
third-party backend, but tabbott has tested carefully that it does
work correctly with the Apache basic auth backend in our test
environment.
2018-03-21 14:01:05 -07:00
Aditya Bansal 1e48dac8f3 auth.py: Make redirects to 'next' url work for google and github.
In this commit we start to support redirects to urls supplied as a
'next' param for the following two backends:
* GoogleOAuth2 based backend.
* GitHubAuthBackend.
2018-03-21 13:35:44 -07:00
Aditya Bansal 9a100b1f55 auth.py: Make redirects to 'next' url work for dev environment.
This makes these redirects work for the local authentication
backend.
2018-03-21 13:35:44 -07:00
Harshit Bansal a49655e0d4 emoji: Migrate realm emoji to be addressed by `id` rather than `name`.
This commit migrates realm emoji to be addressed by their `id` rather
than their name. This fixes a long standing issue which was causing
an error on uploading an emoji with same name as a deactivated realm
emoji.

Fixes: #6977.
2018-03-20 22:24:44 -07:00
Tim Abbott 0e5c954393 users: Fix error string for disposable email addresses.
This should not use the term "realm", and doesn't need to name the
organization, either.
2018-03-16 16:59:01 -07:00
Tim Abbott 149f3efe5a realm: Clean up use of "Realm" in more user-facing strings.
We should be talking about organization names.
2018-03-16 16:59:01 -07:00
Vishnu Ks b13150a438 models: Do the check for disposable email in email_allowed_for_realm. 2018-03-15 14:35:24 -07:00
Vishnu Ks 951b88dd30 models: Make email_allowed_for_realm raise exception. 2018-03-15 14:35:24 -07:00
Steve Howell a4a8527ec5 search: Return info flags in payload.
We now return these:
    anchor
    found_anchor
    found_oldest
    found_newest

Fixes #8639
2018-03-15 12:36:06 -07:00
Steve Howell c6839e07c0 search: Fix num_after/num_before semantics precisely.
We now post-process query results so that you never get
more than `num_after` rows with id < `anchor`, and likewise
for `num_before`.
2018-03-15 12:36:06 -07:00
Steve Howell 6f3ebf6c4c Add post_process_limited_query(). 2018-03-15 12:36:06 -07:00
Steve Howell bd95b37d67 search: Make `num_after`/`num_after` more consistent.
We now consistently set our query limits so that we get at
least `num_after` rows such that id > anchor.  (Obviously, the
caveat is that if there aren't enough rows that fulfill the
query, we'll return the full set of rows, but that may be less
than `num_after`.)  Likewise for `num_before`.

Before this change, we would sometimes return one too few rows
for narrow queries.

Now, we're still a bit broken, but in a more consistent way.  If
we have a query that does not match the anchor row (which could
be true even for a non-narrow query), but which does match lots
of rows after the anchor, we'll return `num_after + 1` rows
on the right hand side, whether or not the query has narrow
parameters.

The off-by-one semantics here have probably been moot all along,
since our windows are approximate to begin with.  If we set
num_after to 100, its just a rough performance optimization to
begin with, so it doesn't matter whether we return 99 or 101 rows,
as long as we set the anchor correctly on the subsequent query.

We will make the results more rigorous in a follow up commit.
2018-03-15 12:36:06 -07:00