Commit Graph

2078 Commits

Author SHA1 Message Date
Yashashvi Dave 47aaf4e20a users: Replace duplication with generic func to validate bot id.
This adds a common function `access_bot_by_id` to access bot id within
same realm.  It probably fixes some corner case bugs where we weren't
checking for deactivated bots when regenerating API keys.
2018-05-29 15:47:27 -07:00
Tim Abbott 91ec0aba09 auth: Improve interactions between LDAPAuthBackend and EmailAuthBackend.
Previously, if you had LDAPAuthBackend enabled, we basically blocked
any other auth backends from working at all, by requiring the user's
login flow include verifying the user's LDAP password.

We still want to enforce that in the case that the account email
matches LDAP_APPEND_DOMAIN, but there's a reasonable corner case:
Having effectively guest users from outside the LDAP domain.

We don't want to allow creating a Zulip-level password for a user
inside the LDAP domain, so we still verify the LDAP password in that
flow, but if the email is allowed to register (due to invite or
whatever) but is outside the LDAP domain for the organization, we
allow it to create an account and set a password.

For the moment, this solution only covers EmailAuthBackend.  It's
likely that just extending the list of other backends we check for in
the new conditional on `email_auth_backend` would be correct, but we
haven't done any testing for those cases, and with auth code paths,
it's better to disallow than allow untested code paths.

Fixes #9422.
2018-05-28 22:47:47 -07:00
Tim Abbott 8119670da1 user_settings: Prevent LDAP users from setting a Zulip password.
Previously, if both EmailAuthBackend and LDAPAuthBackend were enabled,
LDAP users could set a password using EmailAuthBackend and continue to
use that password, even if their LDAP account was later deactivated.

That configuration wasn't supported at all before, so this doesn't fix
a pre-existing security issue, but now that we're making that a valid
configuration, we need to cover this case.
2018-05-28 22:47:47 -07:00
Sampriti Panda 250015a5d5 pgroonga: Fix issues with HTML escaping in queries. 2018-05-28 16:53:30 -07:00
Yashashvi Dave e82c879b85 custom fields: Add user type of custom fields.
Fixes #8878
2018-05-27 23:01:21 -07:00
Tim Abbott 6967b6519d settings: Add a development-only setting for less dense mode.
This should make it easier for us to iterate on a less-dense Zulip.

We create two classes on body, less_dense_mode and more_dense_mode, so
that it's easy as we refactor to separate the two concepts from things
like colors that are independent.
2018-05-24 12:31:37 -07:00
Vishnu Ks 54a002c2e2 requirements: Upgrade pyflakes to 2.0.0.
We fix a few errors that only the new version finds.
2018-05-24 11:31:36 -07:00
Steve Howell 69517f5ac5 Support zform-based widget content in the server.
API users, particularly bots, can now send a field
called "widget_content" that will be turned into
a submessage for the web app to look at.  (Other
clients can still rely on "content" to be there,
although it's up to the bot author to make the
experience good for those clients as well.)

Right now widget_content will be a JSON string that
encodes a "zform" widget with "choices."  Our first
example will be a trivia bot, where users will see
something like this:

    Which fruit is orange in color?

        [A] orange
        [B] blackberry
        [C] strawberry

The letters will be turned into buttons on the webapp
and have canned replies.

This commit has a few parts:
    - receive widget_content in the request (simply
        validating that it's a string)
    - parse the JSON in check_message and deeply
        validate its structure
    - turn it into a submessage in widget.py
2018-05-24 09:30:22 -07:00
Shubham Dhama fc3dcbfb5a messages: Allow "no topic" topics editable indefinitely.
Fixes: #9484.
2018-05-24 07:35:52 -07:00
Umair Khan adaeaccd20 2FA: Add link in settings to setup two-factor. 2018-05-23 15:49:41 -07:00
Umair Khan 0b1e25f453 2FA: Integrate with login feature. 2018-05-23 15:46:57 -07:00
Umair Khan a2d3aea027 2FA: Add two-factor related code.
This commit adds a view which will be used to process login requests,
adds an AuthenticationTokenForm so that we can use TextField widget for
tokens, and activates two factor authentication code path whenever user
tries to login.
2018-05-23 15:46:56 -07:00
Vishnu Ks 8369e2b15e registration: Allow users to import settings from other realm.
This should significantly improve the user experience for creating
additional accounts on zulipchat.com.

Currently, disabled in production pending some work on visual styling.
2018-05-23 10:30:00 -07:00
Michael J. Sullivan d5ee801d60 mypy: Set local_partial_types = True.
The daemon implicitly sets this flag, so set it explicitly and fix all
of the type errors.
2018-05-21 22:41:00 -07:00
Aditya Bansal e41d32a15a reminders: Add various design improvements for the UI/reminder msg. 2018-05-21 09:07:37 -07:00
Umair Khan 18defcc673 2FA: Fix check for context_data attr.
This attribute only makes sense for responses that are generated using a
template.
2018-05-21 07:50:05 -07:00
Tim Abbott 2609274719 streams: Don't allow adding deactivated users to streams.
This query was incorreclty not checking whether a user was deactivated
before managing their subscriptions.

This isn't an important bug, but should prevent some weird corner
cases (like trying to send a notification PM to a deactivated user,
which fails).
2018-05-20 19:17:15 -07:00
Tim Abbott cedad52c59 presence: Extract and use get_active_user helper.
This adds a new reusable function for fetching just active users.
2018-05-20 19:07:29 -07:00
Tim Abbott 44b3aeb08d home: Improve logging for narrow parsing exceptions.
This now includes the request and (and thus the related metadata).
2018-05-20 18:47:43 -07:00
Yashashvi Dave 38db31779b /json/bots: Return "No user" if access other realms bot.
Instead of returning warning "Insufficient Permission", return
"No such bot" warning if user tries to access other realms bot."
2018-05-18 15:20:43 -07:00
Yashashvi Dave 4162e61f33 /json/users: Replace email with user_id in API to reactivate user. 2018-05-18 15:20:43 -07:00
Yashashvi Dave 06e7e933cc /json/users: Replace email with user_id in API to update/remove users. 2018-05-18 15:20:43 -07:00
Steve Howell de47eeb6f1 Add /submessage endpoint. 2018-05-16 15:13:33 -07:00
Yashashvi Dave b949d10592 /json/bots: Replace email with user_id in API to generate bot_api_key.
Fixes #3643.
2018-05-15 16:37:06 -07:00
Yashashvi Dave d6e2f9fc88 /json/bots: Replace email with user_id in API to update bots. 2018-05-15 16:34:17 -07:00
Shubham Padia 897ed17f0c api: Allow realm_admins to make a stream announcement_only. 2018-05-13 09:11:51 -07:00
Vishnu Ks 99186952f6 integrations: Make integration_doc endpoint work only on ajax. 2018-05-13 08:56:04 -07:00
Raymond Akornor f67efd5291 zerver: Migrate away from six. 2018-05-09 18:38:25 +00:00
Tim Abbott 602b13db34 home: Make emojiset an explicit parameter.
This is cleaner than accessing the UserProfile object directly (we
don't really do that anywhere else).
2018-05-06 22:36:29 -07:00
Tim Abbott 956bd74905 attachments: Send events for attachment updates.
We send add events on upload, update events when sending a message
referencing it, and delete updates on removal.

This should make it possible to do real-time sync for the attachments
UI.

Based in part on work by Aastha Gupta.
2018-05-04 16:22:27 -07:00
Tim Abbott 508dc5b6ed decorators: Add new decorators for guest users.
These decorators will be part of the process for disabling access to
various features for guest users.

Adding this decorator to the subscribe endpoint breaks the guest users
test we'd just added for the subscribe code path; we address this by
adding a more base-level test on filter_stream_authorization.
2018-05-04 10:25:52 -07:00
Aditya Bansal c38b70566c views/archive.py: Make 'StreamDoesNotExist' return a error msg page. 2018-05-02 15:23:33 -07:00
Aditya Bansal 5d7907b59f public_archives: Add basic infra for displaying topics.
We add very basic infra so that we can view any discussion which
happened under a topic of a global public stream without
authorization.
2018-05-02 15:23:33 -07:00
Umair Khan b778259547 login-page: Return early if resp is a redirect.
Redirect response does not need a context.
2018-05-02 15:15:37 -07:00
Umair Khan aef2234e97 login_page: Create update_login_page_context().
This will also be used from two factor login.
2018-05-02 14:30:02 -07:00
Tim Abbott a0e8a37e7f topics: Fix get_topics_backend logic for zephyr realms.
This removes a check on invite_only, that should have been a check on
history_public_to_subscribers.  In addition to fixing a bug for zephyr
realms, it also makes "more topics" work correctly for realms using
the new settings for stream history being public to subscribers.
2018-05-02 09:02:57 -07:00
Tim Abbott d92edb8ea5 css: Remove support for legacy desktop app.
We haven't seen significant traffic from the legacy desktop app in
over a year, and users using it get a warning to upgrade since last
summer, so it's probably OK to stop providing special fonts for it.
2018-05-01 09:49:28 -07:00
Angelika Serwa f4f64243dd custom_profile_fields: Support changing the sort order of the fields.
Tweaked by tabbott for variable naming and the URL.

Closes #8879.
2018-04-30 18:17:41 -07:00
Vishnu Ks 69b0783b35 emails: Pass realm_name instead of realm in notify_change_in_email context. 2018-05-01 00:11:44 +05:30
Yashashvi Dave a97a00a4c6 custom fields: Replace field id with field name in error message. 2018-04-30 10:53:23 -07:00
Yashashvi Dave 7e9ccead2e stream settings: Fix server error on long stream description.
Add backend validations to check stream description length.
2018-04-30 10:11:23 -07:00
Eeshan Garg 057ff9c91e models: Add Stream.history_public_to_subscribers.
This commit adds a new field history_public_to_subscribers to the
Stream model, which serves a similar function to the old
settings.PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS; we still use that
setting as the default value for new streams to avoid breaking
backwards-compatibility for those users before we are ready with an
actual UI for users to choose directly.

This also comes with a migration to set the value of the new field for
existing streams with an algorithm matching that used at runtime.

With significant changes by Tim Abbott.

This is an initial part of our efforts on #9232.
2018-04-28 22:54:04 -07:00
Tim Abbott e087be6630 home: Ignore handlebars errors in the test suite.
The handlebars error message is just for the manual development
environment; this prevents the state of compiling handlebars templates
from run-dev.py from potentially causing the unit tests to fail.
2018-04-28 13:49:24 -07:00
Tim Abbott 7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
Tim Abbott a3fc7d1371 message_edit: Fix spelling of "has passed". 2018-04-27 19:24:14 -07:00
YJDave 6bef44a9fa org setting: Add time limits for message deleting.
Add realm setting to set time limit for message deleitng.
Set default value of message_content_delete_limit_seconds
to 600 seconds(10 min).

Thanks to Shubham Dhama for rebasing and reworking this.  Some final
edits also done by Tim Abbott.

Fixes #7344.
2018-04-27 19:22:28 -07:00
Vishnu Ks 185811f436 emails: Add translation tags to find_team. 2018-04-27 11:59:36 -07:00
Umair Khan cf2f6b38dd profile: Add choice field.
Fixes part of #8878
2018-04-26 00:35:51 -07:00
Umair Khan a1b384039c profile: Check field_type against FIELD_TYPE_CHOICES. 2018-04-25 23:28:27 -07:00
Tim Abbott 79e8bff8fa views: Change use of typing.Text to str.
This is the first part of a general migration of our typing codebase
to use the simpler `str` for strings.
2018-04-23 18:51:23 -07:00
Aditya Bansal 8c0a5c69f3 csp: Dynamically generate and add nonce to script tags. 2018-04-24 06:13:21 +05:30
Tim Abbott 1703e23980 templates: Move all core app templates into a subdirectory.
This should make it easier to find the templates that are actually
part of the core webapp, instead of having them all mixed together
with the portico pages.
2018-04-23 16:46:37 -07:00
Vishnu Ks c9e932a7ce settings: Add support for Hangouts as the video chat provider.
The only thing that's annoying about this feature is that you need to
be a paying G Suite customer to use it.
2018-04-23 09:39:47 -07:00
Tim Abbott c88163eea8 auth: Make "Continue to registration" actually register you.
The main change here is to send a proper confirmation link to the
frontend in the `confirm_continue_registration` code path even if the
user didn't request signup, so that we don't need to re-authenticate
the user's control over their email address in that flow.

This also lets us delete some now-unnecessary code: The
`invalid_email` case is now handled by HomepageForm.is_valid(), which
has nice error handling, so we no longer need logic in the context
computation or template for `confirm_continue_registration` for the
corner case where the user somehow has an invalid email address
authenticated.

We split one GitHub auth backend test to now cover both corner cases
(invalid email for realm, and valid email for realm), and rewrite the
Google auth test for this code path as well.

Fixes #5895.
2018-04-22 19:55:05 -07:00
Tim Abbott 2dcec3704c auth: Introduce confirmation_link local variable.
This is just a prepartory refactor in maybe_send_to_registration.
2018-04-22 16:24:43 -07:00
Tim Abbott 8026b4f9db auth: Refactor login_or_register_remote_user interface.
By moving all of the logic related to the is_signup flag into
maybe_send_to_registration, we make the login_or_register_remote_user
function quite clean and readable.

The next step is to make maybe_send_to_registration less of a
disaster.
2018-04-22 16:24:43 -07:00
Tim Abbott 64023fc563 auth: Fix incorrect use of get_realm_from_request.
The code in maybe_send_to_registration incorrectly used the
`get_realm_from_request` function to fetch the subdomain.  This usage
was incorrect in a way that should have been irrelevant, because that
function only differs if there's a logged-in user, and in this code
path, a user is never logged in (it's the code path for logged-out
users trying to sign up).

This this bug could confuse unit tests that might run with a logged-in
client session.  This made it possible for several of our GitHub auth
tests to have a totally invalid subdomain value (the root domain).

Fixing that bug in the tests, in turn, let us delete a code path in
the GitHub auth backend logic in `backends.py` that is impossible in
production, and had just been left around for these broken tests.
2018-04-22 16:24:43 -07:00
Tim Abbott b36298efda mypy: Fix a missing type annotation in auth code. 2018-04-22 16:24:43 -07:00
Tim Abbott 00c9f45821 auth: Remove dead invalid_subdomain code path.
This code path has actually been dead for a while (since
`invalid_subdomain` gets set to True only when `user_profile` is
`None`).  We might want to re-introduce it later, but for now, we
eliminate it and the artificial test that provided it with test
coverage.
2018-04-22 14:44:06 -07:00
Tim Abbott 65025e8327 auth: Add return_data for RemoteUserBackend.
This is done mainly because this backend has the simplest code path
for calling login_or_register_remote_user, more than because we expect
this case to come up.  It'll make it easier to write unit tests for
the `invalid_subdomain` corner case.
2018-04-22 14:44:06 -07:00
Tim Abbott f806526551 auth: Remove unused request.verified_email variable.
This has never been used since it was introduced in the very first
version of our Google auth code in April 2013.
2018-04-22 14:44:06 -07:00
Shubham Dhama 26d2ffa821 populate_db: Add new user of "guest" type.
The purpose of this user is to act as a guest.
(This is a preliminary step in adding the guest type of user
and is a part of #8385.)
2018-04-20 16:20:00 -07:00
Umair Khan 0420b89468 profile: Add hint attribute to custom profile fields.
This is the model for #8876.
2018-04-19 11:32:35 -07:00
Aditya Bansal 6c1a50da76 csp_reports: Add endpoint to handle logging of reports sent by clients. 2018-04-11 23:01:13 +05:30
Vishwesh Jainkuniya 383c62fb03 dev_login: Identify each user's realm when listing them.
This is a mobile-specific endpoint used for logging into a dev server.
On mobile without this realm_uri it's impossible to send a login request
to the corresponding realm on the dev server and proceed further; we can
only guess, which doesn't work for using multiple realms.

Also rename the endpoint to reflect the additional data.

Testing Plan:
Sent a request to the endpoint, and inspected the result.

[greg: renamed function to match, squashed renames with data change,
 and adjusted commit message.]
2018-04-10 17:03:36 -07:00
Vishnu Ks e92838a31f registration: Catch email validation error and show error message. 2018-04-06 15:18:32 -07:00
Tim Abbott ad861c5fae messages: Improve comment on need_messages. 2018-04-06 08:57:46 -07:00
Tim Abbott b0b134cb4c help: Clean up settings_html and subscriptions_html.
After some thinking, I don't think there's any actual value to doing
the ../ style relative links here, whereas there is actual harm from
the links being slightly broken in the current model.  We fix this by
just using /#settings as the URL.

Fixes #8978.
2018-04-05 14:48:26 -07:00
Steve Howell 36844418e9 bug fix: Respect include_history for certain queries.
For certain queries where both include_history and
use_first_unread_anchor are set to True, we were excluding
historical rows.  Now we only use the use_first_unread_anchor
flag to filter rows that we use to find the anchor, without
having it filter the actual search results.

The bug went unreported for a long time, because it only
affected mobile users who had newly subscribed to streams.

Note that we make a small change to the test called
test_use_first_unread_anchor_with_muted_topics, which has
a very scary comment about being "arcane" and "be
absolutely sure you know what you're doing."  I think it's
fine.

Also, the new test code would fail before this fix, so it
should help prevent future regressions.

Fixes #8958
2018-04-05 17:16:41 -04:00
Steve Howell b64117d872 refactor: Build query in find_first_unread_anchor().
This is a bit more than a pure refactor, because we duplicate a
chunk of code to calculate a query inside of
find_first_unread_anchor(), so we're doing a bit more work
than before.

We need this refactoring to start decoupling find_first_unread_anchor
from get_messages_backend for the case where include_history is
True.  This will happen in a subsequent commit.

The only test that changes here is a direct test on
find_first_unread_anchor().  All other tests pass without
modification, and we have decent coverage on get_messages_backend.
2018-04-05 17:16:41 -04:00
Steve Howell 345d44b5f1 Extract get_base_query_for_search(). 2018-04-05 17:16:41 -04:00
Steve Howell 59a9b69c25 Simplify search code for keyword searches.
We use an array now to build up the list of search operands and
then consolidate the special search handling after the loop (which
means setting the flag, putting two more columns in the query, and
using ' '.join to build the string).
2018-04-05 13:27:31 -07:00
Steve Howell d521906fb6 search refactor: Extract add_narrow_conditions().
This code was basically pulled from two near-to-each-other
sections of get_messages_backend, and it does an early
return for narrow-is-None.
2018-04-05 13:27:31 -07:00
Steve Howell 3ac660d972 minor: Show `narrow` for UnicodeDecodeError.
We have a debugging statement for some obscure errors we get
when narrows have search terms.  We now show all the narrow
operators.  This isn't really to improve debugging; it's more
to make it easier in the next commit to extract a function
that would make search_term have to be passed back in a tuple.
But it shouldn't hurt debugging either.
2018-04-05 13:27:31 -07:00
Tim Abbott 98889608a2 help: Fix structure of markdown context logic.
Refactoring in this file had resulted in the logic for
html_settings_link being duplicated and extra logic being needed to
ensure these variables were set where they were needed.

This fixes subscriptions_html not being rendered properly in the /help
and /api pages, in addition to removing duplicate code.
2018-04-05 12:22:41 -07:00
Steve Howell 4cba679d38 Move code for find_first_unread_anchor().
This is a pure refactoring and just pulls the function out
to the top level of the module.  (The prior commit extracted
it inside a larger function to make a nicer diff.)
2018-04-05 09:55:54 -07:00
Steve Howell d8a95c6517 Extract find_first_unread_anchor().
This is a pure refactoring.
2018-04-05 09:55:54 -07:00
Tim Abbott 228f41e916 messages: Pass UserProfile to is_public_stream_by_name and rename.
The new name can_access_stream_history_by_name gets to the point of
what this function actually does.  And passing in a user object lets
us define what this does based on the user subscribed.
2018-04-04 15:13:11 -07:00
Tim Abbott 5e82d750c5 get_messages: Refactor ok_to_include_history to accept a UserProfile.
If we make history accessible to some stream subscribers of private
streams, we'll need the UserProfile to be available here.
2018-04-04 15:06:53 -07:00
Tim Abbott 758d7b9146 bots: Clean up editing bots impacting non-bot users.
This fixes a bug where the endpoint for editing bot users would allow
an organization administrator to edit the full name of a bot user.

A combination of this an another recently fixed bug made it possible
for this process to set a `bot_owner` for a non-bot user; so we also
include a migration to fix that for any users that might have had our
model invariants corrupted in that way.
2018-04-02 18:58:26 -07:00
Tim Abbott 938c4cee08 settings: Add option to control Jitsi video chat integration.
Fixes #8922.
2018-04-02 16:55:16 -07:00
Shubham Dhama b7aae89029 org settings: Add realm level default time format setting. 2018-03-31 16:23:26 -07:00
Tim Abbott 656f882a44 bots: Eliminate NEW_USER_BOT.
This bot was basically a duplicate of NOTIFICATION_BOT for some
specific corner cases, and didn't add much value.  It's better to just
eliminate it, which also removes some ugly corner cases around what
happens if the user account doesn't exist.
2018-03-29 12:01:21 -07:00
Tarun Kumar 86eaa553a6 user-groups: Add backend enforcing for new modification settings.
Add function in user-groups.py for getting member ids
for a group.
Update view to enforce checks for modifying user-groups.
Only admins and user group members can modify user-groups.
2018-03-23 14:44:47 -07:00
Tim Abbott fcd265304c message editing: Fix buggy timestamp check.
This somehow got messed up in a final refactoring before merging
91197fa4f1.
2018-03-22 16:26:18 -07:00
Sarah 91197fa4f1 org settings: Add logic for applying allow_community_topic_editing.
Applies the logic to allow community members to edit topics
of others' messages if this setting is True. Otherwise,
only administrators can update the topic of others' messages.

This logic includes a 24-hour time limit for community topic editing.
2018-03-22 16:13:36 -07:00
Sarah f5c2fb8438 org settings: Create backend api for allow_community_topic_editing.
Adds the code for updating the allow_community_topic_editing
setting.
2018-03-22 16:02:27 -07:00
Jack Weatherilt 8535625341 parse_user_agent: Assert user agent is not None.
This commit asserts that parse_user_agent never returns None. The
RegEx will match any string, so that `match` is never None. This
brings test coverage of lib/user_agent.py to 100%. Changes were also
made in test/test_decorators.py and views/compatibility.py to reflect
that parse_user_agent cannot return None.

Improves: #7089.
Fixes: #8779.
2018-03-22 14:29:29 -07:00
Aditya Bansal 2a2df0ef5e auth: Make redirects to next work for REMOTE_USER based Apache SSO.
It's possible that this won't work with some versions of the
third-party backend, but tabbott has tested carefully that it does
work correctly with the Apache basic auth backend in our test
environment.
2018-03-21 14:01:05 -07:00
Aditya Bansal 1e48dac8f3 auth.py: Make redirects to 'next' url work for google and github.
In this commit we start to support redirects to urls supplied as a
'next' param for the following two backends:
* GoogleOAuth2 based backend.
* GitHubAuthBackend.
2018-03-21 13:35:44 -07:00
Aditya Bansal 9a100b1f55 auth.py: Make redirects to 'next' url work for dev environment.
This makes these redirects work for the local authentication
backend.
2018-03-21 13:35:44 -07:00
Harshit Bansal a49655e0d4 emoji: Migrate realm emoji to be addressed by `id` rather than `name`.
This commit migrates realm emoji to be addressed by their `id` rather
than their name. This fixes a long standing issue which was causing
an error on uploading an emoji with same name as a deactivated realm
emoji.

Fixes: #6977.
2018-03-20 22:24:44 -07:00
Tim Abbott 0e5c954393 users: Fix error string for disposable email addresses.
This should not use the term "realm", and doesn't need to name the
organization, either.
2018-03-16 16:59:01 -07:00
Tim Abbott 149f3efe5a realm: Clean up use of "Realm" in more user-facing strings.
We should be talking about organization names.
2018-03-16 16:59:01 -07:00
Vishnu Ks b13150a438 models: Do the check for disposable email in email_allowed_for_realm. 2018-03-15 14:35:24 -07:00
Vishnu Ks 951b88dd30 models: Make email_allowed_for_realm raise exception. 2018-03-15 14:35:24 -07:00
Steve Howell a4a8527ec5 search: Return info flags in payload.
We now return these:
    anchor
    found_anchor
    found_oldest
    found_newest

Fixes #8639
2018-03-15 12:36:06 -07:00
Steve Howell c6839e07c0 search: Fix num_after/num_before semantics precisely.
We now post-process query results so that you never get
more than `num_after` rows with id < `anchor`, and likewise
for `num_before`.
2018-03-15 12:36:06 -07:00
Steve Howell 6f3ebf6c4c Add post_process_limited_query(). 2018-03-15 12:36:06 -07:00
Steve Howell bd95b37d67 search: Make `num_after`/`num_after` more consistent.
We now consistently set our query limits so that we get at
least `num_after` rows such that id > anchor.  (Obviously, the
caveat is that if there aren't enough rows that fulfill the
query, we'll return the full set of rows, but that may be less
than `num_after`.)  Likewise for `num_before`.

Before this change, we would sometimes return one too few rows
for narrow queries.

Now, we're still a bit broken, but in a more consistent way.  If
we have a query that does not match the anchor row (which could
be true even for a non-narrow query), but which does match lots
of rows after the anchor, we'll return `num_after + 1` rows
on the right hand side, whether or not the query has narrow
parameters.

The off-by-one semantics here have probably been moot all along,
since our windows are approximate to begin with.  If we set
num_after to 100, its just a rough performance optimization to
begin with, so it doesn't matter whether we return 99 or 101 rows,
as long as we set the anchor correctly on the subsequent query.

We will make the results more rigorous in a follow up commit.
2018-03-15 12:36:06 -07:00
Steve Howell ec5299b1f2 search refactor: Rename query_result to rows.
The name `query_result` seems to suggest something more complex.
2018-03-15 12:36:06 -07:00
Aditya Bansal d4360e2287 uploads: Make django-sendfile to force downloading attachments.
We start to force downloads for the attachment files. We do this
for all files except images or pdf's. We would like images or pdf's
to open up in browser itself.

Tweaked by tabbott for comment clarity and correctness.
2018-03-14 11:22:10 -07:00
YJDave 93ee0aace7 stream settings: Allow realm admins to remove others from any stream.
This will allow realm admins to remove others from private stream to
which the realm administrator is not subscribed; this is important for
managing those streams, because previously nobody could remove users
from private streams that didn't have any realm administrators
subscribed.
2018-03-13 14:59:09 -07:00
YJDave 2031118545 stream settings: Allow realm admins to access all private stream subs.
This will allow realm admins to access subscribers of unsubscribed
private stream.  This is a preparatory commit for letting realm admins
remove those users.
2018-03-13 14:59:09 -07:00
YJDave 37f9d5c193 stream settings: Allow realm admins to update any stream name & description.
This will allow realm admins to update the names and descriptions of
private streams even if they are not subscribed, which fixes the buggy
behavior that previously nobody could(!).
2018-03-13 14:59:09 -07:00
Steve Howell 1ff653c986 search refactor: Extract `limit_query_to_range`.
This generic function isolates the before/after logic that really
is independent of Message and doesn't need to clutter up
`get_messages_backend`.  Also, introducing a new namespace
reduces some shadowing/mutation with variables like `query`.

It's a pure code move, with some very minor renaming (e.g.
inner_msg_id_col -> id_col).
2018-03-13 13:51:22 -07:00
Steve Howell 74a4a69333 search refactor: Move code that increments num_after/num_before.
We move the code that increments num_after/num_before to be closer
to where we do all the before/after logic.
2018-03-13 13:51:22 -07:00
Steve Howell 63c21707ee search refactor: Tighten up before/after logic.
If anchor is 0, there is no sense doing a before_query.

Likewise, if anchor is `LARGER_THAN_MAX_MESSAGE_ID`, there is
no sense doing an after_query.

We introduce variables called `need_before_query` and
`need_after_query` to enforce those conditions.

This also adds some comments explaining the fallthrough case
where neither query makes sense.
2018-03-13 13:51:22 -07:00
Steve Howell 129faa2c21 search refactor: Avoid `message_id >= 0` in queries.
We don't need things like `AND message_id >= 0` in our queries.

We can short circuit the syntax when `anchor` or
`first_message_visible` are zero.
2018-03-13 13:51:22 -07:00
Steve Howell e232a0bd57 search refactor: Streamline queries for caught-up users.
If use_first_unread_anchor is set and we don't have any unread
messages, then our anchor is effectively "positive infinity" and
we can streamline queries.

In the past we'd have clauses like `message_id <= 999999999999999`
in the query that were harmless but crufty.
2018-03-13 13:51:22 -07:00
Steve Howell e77350dc8e search refactor: Replace confusing `!= 0` idiom in search code.
We want to say `if num_after > 0` when we expect num_after to be
a positive integer.  We don't want any confusion that we will
execute the blocks for values of -7 or None.
2018-03-13 13:51:22 -07:00
neiljp (Neil Pilgrim) 9e1dbde82d mypy: Final small migrations to python3.5 annotations in many files. 2018-03-12 11:23:30 -07:00
neiljp (Neil Pilgrim) 6b9671c95c mypy: Migrate views/storage.py to python3.5 type annotations. 2018-03-12 11:23:30 -07:00
neiljp (Neil Pilgrim) 6bda59f3d7 mypy: Almost migrate views/streams.py to python3.5 annotations.
One remaining issue with FuncKwargPair use.
2018-03-12 11:23:30 -07:00
Vishnu Ks a44255eedb emails: Add backend for disallowing disposable email addresses. 2018-03-11 22:05:58 -07:00
Tim Abbott ef92fcbe2b topic history: Fix fetching topic history of public streams.
Apparently, we did essentially all the work to support showing full
topic history to newly subscribed users from a data flow perspective,
but didn't actually enable this feature by having the topic history
endpoint grant access to historical topics.  This fixes that gap.

I'm not altogether happy with how the code and tests read for this
feature; the code itself has more duplication than I'd like, and the
tests do too, but it works.
2018-03-11 20:59:20 -07:00
YJDave c94b21e9ac settings: Add setting to disable message content in missed message emails.
Fixes #6938.
2018-03-09 21:16:02 -08:00
Umair Khan 54e56481e6 auth: Retain email value if login fails.
Fixes #7795
2018-03-09 14:51:24 -08:00
Shubham Dhama 777b6de689 org settings: Add setting to prevent users from adding bots.
Fixes: #7908.
2018-03-09 13:21:55 -08:00
Robert Hönig 649e76e932 Display error when creating embedded bot with incorrect config data.
"incorrect" here means rejected by a bot's validate_config() method.
A common scenario for this is validating API keys before the bot is
created. If validate_config() fails, the bot will not be created.
2018-03-08 15:05:42 -08:00
Tim Abbott c3964dff6e i18n: Fix a last few strings mentioning realms. 2018-03-07 17:15:29 -08:00
Tim Abbott c47403b024 i18n: Fix message-send error messages using 'realm'.
We also do some small quality improvements to these strings, while
we're at it.
2018-03-07 17:15:29 -08:00
Tim Abbott e2a6541133 i18n: Fix use of 'realm administrator' in translated strings.
These are user-facing and thus should refer to being an "organization
administrator".
2018-03-07 17:15:29 -08:00
Tim Abbott 342d8cd4e0 i18n: Fix use of realm to refer to an organization. 2018-03-07 17:15:29 -08:00
Marco Burstein bdb86f1b5e emoji: Add support for translating emoticons.
Add `translate_emoticons` to `prop_types` and `expected_keys`.
Furthermore, create a emoji-translating Markdown inline pattern.

Also use a JavaScript version of `translate_emoticons` and then use
this function during Markdown previews and as a preprocessor. This
is only needed for previews, because usually emoticon translation
happens on the backend after sending.

Add tests for emoticon translation, a settings UI, and a /help/ page
as well.

Tweaked by tabbott to fix various test failurse as well as how this
handles whitespace, requiring emoticons to not have adjacent
characters.

Fixes #1768.
2018-03-04 15:37:24 -08:00
YJDave ce46cd914a settings: Allow admin to change email/name even if it is disabled in realm.
Allow realm admin users to change their email or name even,
changing name or email is disabled in realm.
2018-03-02 12:17:00 -08:00
Vishnu Ks f0d651f9fe invites: Only admins should be able to create multiuse invites. 2018-03-02 11:59:32 -08:00
Vishnu Ks 56e54262c3 api: Create api for creating multiuse invites. 2018-03-02 11:28:36 +00:00
rht 71ff8c370e django-2.0: Don't assign directly to Many-to-Many field.
The old pattern of setting the value and then using .save() here has
been deprecated.  set() also saves the record.
2018-03-01 08:49:35 -08:00
Robert Hönig 48b2e4eb66 backend: Add bot config data patching. 2018-03-01 08:25:43 -08:00
Aastha Gupta d124597f4b org-settings: Add setting to turn off welcome emails.
This adds an organization-level setting to provide an option to turn
off the welcome emails.

Fixes #8000.
2018-02-28 12:39:01 -08:00
Tim Abbott 710f5f7c97 auth: Add support for mobile_flow_otp for RemoteUserBackend.
Because we have a pretty good framework for the existing
mobile_flow_otp system, this requires very little new code.

Fixes #8291.
2018-02-24 08:14:17 -08:00
Tim Abbott 34efab9157 auth: Report to mobile apps the availability of RemoteUserBackend.
This is necessary for mobile apps to do the right thing when only
RemoteUserBackend is enabled, namely, directly redirect to the
third-party SSO auth site as soon as the user enters the server URL
(no need to display a login form, since it'll be useless).
2018-02-24 08:14:17 -08:00
Eeshan Garg df4c645587 integrations: Make sure settings.EXTERNAL_URI_SCHEME renders.
This got broken at some point when we moved around the context
processing logic for integrations/webhooks. Thankfully, the
context value for external_uri_scheme was only used in a couple of
our less popular integration docs. It should render perfectly now.
2018-02-24 06:21:02 -05:00
Umair Khan d22639717c auth: Redirect to an error page instead of 500.
Previously, we used to raise an exception if the direct dev login code
path was attempted when:

* we were running under production environment.
* dev. login was not enabled.

Now we redirect to an error page and give an explanatory message to the
user.

Fixes #8249.
2018-02-20 22:34:53 -08:00
Vishnu Ks d34dd4cd02 signup: Show subdomain availability during signup.
This uses an actual query to the backend to check if the subdomain is
available, using the same logic we would use to check when the
subdomain is in fact created.
2018-02-19 10:45:17 -08:00
neiljp (Neil Pilgrim) 3e98e77638 mypy: Amend typing of storage parameter to update_storage to be non-Optional. 2018-02-19 09:24:50 -08:00
neiljp (Neil Pilgrim) 2176f4789b mypy: Rewrite conditional to clarify message-sending defer_until code.
This actually fixes a small bug.
2018-02-19 09:24:28 -08:00
Shubham Dhama 9feae472f8 org settings: Add button to deactivate organization.
This adds button under "Organization profile" settings, which
deactivates the organization and sends an "event" to all the
active user and log out them.

Fixes: #8212.
2018-02-18 10:20:38 -08:00
Shubham Padia a4b686297a api: Return anchor in get_messages when use_first_unread_anchor=True.
This may be helpful for some API clients, since it avoids them needed
to do somewhat messy post-processing on the results (the data was
always available via scanning for the first unread message in the result).

Fixes #6244.
2018-02-16 10:06:20 -08:00
Aditya Bansal efe8545303 local-uploads: Start running authentication checks on file requests.
From here on we start to authenticate uploaded file request before
serving this files in production. This involves allowing NGINX to
pass on these file requests to Django for authentication and then
serve these files by making use on internal redirect requests having
x-accel-redirect field. The redirection on requests and loading
of x-accel-redirect param is handled by django-sendfile.

NOTE: This commit starts to authenticate these requests for Zulip
servers running platforms either Ubuntu Xenial (16.04) or above.

Fixes: #320 and #291 partially.
2018-02-16 05:06:37 +05:30
neiljp (Neil Pilgrim) b8f81801c1 mypy: Annotate existing_user_profile in accounts_register(). 2018-02-13 11:40:52 -08:00
neiljp (Neil Pilgrim) 16b939a964 REQ: Ensure only initial parameter is positional & adjust views/messages.py. 2018-02-13 11:40:52 -08:00
Shubham Dhama 03e4026c62 bots: Check bot owner for deactivated users and bots.
We should not allow deactivated users and bots as a
bot owner.
2018-02-13 09:24:42 -08:00
Shubham Dhama adfc905c3f bots: Handle exception on changing bot owner to invalid user.
It catches the `UserProfile.DoesNotExist` exception and
hence prevent internal server error.
Also remove option to select empty bot owner.
Fixes: #8334.
2018-02-13 09:23:13 -08:00
Greg Price ecbc72b857 push notifs: Add a diagnostic in API of whether push notifs enabled.
When the answer is False, this will allow the mobile app to show a
warning that push notifications will not work and the server admin
should set them up.

Based partly on Kunal's PR #7810.  Provides the necessary backend API
for zulip/zulip-mobile#1507.
2018-02-12 14:34:59 -08:00
Robert Hönig c0a0e6fd70 Set add_bot_backend() config_data param default to {}.
Previously, the default was None. This caused 500s when
adding a bot without adding a config_data argument.
2018-02-09 12:30:24 -08:00
Robert Hönig 146b833454 marketing: Update integration count.
This also automizes the integrations count update for
the /integrations page.
2018-02-06 08:50:41 -08:00
Roman Godov b875fe07eb settings: Added setting to turn on and off realm name in email subject.
Users having only account in one realm will not be distracted by realm
name in subject lines of every email.  Users who have multiple
accounts in realms can turn this setting on and receive a
corresponding realm name in email's subject.

Tweaked by tabbott to rebase and address a few small issues.

Fixes #5489.
2018-02-05 18:01:54 -08:00
Brock Whittaker f485abf522 page_params: Pass through settings.WARN_NO_EMAIL, as `warn_no_email`.
[greg: fixed test]
2018-02-05 16:00:33 -08:00
Greg Price 48791b731e create_realm: Let server admin skip confirming email.
This will let us defer configuring outbound email to the end of the
install procedure, so we can greatly simplify it by consolidating
several scripted steps.

The new flow could be simplified further by giving the user the full
form in the first place, rather than first a form for just their
email address and then a form with the other details.  We'll leave
that improvement for a separate change.
2018-02-05 12:59:12 -08:00
Greg Price 0dceeebd05 create_realm: Refactor to deal ASAP with key record, not string.
Now, there's just one spot at the beginning of the function where we
inspect the string key the user gave us; and after that point, we not
only have validated that string but in fact are working from our own
record that it pointed to, not the string itself.

This simplifies the code a bit, e.g. by not repeatedly searching the
database for the key (and hoping everything agrees so that we keep
getting the same row), and it will simplify adding logic to inspect
row attributes like `presume_email_valid`.
2018-02-05 12:59:12 -08:00
Greg Price f766ac6c10 create_realm: Reject invalid key even if no key required.
There's no use case for presenting a key that's invalid; if we haven't
given the user a valid key, we needn't send them to a URL that
presents an invalid one.  And the code is simpler to think about if
the only keys that can exist (after the validation at the top of the
function) are valid ones.

Apart from the case where creation_key is None, but invalid, and
settings.OPEN_REALM_CREATION is True so that we'd previously let the
invalid key slide, this is a pure refactor.
2018-02-05 12:59:12 -08:00
rht fae8d23039 zerver/views: Remove u prefix from strings. 2018-02-05 12:11:33 -08:00
rht 9a8d2244ca django-2.0: Shift to resolvers from urlresolvers.
The old name is deprecated.
2018-01-30 10:53:54 -08:00
Vishnu Ks 43a6439b3b upload: Enforce per-realm quota. 2018-01-29 16:06:11 -08:00
Greg Price 55cf54c087 upload: Remove old per-user quota feature.
We'll replace this primarily with per-realm quotas (plus the simple
per-file limit of settings.MAX_FILE_UPLOAD_SIZE, 25 MiB by default).

We do want per-user quotas too, but they'll need some more management
apparatus around them so an admin has a practical way to set them
differently for different users.  And the error handling in this
existing code is rather confused.  Just clear this feature out
entirely for now; then we'll build the per-realm version more cleanly,
and then we can later add back per-realm quotas modelled after that.

The migration to actually remove the field is in a subsequent commit.

Based in part on work by Vishnu Ks (hackerkid).
2018-01-29 16:06:11 -08:00
Greg Price 0f99610478 signup: Small refactor to isolate actually sending mail.
This is a little cleaner in that the try/except blocks for
SMTPException are a lot narrower; and it'll facilitate an upcoming
change to sometimes skip sending mail.
2018-01-29 10:27:11 -08:00
Greg Price 4bdecb7150 create_realm: Small refactor to check key just once. 2018-01-24 14:34:30 -08:00
YJDave 15dd94035c org settings: Handle NEW_USER_BOT not being set.
If NEW_USER_BOT is not configured in organization, then disable setting
for selecting stream for new user notification.
2018-01-23 15:23:25 -05:00
Brock Whittaker 849d63dd1c settings: Change error for "no data" to something more friendly. 2018-01-23 14:38:59 -05:00
Brock Whittaker ffe3a6c127 settings: Remove password confirmation in modal.
This removes the requirement to confirm your new password.
It isn't necessary and can be fixed easily with an email
reset if messed up.
2018-01-23 14:38:59 -05:00
Robert Hönig 9a15c4e3ff Add bot services to page_params.
This is the first step for allowing users
to edit a bot's service entries, name the
outgoing webhook configuration entries. The
chosen data structures allow for a future
with multiple services per bot; right now,
only one service per bot is supported.
2018-01-23 07:29:00 -05:00
Robert Hönig 06fb868482 Send an event to bot owners when a bot is created. 2018-01-23 07:29:00 -05:00
Eeshan Garg 00d3f19c0a integrations/email: Render settings.EMAIL_GATEWAY_EXAMPLE correctly. 2018-01-22 18:05:20 -05:00
Aditya Bansal f272ea9087 scheduledmessages: Start using/expecting delivery_type as a param. 2018-01-19 11:33:11 -05:00
Aditya Bansal c770bdaa3a reminder_bot: Add infra for adding reminder bot to every realm. 2018-01-19 11:33:11 -05:00
Aditya Bansal f46d098558 schedulemessages: Add handle_deferred_message() to handle requests.
This is responsible for:
1.) Handling all the incoming requests at the
messages endpoint which have defer param set. This is similar to
send_message_backend apart from the fact that instead of really
sending a message it schedules one to be sent later on.
2.) Does some preliminary checks such as validating timestamp for
scheduling a message, prevent scheduling a message in past, ensure
correct format of message to be scheduled.
3.) Extracts time of scheduled delivery from message.
4.) Add tests for the newly introduced function.
5.) timezone: Add get_timezone() to obtain tz object from string.
This helps in obtaining a timezone (tz) object from a timezone
specified as a string. This string needs to be a pytz lib defined
timezone string which we use to specify local timezones of the
users.
2018-01-19 11:33:11 -05:00
Robert Hönig d1d8365a6b embedded bots: Add config data UI.
This adds UI fields in the bot settings for specifying
configuration values like API keys for a bot. The names
and placeholder values for each bot's config fields are
fetched from the bot's <bot>.conf template file in the
zulip_bots package. This also adds giphy and followup
as embedded bots.
2018-01-09 11:50:54 -05:00
Alena Volkova 9d1063d362 urls: Move the json/fetch_api_key endpoint to be an API-style route. 2018-01-08 13:15:52 -05:00
Eeshan Garg 479bc41ca5 api docs: Remove old and outdated endpoint docs. 2018-01-04 10:17:29 -05:00
Alena Volkova 45f0c76c44 settings: Limit the creation of generic bots.
This commit adds a setting to limit creation of generic bots
to admins for realms that want that restriction.  (Generic
bots, apart from being considered spammy on some realms,
have less locked down permissions than webhook bots).

Fixes #7066.
2018-01-02 18:12:22 -05:00
Aastha Gupta daf86eb664 settings: Add "text" option to emoji_set model.
We no longer have a special UI setting and model
field ("emoji_alt_code") for saying users want text-only
emojis.  We now instead make "text" be a fifth choice
for "emojiset".

Fixes #7406
2018-01-02 14:55:01 -05:00
Vishnu Ks 277d463d23 messages: Add option to limit users message history. 2018-01-02 13:45:01 -05:00
YJDave fa44d2ea69 settings: Remove autoscroll_forever setting.
Fixes #6845
2018-01-02 10:35:49 -05:00
rht 1324d9f000 mypy: Use Python 3 type syntax in zerver/views. 2017-12-30 07:34:05 -05:00
YJDave 29905ad5d6 create stream: Before sending notificaion, check if NOTIFICATION_BOT exists.
There might be case that NOTIFICATION_BOT is none, so before sending stream
announce notification, check first if settings.NOTIFICATION_BOT is not none.
2017-12-29 08:59:55 -05:00
Kiy4h 6fa51be822 mypy: Use Python 3 type syntax in zerver/views/push_notifications.py. 2017-12-28 07:26:44 -05:00
Kiy4h 64413e3b91 mypy: Use Python 3 type syntax in zerver/views/email_mirror.py. 2017-12-28 07:26:44 -05:00
Kiy4h 238cd2b132 mypy: Use Python 3 type syntax in zerver/views/alert_words.py. 2017-12-28 07:26:44 -05:00
Xavier Cooney 36b58ff6fc mypy: Use Python 3 type syntax in 'views/users.py'. 2017-12-26 08:34:38 -05:00
nyan-salmon 0fd0332a2c mypy: Use Python 3 type syntax in zerver/views/custom_profile_fields.py. 2017-12-26 08:34:06 -05:00
Viraat Chandra 0494902e6d mypy: Use Python 3 syntax for typing in `zerver/views/auth.py`. 2017-12-26 08:31:43 -05:00
Xavier Cooney 61704dc20b mypy: Use Python 3 type syntax in views/pointer.py. 2017-12-26 08:29:30 -05:00
Callum Fraser b0c5eb5482 mypy: Use Python 3 type syntax in zerver/views/typing.py. 2017-12-18 11:32:52 -05:00
Greg Price 17a6632c43 zerver/views: Revert to Python 2 typing syntax for now (storage, streams).
This reverts commit 620b2cd6e.

Contributors setting up a new development environment were getting
errors like this:
```
++ dirname tools/do-destroy-rebuild-database
[...]
+ ./manage.py purge_queue --all
Traceback (most recent call last):
[...]
  File "/home/zulipdev/zulip/zproject/legacy_urls.py", line 3, in <module>
    import zerver.views.streams
  File "/home/zulipdev/zulip/zerver/views/streams.py", line 187, in <module>
    method_kwarg_pairs: List[FuncKwargPair]) -> HttpResponse:
  File "/usr/lib/python3.5/typing.py", line 1025, in __getitem__
    tvars = _type_vars(params)
[...]
  File "/usr/lib/python3.5/typing.py", line 277, in _get_type_vars
    for t in types:
TypeError: 'ellipsis' object is not iterable
```

The issue appears to be that we're using the `typing` module from the
3.5 stdlib, rather than the `typing=3.6.2` in our requirements files,
and that doesn't understand the `Callable[..., HttpResponse]` that
appears in the definition of `FuncKwargPair`.

Revert for now to get provision working again; at least one person
reports that reverting this sufficed.  We'll need to do more testing
before putting this change back in.
2017-12-13 10:20:11 -08:00
rht 620b2cd6e2 zerver/views: Use Python 3 syntax for typing (storage, streams). 2017-12-12 17:40:50 -08:00
Rhea Parekh 8ed7f73199 IRC mirror: No longer require IRC stream names to start with #. 2017-12-12 10:04:31 -06:00
Xavier Cooney 7887857c81 mypy: Use Python 3 type syntax in user_settings.py 2017-12-11 20:33:00 -08:00
picapi_ 2adeaab6ab mypy: Use Python 3 type syntax in zerver/views/invite.py. 2017-12-11 20:30:19 -08:00
Shreyansh Dwivedi 47fcb27e39 invitations: Remove custom_body.
Fixes #7672
2017-12-11 19:23:54 -08:00
Callum Fraser 00178d114b mypy: Use Python 3 type syntax in zerver/views/streams.py.
[greg: small formatting tweaks]
2017-12-11 18:43:24 -08:00
Rishi Gupta d9ce33dcf0 registration: Clarify realm_creation conditionals in accounts_register.
No change in behavior.
2017-12-11 18:36:14 -08:00
Rishi Gupta e545a681fc registration: Use standard pattern for returning link_does_not_exist.
Eventually this check for the realm will be done in get_object_from_key
itself. Rewriting this to fit the pattern in get_object_from_key.

No change to behavior.
2017-12-11 18:36:14 -08:00
Rishi Gupta dbe3576706 registration: Enforce realm is None only if realm_creation.
Commit d4ee3023 and its parent have the history behind this code.

Since d4ee3023^, all new PreregistrationUser objects, except those for
realm creation, have a non-None `realm`.  Since d4ee3023, any legacy
PreregistrationUsers, with a `realm` of None despite not being for
realm creation, are treated as expired.  Now, we ignore them
completely, and remove any that exist from the database.

The user-visible effect is to change the error message for
registration (or invitation) links created before d4ee3023^ to be
"link does not exist", rather than "link expired".

This change will at most affect users upgrading straight from 1.7 or
earlier to 1.8 (rather than from 1.7.1), but I think that's not much
of a concern (such installations are probably long-running
installations, without many live registration or invitation links).

[greg: tweaked commit message]
2017-12-11 18:36:14 -08:00
Arseny Chekanov e5c05f128a mypy: Use Python 3 type syntax in zerver/views/realm_filters.py. 2017-12-09 16:50:12 -08:00
Xavier Cooney aa37b50d1e mypy: User Python 3 syntax for typing in realm.py. 2017-12-08 18:34:26 -08:00
Xavier Cooney 63e793a8d1 mypy: Use Python 3 syntax for typing in views/tutorial.py. 2017-12-08 18:34:26 -08:00
Rishi Gupta 100167fcf8 invitations: Change error messages for resend/revoke views.
We don't have a concept of invitation ID on the frontend (or backend).
2017-12-06 20:35:50 -08:00
Rishi Gupta cb5eb47470 invitations: Move resend/revoke error handling out of actions.py. 2017-12-06 20:35:50 -08:00
picapi_ e2c14724d9 mypy: Use Python 3 type syntax in zerver/views/messages.py. 2017-12-03 13:48:18 -08:00
Alena Volkova 3be8e95d13 settings: Send bot types from the backend instead of hardcoding them. 2017-11-30 15:28:53 -08:00
Rishi Gupta 7c9694077e confirmation: Move check_prereg_key_and_redirect to registration.py. 2017-11-29 22:18:05 -08:00
Rishi Gupta 331a9bee6b confirmation: Add Confirmation.REALM_CREATION.
Does not change any behavior. Groundwork for adding a realm field to
Confirmation.
2017-11-29 22:03:09 -08:00
Sarah ba70b542d3 notification settings: Add stream email notifications setting.
Adds a field to UserProfile for enabling emails about stream
messages and a field to Subscriptions about email
notification preferences.
2017-11-28 17:51:18 -08:00
Greg Price 0ec2a9d259 auth: Try switching to register even if user exists on another realm.
For example, this means that if a user already has an account on one
realm and they try to make an account on another by hitting "Sign in
with Google" (rather than following the little "Register" link to a
"Sign up with Google" button instead), they'll get to make an account
instead of getting an error.

Until very recently, if the user existed on another realm, any attempt
to register with that email address had to fail in the end, so this
logic gave the user a useful error message early.  We introduced it in
c23aaa178 "GitHub: Show error on login page for wrong subdomain"
back in 2016-10 for that purpose.  No longer!  We now support reusing
an email on multiple realms, so we let the user proceed instead.

This function's interface is kind of confusing, but I believe when its
callers use it properly, `invalid_subdomain` should only ever be true
when `user_profile` is None -- in which case the revised
`invalid_subdomain` condition in this commit can never actually fire,
and the `invalid_subdomain` parameter no longer has any effect.  (At
least some unit tests call this function improperly in that respect.)
I've kept this commit to a minimal change, but it would be a good
followup to go through the call sites, verify that, eliminate the use
of `invalid_subdomain`, then remove it from the function entirely.
2017-11-28 16:38:41 -08:00
Tim Abbott 586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
Tim Abbott e6f460f511 auth: Replace user_email_is_unique validator.
As we migrate to allow reuse of the same email with multiple realms,
we need to replace the old "no email reuse" validators.  Because
stealing the email for a system bot would be problematic, we still ban
doing so.

This commit only affects the realm creation logic, not registering an
account in an existing realm.
2017-11-28 16:23:10 -08:00
Greg Price 5f38b374b6 register: Activate existing mirror dummy only if on same realm.
The one thing this bit of logic is used for is to decide whether
there's an existing user which is a mirror dummy that we should
activate.  This change causes us to ignore such an existing user if
it's on some other realm, and go straight into `do_create_user`.
2017-11-28 16:23:10 -08:00
Tim Abbott 4f5a5a8547 auth: Extract add_dev_login_context.
This simple refactor cleans up the code for the dev_auth_enabled code
path to be a bit less deeply in the login_page() logic.
2017-11-28 15:17:33 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
rht e538f4dd44 zerver/views: Use Python 3 syntax for typing.
Edited by tabbott to remove state.py and streams.py, because of
problems with the original PR's changes, and wrap some long lines.
2017-11-27 17:10:39 -08:00
Tim Abbott bd8196c3a5 lint: Fix too-long line in auth.py. 2017-11-27 15:28:46 -08:00
Vishnu Ks d4ee3023b2 registration: Require an explicit realm on PreregistrationUser.
This completes the last commit's work to fix CVE-2017-0910, applying
to any invite links already created before the fix was deployed.  With
this change, all new-user registrations must match an explicit realm
in the PreregistrationUser row, except when creating a new realm.

[greg: rewrote commit message]
2017-11-27 14:59:34 -08:00
Vishnu Ks 985768b2fd registration: Check realm against PreregistrationUser realm.
We would allow a user with a valid invitation for one realm to use it
on a different realm instead.  On a server with multiple realms, an
authorized user of one realm could use this (by sending invites to
other email addresses they control) to create accounts on other
realms. (CVE-2017-0910)

With this commit, when sending an invitation, we record the inviting
user's realm on the PreregistrationUser row; and when registering a
user, we check that the PregistrationUser realm matches the realm the
user is trying to register on.  This resolves CVE-2017-0910 for
newly-sent invitations; the next commit completes the fix.

[greg: rewrote commit message]
2017-11-27 14:58:26 -08:00
Tim Abbott 2210f627a5 signup: Switch active mirror-dummy users to an AssertionError.
Previously, this was a ValidationError, but that doesn't really make
sense, since this condition reflects an actual bug in the code.

Because this happened to be our only test coverage the ValidationError
catch on line 84 of registration.py, we add nocoverage there for now.
2017-11-26 16:13:45 -08:00
Vishnu Ks 9f225360dc register: Make /register redirect to find accounts in root domain without realm.
If there is no realm in the root domain don't show
registration form. Instead just redirect to find
accounts page.
2017-11-26 15:32:17 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
rht e169bb0954 zerver: Remove simplejson dependency.
Modified by tabbott to put the third-party code in a new file.

Fixes #6970.
2017-11-21 21:51:48 -08:00
derAnfaenger e526d0c144 embedded bots: Add views to access state. 2017-11-21 21:10:39 -08:00
Eeshan Garg 73d0f1db81 lib/integrations: Remove Integration.doc_context.
An Integration object doesn't need access to the context dict used
to render its doc.md, since the context dict is just passed directly to
render_markdown_path.
2017-11-21 20:39:30 -08:00
Eeshan Garg 600ee90a38 integrations: Add add_context_for_single_integration function.
Previously, when rendering a single integration, we tacked on the
following information to the context dict that was redundant:

* An OrderedDict containing all of the Integration objects for
  all integrations.
* An OrderedDict containing all of the integration categories.

The context dict for rendering a particular integration doc would
contain 4 OrderedDicts, 2 for categories, 2 for Integration objects
because of how many times add_integrations_context had been called.

This was very wasteful, since an Integration object doesn't need
to access any other Integration object (or itself for that matter)
to render its documentation. This commit adds a function that
allows us to only pass in the context values that are necessary.
2017-11-21 20:39:30 -08:00
Tim Abbott 3bfb19b5f3 Convert EmailAuthBackend and LDAPAuthBackend to accept a realm. 2017-11-21 18:23:50 -08:00
Tim Abbott 1b95b098dd auth: Clarify comments explaining the GoogleMobileOauth2Backend. 2017-11-21 18:23:50 -08:00
Tim Abbott a7d51127fb auth: Convert GoogleMobileOAuth2Backend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 07bc31f818 auth: Convert DevAuthBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 4968631d1b auth: Convert DevAuthBackend to use a unique argument pattern.
This helps ensure that we won't accidentally activate this backend on
other code paths.
2017-11-21 18:23:49 -08:00
Tim Abbott 387c9109ec auth: Convert RemoteUserBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott f17974ab32 DummyAuthBackend: Require being passed a realm object.
We should now always know the realm in our auth code paths.
2017-11-21 18:22:37 -08:00
Vishnu Ks 34689370cd settings: Create UI and backend for setting signups stream. 2017-11-21 17:39:51 -08:00
Rishi Gupta 42652713c8 onboarding: Refactor away setup_initial_private_stream.
It is a confusing name, now that the stream creation part has been taken
away.
2017-11-21 17:39:50 -08:00
Tim Abbott 29408d24ff emoji: Rework reactions validation to have a single function.
This feels more readable to me, and also identified a couple cases
where we were missing test coverage.
2017-11-20 15:27:21 -08:00
Tim Abbott 2d4c7b771a reactions: Move legacy endpoint code further down in file.
This helps ensure that new contributors working on the reactions code
read the stuff we use now first.
2017-11-20 15:27:21 -08:00
Tim Abbott 80a2a36dd1 reactions: Rename legacy reactions for greater clarity. 2017-11-20 15:27:21 -08:00
Tim Abbott 25ddba99f6 lint: Ban general use of user_profile.save().
This often can cause minor caching problems.

Obviously, it'd be better if we had access to the AST and thus could
do this rule for UserProfile objects in general.
2017-11-20 10:57:08 -08:00
Vishnu Ks 432c9d3449 email_log: Reset the value of email after email change process.
This commit actually needs only user_profile.save() line
but it's more readable with both the lines.
2017-11-20 10:45:16 -08:00
Tim Abbott c8edbae21c password reset: Fix error message for invalid realm.
This is a lot cleaner than the previous model.

Basically rewritten by Vishnu Ks to actually work :).
2017-11-20 10:34:55 -08:00
Eeshan Garg 9138d3b8d7 integrations: Move integration-specific context to integration_doc.
Instead of populating the context dict with integration-specific
information in render_markdown_path, we now do that in
zerver.views.integrations.integration_doc instead.

Fixes #7401.

Tweaked by tabbott to use cast to handle the typing issues here.
2017-11-19 20:48:46 -08:00
Tim Abbott 18369cb5ad JWT: Fetch the subdomain earlier in the auth code path. 2017-11-17 17:33:28 -08:00
Harshit Bansal d9c2f613e3 api: Add new endpoint for reactions.
This endpoint will allow us to add/delete emoji reactions whose emoji
got renamed during various emoji infra changes. This was also a
required change for realm emoji migration.

This commit was tweaked significantly by tabbott for greater clarity
(with no changes to the actual logic).
2017-11-16 20:52:15 -08:00
Sampriti Panda a696141a25 search: Add more extensive logging for Unicode errors in highlight_string.
Tweaked by tabbott to provide finer-grained logging.
2017-11-16 15:08:16 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Umair Khan cc76f7deac user-groups: Create check_delete_user_group. 2017-11-15 22:35:43 -08:00
Umair Khan 66179ea295 bulk_add_members_to_user_group: Pass user group.
Previously we passed user group id. If we pass user group, send event is
easier.
2017-11-15 22:35:23 -08:00
Tim Abbott c7a975e4df users: Move check_change_full_name to actions.py.
This avoids an import loop in the next commit, and better matches our
usual code structure.
2017-11-15 17:39:09 -08:00
Tim Abbott 03b4a0240c user_settings: Fix mypy annotations.
While we're at it, convert this to use the Python 3 style, which isn't
prone to this sort of bug.
2017-11-15 17:38:19 -08:00
Brock Whittaker f9f0f356be settings: Add backend storage and interface for night mode.
This allows the night mode setting to be stored in the backend.
2017-11-15 16:36:06 -08:00
Tim Abbott 85844107fd integrations: Remove code for legacy Hubot lozenges.
We no longer display the only model anyway.
2017-11-15 14:04:19 -08:00
rht bafba95eb7 Text-wrap the remaining long lines exceeding 110. 2017-11-15 10:58:03 -08:00
Tim Abbott 5aa6751e7a lint: Fix long lines in newly added features. 2017-11-15 10:58:03 -08:00