Commit Graph

1643 Commits

Author SHA1 Message Date
Vishnu Ks 74351fb250 send_email: Add function to generate tokenized noreply in FromAddress. 2018-06-23 12:03:30 -07:00
Shubham Padia 6853530eba search: Add search_pill.js with basic operations.
Adds search_pill.js to the static asset pipeline. The items
for search pill contain 2 keys, display_value and search_string.
Adding all the operator information i.e the operator, operand and
negated fields along with the search_string and description was tried out.
It was dropped because it didn't provide any advantage as one had to
always calculate the search_string and the description from the operator.
2018-06-22 09:05:36 -04:00
Aditya Bansal d8e0c959bc cache: Enable compression in pylibmc for cache stored in memcached.
This fixes an important issue where the realm_users cache could grow
beyond 1MB when a Zulip server had more than about 10K users.  The
result was that Zulip would start 500ing with that size of userbase.

There are probably better long-term fixes, but because the realm_users
data set caches well, this change should be sufficient to let us
handle to 50-100K users or more on that metric (though at some point,
we'll start having other problems interacting with the realm_users
data set).
2018-06-16 05:24:42 -07:00
Aditya Bansal 998fb038a9 django-pylibmc: Add dependancy and use as cache backend for memcached. 2018-06-16 05:24:19 -07:00
Yashashvi Dave f7f039e772 custom fields: Add endpoint to delete value of custom profile data.
Add delete endpoint at `users/me/profile_data` to delete value of
custom profile data.

Fixes #9669
2018-06-07 08:35:04 -07:00
Steve Howell 8ce8e93645 refactor: Extract settings_panel_menu.js. 2018-06-06 09:42:33 -07:00
Tim Abbott 5ce69b3ecb backends: Fix unnecessary duplicate query to realm in social auth.
This is just a small cleanup to the social auth backend code.
2018-06-06 00:31:59 -07:00
Tim Abbott 35c4a9f1d2 auth: Rewrite our social auth integration to use pipeline.
This new implementation model is a lot cleaner and should extend
better to the non-oauth backend supported by python-social-auth (since
we're not relying on monkey-patching `do_auth` in the OAuth backend
base class).
2018-06-05 23:24:48 -07:00
Raymond Akornor 0335da7e05 python: Use python3 style super() in 2FA code paths. 2018-06-04 20:55:42 -07:00
Steve Howell ad7e3ddd9f casper: Show casper debug files in the dev server.
Example:
    http://localhost:9991/casper/casper-failure1.png
2018-06-04 09:02:58 -07:00
Tim Abbott 714ad67db1 storage: Move library code into zerver/lib.
There's no particularly good reason for this to have been a top-level
file.
2018-06-03 16:02:02 -07:00
Steve Howell 4b2e8b83c4 slash commands: Add /ping command (via zcommand).
This adds a /ping command that will be useful for users
to see what the round trip to the Zulip server is (including
only a tiny bit of actual server time to basically give a
200).

It also introduce the "/zcommand" endpoint and zcommand.js
module.
2018-06-02 09:40:12 -07:00
Tim Abbott 2d6993f6dc settings: Enable caching with webpack-loader. 2018-05-31 16:45:51 -07:00
Tim Abbott 47824a97a4 ldap: Add return_data for the ldap_missing_attribute property.
This should make it possible in the future to do better error output
for this case.
2018-05-31 14:16:03 -07:00
Tim Abbott ecb3a2ccef ldap: Clarify outside_ldap_domain exception logic.
The previous logic made it look like catching ZulipLDAPException on
the authenticate() line was possible, but it isn't, because that
exception is actually being handled inside django-auth-ldap's
authenticate method.
2018-05-31 14:12:06 -07:00
Sampriti Panda 46711a43f0 pgroonga: Upgrade to PGroonga 2 API. 2018-05-31 13:00:34 -07:00
Yashashvi Dave 07b63e4886 jquery-form: Remove jquery-form plugin dependency.
Fixes #9430.
2018-05-29 15:44:40 -07:00
Tim Abbott 91ec0aba09 auth: Improve interactions between LDAPAuthBackend and EmailAuthBackend.
Previously, if you had LDAPAuthBackend enabled, we basically blocked
any other auth backends from working at all, by requiring the user's
login flow include verifying the user's LDAP password.

We still want to enforce that in the case that the account email
matches LDAP_APPEND_DOMAIN, but there's a reasonable corner case:
Having effectively guest users from outside the LDAP domain.

We don't want to allow creating a Zulip-level password for a user
inside the LDAP domain, so we still verify the LDAP password in that
flow, but if the email is allowed to register (due to invite or
whatever) but is outside the LDAP domain for the organization, we
allow it to create an account and set a password.

For the moment, this solution only covers EmailAuthBackend.  It's
likely that just extending the list of other backends we check for in
the new conditional on `email_auth_backend` would be correct, but we
haven't done any testing for those cases, and with auth code paths,
it's better to disallow than allow untested code paths.

Fixes #9422.
2018-05-28 22:47:47 -07:00
Tim Abbott 8119670da1 user_settings: Prevent LDAP users from setting a Zulip password.
Previously, if both EmailAuthBackend and LDAPAuthBackend were enabled,
LDAP users could set a password using EmailAuthBackend and continue to
use that password, even if their LDAP account was later deactivated.

That configuration wasn't supported at all before, so this doesn't fix
a pre-existing security issue, but now that we're making that a valid
configuration, we need to cover this case.
2018-05-28 22:47:47 -07:00
Eeshan Garg 0a1b429d04 settings: Remove PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS.
Now that we have a UI for this feature, we don't need this setting
anymore.
2018-05-24 17:18:14 -07:00
Vishnu Ks 54a002c2e2 requirements: Upgrade pyflakes to 2.0.0.
We fix a few errors that only the new version finds.
2018-05-24 11:31:36 -07:00
Steve Howell 1b57e568ff Add "zform" functionality to the web client.
A "zform" knows how to render data that follows our
schema for widget messages with form elements like
buttons and choices.

This code won't be triggered until a subsequent
server-side commit takes widget_content from
API callers such as the trivial chat bot and
creates submessages for us.
2018-05-24 09:30:22 -07:00
Steve Howell 435719c65b Add a schema checking concept to the web app.
This starts the concept of a schema checker, similar to
zerver/lib/validator.py on the server.  We can use this
to validate incoming data.  Our server should filter most
of our incoming data, but it's useful to have client-side
checking to defend against things like upgrade
regressions (i.e. what if we change the name of the field
on the server side without updating all client uses).
2018-05-24 09:30:22 -07:00
Umair Khan 9502cbbfab 2FA: Enable Django template backend.
The only purpose of this commit is to make the django templates
of Two Factor Auth work. We probably won't need this commit once
we upgrade the admin backend of Two Factor Auth to use handlebar
templates.
2018-05-23 15:46:56 -07:00
Armaan Ahluwalia 1525e92058 logging/errors/webpack: Improve error stack traces with blueslip.
This commit improves the output that blueslip produces while
showing error stack traces on the front-end. This is done by
using a library called error-stack-parser to format the stack
traces.

This commit also edits the webpack config to use a different
devtool setting since the previous one did not support sourcemaps
within stack traces. It also removes a plugin that was obviated
by this change.
2018-05-22 14:36:41 -07:00
Umair Khan f38d6ac6fe ldap: Make Zulip compatible with django-auth-ldap==1.5.
In version 1.5, get_or_create_user method is not used. It exists just
for the compatibility. The main function to use now is
get_or_build_user.

See the changelog:
https://django-auth-ldap.readthedocs.io/en/latest/changes.html#id1

Fixes #9307
2018-05-22 08:13:41 -07:00
Aditya Bansal 81a677e02b reminders: Refactor relevant code to live in a separate reminder.js. 2018-05-21 09:03:31 -07:00
Aditya Bansal 57f247283d scheduledmessages: Handle multiple servers running this delivery job.
We add conditional infinite sleep to this delivery job as a means to
handle case of multiple servers in service to a realm running this
job. In such a scenerio race conditions might arise leading to
multiple deliveries for same message. This way we try to match the
behaviour of what other jobs do in such a case.
Note: We should eventually do something to make such jobs work
while being running on multiple servers.
2018-05-21 08:53:14 -07:00
Sampriti Panda 3f4200db3c tests: Disable slow query messages in test environment.
Slow queries during backend tests sends messages to Error Bot
which affects the database state causing the tests to fail.
This fixes the occasional flakes due to that.
2018-05-20 10:16:53 -07:00
Yashashvi Dave 4162e61f33 /json/users: Replace email with user_id in API to reactivate user. 2018-05-18 15:20:43 -07:00
Yashashvi Dave 06e7e933cc /json/users: Replace email with user_id in API to update/remove users. 2018-05-18 15:20:43 -07:00
Steve Howell aa7c031f36 Add tictactoe widget.
Thanks to @amanagr for a few refinements to my original
version, which was really raw.  (It's still raw.)
2018-05-16 15:13:33 -07:00
Steve Howell d0e70a9700 Add poll widget. 2018-05-16 15:13:33 -07:00
Steve Howell b48f052b0c Add basic widgets framework (JS side). 2018-05-16 15:13:33 -07:00
Steve Howell a84bd6da57 Add client code for handling submessages.
This commit lays the foundation to handle submessages for
plugin widgets.  Right now it just logs events, but subsequent
commits will add widget functionality.
2018-05-16 15:13:33 -07:00
Steve Howell de47eeb6f1 Add /submessage endpoint. 2018-05-16 15:13:33 -07:00
Steve Howell 33d753d174 Add setting for ALLOW_SUB_MESSAGES. 2018-05-16 15:13:33 -07:00
Tim Abbott e63541e616 docs: Extract documentation for file upload backends.
This moves the documentation for this feature out of
prod_settings_template.py, so that we can edit it more easily.

We also add a bucket policy, which is part of what one would want to
use this in production.

This addresses much, but not all, of #9361.
2018-05-16 15:00:54 -07:00
Aman Ghebreweldi a781b89ec2 third: Use Underscore throttle function instead of jQuery.
Since we already have _.js and it has this feature, there's no need
for a separate library.
2018-05-16 11:56:49 -07:00
Steve Howell d2ee76896c Extract local_message.js.
We will eventually want these functions to support things
other than local echo, such as ephemeral messages from chat
bots.
2018-05-16 09:30:49 -07:00
Yashashvi Dave b949d10592 /json/bots: Replace email with user_id in API to generate bot_api_key.
Fixes #3643.
2018-05-15 16:37:06 -07:00
Yashashvi Dave d6e2f9fc88 /json/bots: Replace email with user_id in API to update bots. 2018-05-15 16:34:17 -07:00
Aditya Bansal 83d422d5bc zproject: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
Raymond Akornor 1a215d5504 zproject: Migrate away from six. 2018-05-09 18:57:01 +00:00
Tim Abbott c7b44d44e4 third: Remove now-unused LazyLoad library.
We don't reference this anymore (it was only ever used by the Dropbox
integration, which was hardcoded-off for years before being removed in
e6833b6427)
2018-05-08 14:38:27 -07:00
Steve Howell 27b7461e0a Revert "js: Implement DynamicText class."
This reverts commit 6e7305f784.

We never ending up using this class.
2018-05-08 12:13:55 -07:00
Tim Abbott cfd22c6832 zilencer: Clean up logic around mobile push notifications signup.
This fixes exceptions when sending PMs in development (where we were
trying to connect to the localhost push bouncer, which we weren't
authorized for, but even if we were, it wouldn't work, since there's
no APNS/GCM certs).

At the same time, we also set and order of operations that ensures one
has the opportunity to adjust the server URL before submitting
anything to us.
2018-05-05 16:42:00 -07:00
Steve Howell 264dcb6f40 refactor: Extract MessageListData class.
Most of this was straightforward.

Most functions that were grabbed verbatim and whole from
the original class still have one-line wrappers.

Many functions are just-the-data versions of functions that
remain in MessageList:  see add, append, prepend, remove as
examples.  In a typical pattern the MessageList code becomes
super simple:

    prepend: function MessageList_prepend(messages) {
        var viewable_messages = this.data.prepend(messages);
        this.view.prepend(viewable_messages);
    },

Two large functions had some minor surgery:

    triage_messages =
            top half of add_messages +
            API to pass three lists back

    change_message_id =
            original version +
            two simple callbacks to list

For the function update_muting_and_rerender(), we continue
to early-exit if this.muting_enabled is false, and we copied
that same defensive check to the new function
named update_items_for_muting(), even though it's technically
hidden from that codepath by the caller.
2018-05-05 06:31:51 -07:00
Tim Abbott 43098a6f7c zilencer: Add automated signup system for push notifications.
Based on an initial version by Rishi Gupta.

Fixes #7325.
2018-05-03 21:27:49 -07:00
Armaan Ahluwalia cca10beb78 css/webpack: Moved archive styles to webpack bundle.
This commit moves the stylesheets under the archive bundle in
the Django pipeline to being compiled by webpack instead. It
also removes a remaining call to a portico stylesheet that no
longer exists.
2018-05-03 19:15:49 -07:00
Armaan Ahluwalia 54bf2a6231 css/webpack: Transition landing-page.css to webpack.
This commit transitions landing-page.css from the Django pipeline
to being compiled by webpack as landing-page.scss under the
'landing-page' and 'integration' bundles.
2018-05-03 19:15:23 -07:00
Armaan Ahluwalia 1c016e990d css/webpack: Transition common.css to webpack.
This commit transitions common.css from the Django pipeline
to being compiled by webpack under the common bundle.
2018-05-03 19:15:23 -07:00
Umair Khan 9a3b310db9 2FA: Enable apps and middlewares unconditionally. 2018-05-03 11:59:32 -07:00
Priyank Patel 1e1b72f6c8 webpack: Transition stats.css to use webpack.
The stats.scss file is added to activity bundle.
2018-05-03 08:08:23 -07:00
Priyank Patel c780bc33ba webpack: Transition activity.css to use webpack.
The activity.scss file is added to activity bundle.
2018-05-03 07:53:24 -07:00
Armaan Ahluwalia 5f7b47e20c css: Transition 'app.css' to SCSS.
This commit transitions all styles in app.css in the Django pipeline
to being compiled by webpack in an app-styles bundle, and renames the
various files to now be processed as SCSS.

To implement this transition, we move the old CSS file refernces in
settings.py and replace them with a bundle declared in
`webpack.assets.json` and includedn in the index.html template

Tweaked by tabbott to keep the list of files in `app.css` in
`webpack.assets.json`, and to preserve the ordering from the old
`settings.py`.
2018-05-02 17:13:16 -07:00
Aditya Bansal 583f50179c public_archives: Add floating recipient bar. 2018-05-02 15:23:33 -07:00
Aditya Bansal 898c281692 public_archives: Add styling to the public topics. 2018-05-02 15:23:33 -07:00
Aditya Bansal 5d7907b59f public_archives: Add basic infra for displaying topics.
We add very basic infra so that we can view any discussion which
happened under a topic of a global public stream without
authorization.
2018-05-02 15:23:33 -07:00
Umair Khan 29e3a1d576 two_factor: Add templates for 2-factor-auth setup.
Note from Tim: We'll likely need to do some work on the strings in
these before translating, so I removed some translation tags.
2018-05-02 15:20:49 -07:00
Umair Khan e8b0ae821b two_factor: Fix namespace error.
If you specify namespace, Django throws an error.
2018-05-02 14:59:14 -07:00
Armaan Ahluwalia fce6882eb9 webpack: Move the styles consumed by 5xx.html to webpack.
This commit removes the need for portico.css to be generated
by the Django pipeline and makes the error page use the css
file compiled by webpack instead.
2018-05-02 09:45:01 -07:00
Tim Abbott d92edb8ea5 css: Remove support for legacy desktop app.
We haven't seen significant traffic from the legacy desktop app in
over a year, and users using it get a warning to upgrade since last
summer, so it's probably OK to stop providing special fonts for it.
2018-05-01 09:49:28 -07:00
Angelika Serwa f4f64243dd custom_profile_fields: Support changing the sort order of the fields.
Tweaked by tabbott for variable naming and the URL.

Closes #8879.
2018-04-30 18:17:41 -07:00
Tim Abbott 7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
Steve Howell 97b9367d20 refactor: Extract scroll_util.js.
We now use scroll_element_into_container() in two different
places, so it's worth extracting.
2018-04-28 11:15:14 -07:00
Steve Howell fb712027bf buddy list: Fix and simplify up/down navigation.
This introduces a generic class called list_cursor to handle the
main details of navigating the buddy list and wires it into
activity.js.  It replaces some fairly complicated code that
was coupled to stream_list and used lots of jQuery.

The new code interacts with the buddy_list API instead of jQuery
directly.  It also persists the key across redraws, so we don't
lose our place when a focus ping happens or we type more characters.

Note that we no longer cycle to the top when we hit the bottom, or
vice versa.  Cycling can be kind of an anti-feature when you want to
just lay on the arrow keys until they hit the end.

The changes to stream_list.js here do not affect the left sidebar;
they only remove code that was used for the right sidebar.
2018-04-28 11:15:14 -07:00
Steve Howell 65d8eb3189 buddy list: Extract user_search.js.
This was a bit more than moving code.  I extracted the
following things:

        $widget (and three helper methods)
        $input
        text()
        empty()
        expand_column
        close_widget
        activity.clear_highlight

There was a minor bug before this commit, where we were inconsistent
about trimming spaces.  The introduction of text() and empty() should
prevent bugs where users type the space bar into search.
2018-04-28 11:15:14 -07:00
Armaan Ahluwalia 93ac40105f CSS: Move portico styles to webpack compilation.
static/styles/scss/portico.scss is now compiled by webpack
and supports SCSS syntax.

Changed the server-side templates to render the portico-styles
bundle instead of directly requiring the portico stylesheet. This
allows webpack to handle stylesheet compilation and minification.

We use the mini-css-extract-plugin to extract out css from the
includes in webpack and let webpacks production mode handle
minification. Currently we're not able to use it for dev mode
because it does not support HMR so we use style-loader instead.
Once the plugin supports HMR we can go on to use it for both
dev and prod.

The downside of this is that when reloading pages in the development
environment, there's an annoying flash of unstyled content :(.

It is now possible to make a change in any of the styles included
by static/styles/scss/portico.scss and see the code reload live
in the browser. This is because style-loader which we currently
use has the module.accept code built-in.
2018-04-27 09:04:50 -07:00
Tim Abbott e4ca6e947b ldap: Disable django-auth-ldap caching of users.
This shouldn't have a material performance impact, since we don't
query these except during login, and meanwhile this fixes an issue
where users needed to restart memcached (which usually manifested as
"needing to reboot the whole server" after updating their LDAP
configuration before a user who was migrated from one OU to another
could login).

Fixes #9057.
2018-04-24 09:57:55 -07:00
Aditya Bansal ae398dc48b csp_nonce: Add nonce to script tags loading minified JS. 2018-04-24 06:13:21 +05:30
Steve Howell 3f1930f9c5 buddy list: Extract buddy_data.js. 2018-04-22 20:08:08 -07:00
Steve Howell 536236d9b1 buddy list: Extract buddy_list.js. 2018-04-22 20:08:08 -07:00
Tim Abbott a9fb02b712 test_auth_backends: Add a test for GitHub auth mobile_flow_otp. 2018-04-22 19:55:05 -07:00
Tim Abbott 64023fc563 auth: Fix incorrect use of get_realm_from_request.
The code in maybe_send_to_registration incorrectly used the
`get_realm_from_request` function to fetch the subdomain.  This usage
was incorrect in a way that should have been irrelevant, because that
function only differs if there's a logged-in user, and in this code
path, a user is never logged in (it's the code path for logged-out
users trying to sign up).

This this bug could confuse unit tests that might run with a logged-in
client session.  This made it possible for several of our GitHub auth
tests to have a totally invalid subdomain value (the root domain).

Fixing that bug in the tests, in turn, let us delete a code path in
the GitHub auth backend logic in `backends.py` that is impossible in
production, and had just been left around for these broken tests.
2018-04-22 16:24:43 -07:00
Tim Abbott 65025e8327 auth: Add return_data for RemoteUserBackend.
This is done mainly because this backend has the simplest code path
for calling login_or_register_remote_user, more than because we expect
this case to come up.  It'll make it easier to write unit tests for
the `invalid_subdomain` corner case.
2018-04-22 14:44:06 -07:00
Tim Abbott fb6cc4cb65 secrets: Make it a bit easier to do GitHub auth in development. 2018-04-22 14:44:05 -07:00
Steve Howell f73bfd2a5c refactor: Extract settings_toggle.js.
This mostly moves code, and we also abandon the lookup mechanism
for finding our toggler.
2018-04-20 13:45:58 -07:00
Tim Abbott aa4b067e68 avatars: Fix 500 with the /avatar/ URL to be a 404.
Apparently, we had a somewhat sloppy regular expression for the URL
for this endpoint.
2018-04-18 12:43:57 -07:00
Tim Abbott e6833b6427 cleanup: Remove the legacy Dropbox file upload integration.
This has been hard-disabled for years, we have no plans to re-enable
it, and it has some hacky code in it.
2018-04-11 11:39:48 -07:00
Aditya Bansal 6c1a50da76 csp_reports: Add endpoint to handle logging of reports sent by clients. 2018-04-11 23:01:13 +05:30
Vishwesh Jainkuniya 383c62fb03 dev_login: Identify each user's realm when listing them.
This is a mobile-specific endpoint used for logging into a dev server.
On mobile without this realm_uri it's impossible to send a login request
to the corresponding realm on the dev server and proceed further; we can
only guess, which doesn't work for using multiple realms.

Also rename the endpoint to reflect the additional data.

Testing Plan:
Sent a request to the endpoint, and inspected the result.

[greg: renamed function to match, squashed renames with data change,
 and adjusted commit message.]
2018-04-10 17:03:36 -07:00
Greg Price 9a0fdf5b8d settings: Add a few more section headings.
This should help a bit more in making this file navigable.

I think there's further work that could be done to organize the
settings better: e.g., group LDAP with the auth section; separate
resource limits, from debugging and error reporting, from configuring
service dependencies like Redis and Rabbit.  That'd require reordering
many settings, and also taking a closer look at many settings one by
one in order to do a good job.  Leaving that for another day.
2018-04-05 21:24:48 -07:00
Greg Price 9a9d3097be settings: Add some visual weight to the section headings.
I've found this file hard to navigate for a while.  We actually have a
little hierarchy of section headings which applies to a lot of the
file already; make the boundaries bolder.
2018-04-05 21:24:48 -07:00
Greg Price f597f0b52e settings: Revise block comment at top of file.
Hopefully this is a bit clearer to read.
2018-04-05 21:24:48 -07:00
Greg Price 9956d61e20 settings: Revise comments on SMTP / outgoing email settings.
Add a clear heading, and use fewer words and simpler sentences.  Also
explain the password thing a bit more, and put that more inline next
to the username.

Also, on checking the Django docs, the default for EMAIL_USE_TLS
is False and for EMAIL_PORT is 25.  So most admins, certainly any that
are using an SMTP service on the public Internet (that is at all
decently run), will need to set those settings.  Mention that.
2018-04-05 21:24:48 -07:00
Steve Howell bd591424e2 Add keydown_util.js module.
This is a pretty thin abstraction to prevent having to put
magic numbers in code, doing the which/keyCode hack, and remembering
to all preventDefault.

Hopefully we'll expand it to handle things like shift/alt keys
for components that want their own keyboard handlers (vs. going
through hotkey.js).
2018-04-04 16:37:39 -07:00
Tim Abbott bec71d7a50 messages: Add a server-level setting to control private stream history.
We don't indend for this server-level setting to exist in the long
term; the purpose of this is just to make it easy to test this code
path for development purposes.

This implements much of the Message side part of #2745.
2018-04-04 16:18:46 -07:00
Tim Abbott 938c4cee08 settings: Add option to control Jitsi video chat integration.
Fixes #8922.
2018-04-02 16:55:16 -07:00
Tim Abbott af004fa6f5 rabbitmq: Connect to rabbitmq via 127.0.0.1 rather than localhost.
This should eliminate the occasional problems that some installations
have had with RabbitMQ trying to load-balance requests between
127.0.0.1 and ::1 (the ipv6 version of localhost).
2018-04-01 16:11:12 -07:00
Steve Howell 9ce9c2f9db Extract info_overlay.js.
There are several ways we open help for keyboard shortcuts,
markdown help, and search operators.

    - from the gear menu
    - from the compose box
    - from the search box
    - hitting ? for keyboard help
    - arrowing/clicking through the tabs

This just moves the relevant code into a module and changes a
bunch of one-line calls in various places.
2018-03-30 09:07:23 -07:00
Umair Khan c14cefc24c settings: Add support for adding/removing custom profile fields.
Now that we have support for displaying custom profile fields, this
adds administrator-level support for creating them.

Tweaked by tabbott to fix a few small bugs and clean up the commit message.

Fixes #1760.
2018-03-29 13:59:16 -07:00
Tim Abbott 656f882a44 bots: Eliminate NEW_USER_BOT.
This bot was basically a duplicate of NOTIFICATION_BOT for some
specific corner cases, and didn't add much value.  It's better to just
eliminate it, which also removes some ugly corner cases around what
happens if the user account doesn't exist.
2018-03-29 12:01:21 -07:00
Tim Abbott 013a8886b2 styles: Rename dark.css to night_mode.css. 2018-03-28 10:42:28 -07:00
Steve Howell 4f05cefb32 Remove obsolete code for stream subscribe buttons.
In 18e43895ff we replaced
stream subscribe buttons with stream links.  The new feature
has been well tested and well received for over a year now,
so it's safe to remove the older feature at this point.

Older sites will have super old messages that still have the
rendered markup; this commit does not attempt to address those
situations.  Most likely, clicking on an old button in the old
message will either do nothing or look like a message reply.
2018-03-24 12:31:36 -04:00
Greg Price b42a7b1701 digest: Add a server setting, and disable by default.
This feature isn't really ready yet -- the relevance isn't good, so
the emails aren't a great experience.  More work needed; pending that,
just don't send them.

There's already a per-realm setting, which doesn't have a control in
the org settings UI but does suppress it in the per-user settings UI.
Piggyback on that to suppress that UI control when the feature is
disabled at the server level too.

Also cut a comment that hasn't really made sense since the logic was
changed months ago -- the comment originally explained why we sent
digests on Tuesday, Wednesday, and Thursday, and doesn't correspond to
why we dialled back to weekly on Tuesdays.
2018-03-23 14:12:01 -07:00
Greg Price e44a8b8de9 logging: Add a setting to aid manual testing of error-notif changes.
This is nicer than the "For manual testing ..." comment. :-)
Also as a proper setting we can have it control some logging I
added locally while testing my recent changes to pika logging.
2018-03-21 18:03:05 -07:00
Greg Price fe0f1edddb settings: Fix double negative in LOGGING_NOT_DISABLED.
Saying "not disabled" just makes it more work to read than it needs to
be -- instead say ENABLED.
2018-03-21 18:03:05 -07:00
Aditya Bansal 1e48dac8f3 auth.py: Make redirects to 'next' url work for google and github.
In this commit we start to support redirects to urls supplied as a
'next' param for the following two backends:
* GoogleOAuth2 based backend.
* GitHubAuthBackend.
2018-03-21 13:35:44 -07:00
Gooca 2363c9955a ui scrollbar: Update perfect-scrollbar to 1.3.0.
Update perfect-scrollbar to fix stutter space-scrolling in #8544. Also
reworked deprecated `element.perfectScrollbar` to `new
PerfectScrollbar(element)`.  Lastly, updated provision version and
changed node module path to new path.

This also refactors perfect-scrollbar in help.js to work with updated
version of perfect-scrollbar. Because the update also changed
perfect-scrollbar's css selectors for all scrollbars in zulip, we
update those too.

Fixes #8544.
2018-03-20 21:59:11 -07:00
Greg Price 2a178b52fb logging: Suppress error mail from routine pika connection failures.
In some environments, these exceptions happen regularly on upgrading
Zulip.  They're harmless because we reconnect, so avoid complaining
noisily about them.
2018-03-20 16:49:05 -07:00
Nikhil Kumar Mishra f29a1918f3 hotspots: Add ALWAYS_SEND_ALL_HOTSPOTS in dev_settings.
Replace the local variable SEND_ALL from get_next_hotspots.
Add unit test for the same.
2018-03-19 10:39:43 -07:00
Greg Price 22071a44a7 antispam: Add a sitewide ratelimit on invites by new realms.
This applies only on a server open for anyone to create a realm.
Moreover, if the server admins have granted any given realm a
max_invites greater than the default, that realm is exempt too.
2018-03-16 18:00:11 -07:00
Greg Price dc1eeef30a antispam: Make a setting for default Realm.max_invites.
This makes this value much easier for a server admin to change than it
was when embedded directly in the code.  (Note this entire mechanism
already only applies on a server open for anyone to create a realm.)

Doing this also means getting the default out of the database.
Instead, we make the column nullable, and when it's NULL in the
database, treat that as whatever the current default is.  This better
matches anyway the likely model where there are a few realms with
specially-set values, and everything else should be treated uniformly.

The migration contains a `RenameField` step, which sounds scary
operationally -- but it really does mean just the *field*, in
the model within the Python code.  The underlying column's name
doesn't change.
2018-03-16 18:00:11 -07:00
feorlen 4631a2bfc3 Set REALM_HOSTS based on EXTERNAL_HOST if it is available.
Also update remote.md to recommend setting EXTERNAL_HOST in an environment
variable.

Fixes #8670
2018-03-13 09:45:37 -07:00
Steve Howell fef2840531 Add FetchStatus class.
We will use this in subsequent commits to replace the
`load_more_enabled` flag.
2018-03-09 14:20:58 -08:00
Steve Howell 8063f73f6d Extract message_scroll.js.
This mostly moves code from ui.js.

We change the arguments to `message_fetch.load_more_messages()`
to be `opts` with callbacks for `show_loading` and `hide_loading`.

We also defer starting the scroll handler until `message_fetch.js`
has been initialized.
2018-03-08 15:03:47 -08:00
Steve Howell 3a1bf04a56 compose: Add pills for typing in PM recipients.
@brockwhittaker wrote the original prototype for having
pills in the recipient box when users compose PMs (either
1:1 or huddle).  The prototype was test deloyed on our
main realm for several weeks.

This commit includes all the original CSS and HTML from
the prototype.

After some things changed with the codebase after the initial
test deployment, I made the following changes:

    * In prior commits I refactored out a module called
      `user_pill.js` that implemented some common functions
      against a more streamlined version of `input_pill.js`,
      and this commit largely integrates with that.

    * I made changes in a prior commit to handle Zephyr
      semantics (emails don't get validated) and tested
      this commit with zephyr.

    * I fixed a reload bug by extracting code out to
      `compose_pm_pill.js` and re-ordering some
      calls to `initialize`.

There are still two flaws related to un-pill-ified text in the
input:

    * We could be more aggressive about trying to pill-ify
      emails when you blur or tab away.

    * We only look at the pills when you send the message,
      instead of complaining about the un-pill-ified text.
      (Some folks may consider that a feature, but it's
      probably surprising to others.)
2018-03-07 15:53:11 -08:00
Steve Howell f0d101edf5 pills: Streamline input pills (for user groups).
The main point of this change is to streamline the core
code for input pills, and we use also modify user groups.

The main change to input_pill.js is that you now
configure a function called `create_item_from_text`, and
that can return an arbitrary object, and it just needs
a field called `display_value`.

Other changes:
    * You now call `input.create(opts)` to create the
      widget.
    * There is no longer a cache, because we can
      write smarter code in typeahead `source` functions
      that exclude ids up front.
    * There is no value/optinalKey complexity, because
      the calling code can supply arbitrary objects and
      do their own external data management on the pill
      items.
    * We eliminate `prependPill`.
    * We eliminate `data`, `keys`, and `values`, and just
      have `items`.
2018-03-07 15:53:11 -08:00
YJDave f5a24efdd8 static/js: Add `settings_ui.js` file for common settings UI functions. 2018-03-03 08:38:54 -08:00
Vishnu Ks 56e54262c3 api: Create api for creating multiuse invites. 2018-03-02 11:28:36 +00:00
Greg Price 6d35a697af antispam: Don't let new accounts in open realms immediately send invites.
We haven't had a problem with this yet, but this should help prevent it.
2018-03-01 14:56:19 -08:00
Vishnu Ks 8152532535 urls: Use /new endpoint for creating new realm.
This is just nicer-looking for potentially nontechnical users than
/create_realm.
2018-02-28 13:47:54 -08:00
Tim Abbott 34efab9157 auth: Report to mobile apps the availability of RemoteUserBackend.
This is necessary for mobile apps to do the right thing when only
RemoteUserBackend is enabled, namely, directly redirect to the
third-party SSO auth site as soon as the user enters the server URL
(no need to display a login form, since it'll be useless).
2018-02-24 08:14:17 -08:00
Umair Khan d22639717c auth: Redirect to an error page instead of 500.
Previously, we used to raise an exception if the direct dev login code
path was attempted when:

* we were running under production environment.
* dev. login was not enabled.

Now we redirect to an error page and give an explanatory message to the
user.

Fixes #8249.
2018-02-20 22:34:53 -08:00
Steve Howell 1f6ddf0110 refactor: Extract transmit.js from compose.js.
We now isolate the code to transmit messages into transmit.js.
It is stable code that most folks doing UI work in compose.js don't
care about the details of, so it's just clutter there.  Also, we may
soon have other widgets than the compose box that send messages.

This change mostly preserves test coverage, although in some cases
we stub at a higher level for the compose path (this is a good thing).
Extracting out transmit.js allows us to lock down 100% coverage on that
file.
2018-02-20 09:29:26 -08:00
Vishnu Ks d34dd4cd02 signup: Show subdomain availability during signup.
This uses an actual query to the backend to check if the subdomain is
available, using the same logic we would use to check when the
subdomain is in fact created.
2018-02-19 10:45:17 -08:00
Tim Abbott be9dbccdfa settings: Link to documentation on changing settings.
This is obviously more important than the pre-existing link to the
developer documentation for this system.

Fixes #7910.
2018-02-19 09:55:50 -08:00
Shubham Dhama 9feae472f8 org settings: Add button to deactivate organization.
This adds button under "Organization profile" settings, which
deactivates the organization and sends an "event" to all the
active user and log out them.

Fixes: #8212.
2018-02-18 10:20:38 -08:00
Tim Abbott b3d37b5412 sendfile: Clean up sendfile settings code. 2018-02-16 05:10:19 +05:30
Aditya Bansal efe8545303 local-uploads: Start running authentication checks on file requests.
From here on we start to authenticate uploaded file request before
serving this files in production. This involves allowing NGINX to
pass on these file requests to Django for authentication and then
serve these files by making use on internal redirect requests having
x-accel-redirect field. The redirection on requests and loading
of x-accel-redirect param is handled by django-sendfile.

NOTE: This commit starts to authenticate these requests for Zulip
servers running platforms either Ubuntu Xenial (16.04) or above.

Fixes: #320 and #291 partially.
2018-02-16 05:06:37 +05:30
Greg Price 8d98ce1648 errors: Note upstream fix for that Django DisallowedHost bug. 2018-02-15 10:37:29 -08:00
Greg Price c6ce6dd5c6 settings: Try to clarify how AUTH_LDAP_USER_ATTR_MAP works.
This comment didn't really explain things unless you were looking at
the code, and/or had a strong enough association for what "cn" means
that it was obvious what must be meant.  Maybe this will be clearer.

There is one other meaningful key, which is optional: "short_name",
for which I guess a typical value if supplied would be "uid" or
"userid".  I'm not sure we really do much with a user's `short_name`,
though, so didn't add a comment for it.  When this key is omitted,
we set the user's `short_name` to the same thing as `full_name`.
2018-02-13 11:18:14 -08:00
Aman Jain a0b58b1560 activity: Add a link to copy list of realm admins.
Tweaked by tabbott to do the database queries properly.

This should help user to copy realm admin emails in a go.
Fixes: #7885
2018-02-12 08:55:45 -08:00
Greg Price 12d878c785 APNs: Drop disused setting from old implementation. 2018-02-09 17:16:21 -08:00
Tim Abbott afda05adc4 rate limiting: Increase default rate limits.
It's too easy to go over the rate limits when using the webapp.

The correct fix for this probably involves some changes to which
routes get covered by what sort of rate limit, but for now, just
increase the limits.
2018-02-09 11:55:08 -08:00
Greg Price 9e70026741 settings: Slightly clarify LDAP setup docs.
There's kind of a lot of prose in this LDAP section; and empirically
this bit about zulip-secrets.conf is easy to miss.
2018-02-08 10:05:25 -08:00
Brock Whittaker fe3a5665da panels: Rename "desktop_notifications_panel" => "panels".
The original code made a 3/4-hearted effort to generically accommodate
more banners/"panels" later, but named itself after the first one made.

[greg: expanded commit message.]
2018-02-05 16:00:52 -08:00
Greg Price 93e29629ab settings: In ALLOWED_HOSTS example, show a list and an IP address.
Different formats for configuration files have a wide variety of ways
of representing lists; so if you're not accustomed to Python syntax,
or aren't thinking of this file as Python code, the syntax for several
ALLOWED_HOSTS entries may not be obvious.  And this setting is one
that an admin is likely to want to touch quite early in using Zulip.
So, demonstrate a multi-element list.

For similar reasons, demonstrate an IP address.  This one is in a
range reserved for documentation (by RFC 5737), like `example.com`.
2018-02-05 10:27:04 -08:00
rht 9a8d2244ca django-2.0: Shift to resolvers from urlresolvers.
The old name is deprecated.
2018-01-30 10:53:54 -08:00
Aditya Bansal 0fcf0c5052 thumbor: Add thumbor on port 9995 in development.
For now, this does nothing in a production environment, but it should
simplify the process of doing testing on the Thumbor implementation,
by integrating a lot of dependency management logic.
2018-01-29 13:10:29 -08:00
Greg Price 3230d35d7f settings: Clean up EMAIL_HOST documentation a bit.
Gmail is a bad example for outbound email; use a generic example.

Also leave the `= None` default out of the config file, as it's
redundant with DEFAULT_SETTINGS in our internal settings.py ; and
explain in the latter why we don't mention the other SMTP settings.
2018-01-24 14:58:21 -08:00
Greg Price 6da9ecef88 settings: Set a flag when email isn't configured, so we can warn. 2018-01-24 14:34:30 -08:00
Greg Price 08bbd7e61d settings: Slightly simplify EMAIL_BACKEND logic.
DEVELOPMENT is defined as just `not PRODUCTION`, but this code made
it look like things might be more complicated than that.
2018-01-24 14:34:30 -08:00
Tim Abbott 56395f40d2 static: Restore bundling KaTeX and zxcvbn via django-pipeline.
Since we need KaTeX to be available for zerver/lib/tex.py and
static/third/katex/cli.js to be able to shell out to it.  However, for
some reason, the KaTeX we bundle using Webpack doesn't seem to be
importable by Node (and it's also kinda a pain to find its filename
from `cli.js`).

So, we work around this by just using the legacy system for KaTeX.

Something similar is needed for zxcvbn.js, in order to support the
settings_account.js use case (basically deferred loading of this
file); that requires JS code to have access to the correct path for
zxcvbn.
2018-01-23 12:30:28 -08:00
Aditya Bansal 76f6f7cb47 datetimepicker: Add flatpickr lib as dependancy. 2018-01-19 11:33:11 -05:00
Aditya Bansal c770bdaa3a reminder_bot: Add infra for adding reminder bot to every realm. 2018-01-19 11:33:11 -05:00
Vishnu Ks 0bca0286a1 billing: Integrate Stripe, using Stripe Checkout.
Stripe Checkout means using JS code provided by Stripe to handle
almost all of the UI, which is great for us.

There are more features we should add to this page and changes we
should make, but this gives us an MVP.

[greg: expanded commit message; fixed import ordering and some types.]
2018-01-17 16:43:54 -08:00
Reid Barton b01b4498cc Fix a typo in prod_settings_template.py. 2018-01-16 08:47:18 -08:00
Aditya Bansal ec1297c1e8 schedulemessages: Add delivery system for scheduled message. 2018-01-10 09:18:02 -05:00
Alena Volkova 9d1063d362 urls: Move the json/fetch_api_key endpoint to be an API-style route. 2018-01-08 13:15:52 -05:00
Vishnu Ks ec207e5697 slack importer: Use standalone database for running tests. 2018-01-07 09:50:27 -05:00
Cynthia Lin 7790e2b23c org settings: Add framework for User groups page. 2018-01-06 11:50:52 -05:00
Eeshan Garg 479bc41ca5 api docs: Remove old and outdated endpoint docs. 2018-01-04 10:17:29 -05:00
Rhea Parekh 092472f42d tests: Add test coverage for uploading avatars to S3UploadBackend.
Add test coverage for the function upload_avatar_image and
ensure_medium_avatar_image in S3UploadBackend.
2017-12-26 09:09:31 -05:00
Greg Price ca5c991994 errors: Simplify manual testing of error emails.
There are two different things you need to patch in order to get error
emails (at `/emails`) in dev.  Flip one of them in dev all the time,
and make the comment on the other a bit more explicit.
2017-12-22 11:52:13 -05:00
Greg Price b15231dfc2 logging: Rename AdminZulipHandler to AdminNotifyHandler.
This name hasn't been right since f7f2ec0ac back in 2013; this handler
sends the log record to a queue, whose consumer will not only maybe
send a Zulip message but definitely send an email.  I found this
pretty confusing when I first worked on this logging code and was
looking for how exception emails got sent; so now that I see exactly
what's actually happening here, fix it.
2017-12-22 11:52:13 -05:00
Vishnu Ks 16d8244c0a tests: Eliminate 'Sending invitation' output spam in test_signup.
Fixes #7563
2017-12-20 18:50:01 -08:00
Greg Price cd5e44491c logging: Drop confusing use of `propagate` in test_settings.
This line was added in e8ab7cd1a as a desperate measure to get the
`set_loglevel` helper to successfully suppress the `zulip.send_email`
logs.

In fact, the reason that just setting `level` there wasn't doing the
job had nothing to do with `propagate`, which doesn't interact with
`level`.  (See the long block comment in `zproject/settings.py`,
searching for "Python logging module", for my attempt at concisely
explaining these semantics.)  Rather, our setting for `level` was
getting clobbered by the use of `create_logger`; and setting
`propagate` to False worked by completely suppressing all logs to
`zulip.send_email` and descendants from reaching the normal handlers,
regardless of the log messages' levels.

Now that the `create_logger` issue is fixed (see a few commits before
this one), drop that.
2017-12-12 17:17:08 -08:00
Greg Price b6491fd52a logging: Eliminate log_level from create_logger.
Because calls to `create_logger` generally run after settings are
configured, these would override what we have in `settings.LOGGING` --
which in particular defeated any attempt to set log levels in
`test_settings.py`.  Move all of these settings to the same place in
`settings.py`, so they can be overridden in a uniform way.
2017-12-12 17:17:08 -08:00
Greg Price 10fc40a3f1 dev_settings: Remove accidentally-pushed OAuth app IDs.
These aren't of much use without the corresponding secrets (despite us
calling one of them a "key", both are public IDs), which are in my
personal dev-secrets.conf.  So, remove them.

I had this commit applied in order to test what became 90944983f...
and then I accidentally included it in what I pushed, as 0fa9a489d.
Oops.
2017-12-11 21:06:35 -08:00
Greg Price 0fa9a489d5 wip test oauth 2017-12-11 20:03:52 -08:00
Brock Whittaker e0236646bf night-mode: Add custom CSS through JS.
This adds custom CSS through JavaScript for things that do not
scope well and will override other inherited styles.

This should ONLY be used for problematic CSS that has no obvious
or easy CSS-only solution.

(Specifically, we need this for the "default link" styling, which is
hard to override because we don't want to start winning ties due to
specificity that we would not have won in the light theme).
2017-11-29 23:06:11 -08:00
Rishi Gupta 7c9694077e confirmation: Move check_prereg_key_and_redirect to registration.py. 2017-11-29 22:18:05 -08:00
Rishi Gupta 3675d97870 confirmation: Refactor views.confirm to be clearer.
Also gives more appropriate error messages for expired user_registration and
invitation links.
2017-11-29 22:03:09 -08:00
Tim Abbott f9347962f6 settings: Only include two-factor apps if two-factor auth is enabled. 2017-11-29 11:09:16 -08:00
Vishnu Ks 954c363a2f dev_settings: Convert hostname to lower case when setting external host. 2017-11-29 10:02:55 -08:00
Tim Abbott 586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
Greg Price b6cc21b438 debug: Add facility to dump tracemalloc snapshots.
Originally this used signals, namely SIGRTMIN.  But in prod, the
signal handler never fired; I debugged fruitlessly for a while, and
suspect uwsgi was foiling it in a mysterious way (which is kind of
the only way uwsgi does anything.)

So, we listen on a socket.  Bit more code, and a bit trickier to
invoke, but it works.

This was developed for the investigation of memory-bloating on
chat.zulip.org that led to a331b4f64 "Optimize query_all_subs_by_stream()".

For usage instructions, see docstring.
2017-11-28 15:52:07 -08:00
Umair Khan 274bba82b9 two_factor: Add configuration and URLs.
This adds django-two-factor to the project, but held behind
settings.TWO_FACTOR_AUTHENTICATION_ENABLED, so that this has no effect
by default.
2017-11-28 15:21:40 -08:00
Tommy Ip ddaff4cd2a refactor: Extract upload mechanics to new JS module.
Tweaked by tabbott to move changes from the next commit that are
required for this to pass tests into this commit.

Note that this exports a few items that were not previously exported.
2017-11-27 21:31:51 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
rht 92888a0cde zproject: Use Python 3 syntax for typing. 2017-11-27 17:01:18 -08:00
Tim Abbott 8b935f4e99 settings: Add setting for SYSTEM_BOT_REALM.
This fixes some subtle JavaScript exceptions we've been getting in
zulipchat.com, caused by the system bot realm there not being "zulip"
interacting with get_cross_realm_users.
2017-11-27 14:46:07 -08:00
Tim Abbott d1ff4293a5 backends: Remove assumption that only one user can have a given email.
I probably should have just done this in the original implementation;
there's only a small downside in the form of an extra database query
when trying to authenticate a user who doesn't exist.
2017-11-26 15:42:48 -08:00
derAnfaenger e526d0c144 embedded bots: Add views to access state. 2017-11-21 21:10:39 -08:00
Tim Abbott 719d6c49df forms: Stop using get_user_profile_by_email in OurAuthenticationForm.
Structurally, the main change here is replacing the `clean_username`
function, which would get called when one accessed
self.cleaned_data['username'] with code in the main `clean` function.

This is important because only in `clean` do we have access to the
`realm` object.

Since I recently added full test coverage on this form, we know each
of the major cases have a test; the error messages are unchanged.
2017-11-21 20:14:12 -08:00
Tim Abbott 36bc037cc2 auth: Convert SocialAuthMixin to use new helper.
This is a pure refactor at this point.
2017-11-21 20:14:12 -08:00
Tim Abbott 22b7de0ccd auth: Move check for social backend earlier.
This better fits the flow that we use in other auth backends.
2017-11-21 20:14:12 -08:00
Tim Abbott 665fc594db auth: Set valid_attestation more unconditionally in social auth. 2017-11-21 20:14:12 -08:00
Tim Abbott ade5b4ea69 auth: Convert SocialAuthMixin to accept a realm object. 2017-11-21 20:14:12 -08:00
Tim Abbott 732dd1b6a3 auth: Improve logic for invalid GitHub emails.
This deletes the old mock-covered test for this, which was mostly
useless.  We have a much less messy test, which we extend to provide
the same test coverage the old one did.

While the result was the same before, this makes it more obvious.
2017-11-21 20:14:12 -08:00
Garvit 52069998eb compose: Preserve links when pasting html.
This uses the to-markdown.js library to do all the hard work of
parsing HTML and turning it into markdown and not e.g. uploaded files.

Tweaked by tabbott to better scope when it activates to just include
pastes of HTML content.

Fixes #5853.
2017-11-21 19:52:47 -08:00
Tim Abbott 1c9a28d0d8 ldap: Use simpler ordering for handling successful auth.
common_get_active_user returns None if it finds any problems.
2017-11-21 19:08:45 -08:00
Tim Abbott e0b56c72de ldap: Simplify logic for user creation.
self._realm can't be None here with the new logic in authenticate().
2017-11-21 19:08:45 -08:00
Tim Abbott e91051b1cd ldap: Remove some unnecessary indentation.
We created this redundant pair of conditionals in a preceding commit,
in order to match the indentation of an `except` block so as to slice
the diffs extra finely as we're refactoring auth code.
2017-11-21 19:06:19 -08:00
Tim Abbott c4c8879cf7 ldap: Fix the error message for deactivated users. 2017-11-21 18:35:05 -08:00
Tim Abbott 97f1c2a72a ldap: Use new helper for checking realm status.
We intentionally don't fix the indentation that now feels ridiculous
below in order to make it easier to see what's actually changing in
this commit.
2017-11-21 18:35:04 -08:00
Tim Abbott 104a8de148 ldap: Shrink unnecessary scope of missing user block.
This is a pure refactor, and will help simplify the change in the next
commit.
2017-11-21 18:30:51 -08:00
Tim Abbott e100935527 auth: Move LDAP check for whether backend is enabled earlier.
The previous logic felt fairly convoluted.
2017-11-21 18:30:51 -08:00
Tim Abbott 195a78ad11 auth: Convert EmailAuthBackend to use new helper.
This lets us delete some duplicate code, since common_get_active_user
handles an account in the wrong subdomain for us.

Also lets us delete the now-unused common_get_active_user_by_email.
2017-11-21 18:30:51 -08:00
Tim Abbott 8c21619be8 auth: Move checks for password_auth_enabled earlier.
This way, we don't attempt to evaluate whether the user's account is
active (etc.) until after we've checked the backend is enabled.  This
won't change the result of actual auth, but feels more readable.
2017-11-21 18:30:29 -08:00
Tim Abbott 3bfb19b5f3 Convert EmailAuthBackend and LDAPAuthBackend to accept a realm. 2017-11-21 18:23:50 -08:00
Tim Abbott 53224a16a9 EmailAuthBackend: Convert a return to assert for a now-impossible case. 2017-11-21 18:23:50 -08:00
Tim Abbott 1b95b098dd auth: Clarify comments explaining the GoogleMobileOauth2Backend. 2017-11-21 18:23:50 -08:00
Tim Abbott 23d791ca1b auth: Convert GoogleMobileOauth2Backend to use new helper.
That logic was now just duplicate code.
2017-11-21 18:23:49 -08:00
Tim Abbott caddef9279 auth: Invert conditionals in GoogleMobileOAuth2Backend.
This will help make the flow more readable.
2017-11-21 18:23:49 -08:00
Tim Abbott fee2e36800 auth: Set valid_attestation for Google auth backend always.
This is a behavior change, though we don't check the value in the
caller regardless.  It just seems more logical for us to correctly
report to the caller whether the Google auth itself was valid
unconditionally.
2017-11-21 18:23:49 -08:00
Tim Abbott 3c15f442fe auth: Check for GoogleMobileOauth2Backend being enabled earlier. 2017-11-21 18:23:49 -08:00
Tim Abbott a7d51127fb auth: Convert GoogleMobileOAuth2Backend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 37acfb4e90 auth: Convert DevAuthBackend to use new helper. 2017-11-21 18:23:49 -08:00
Tim Abbott fa8eab303a auth: Check for DevAuthBackend being enabled earlier. 2017-11-21 18:23:49 -08:00
Tim Abbott 07bc31f818 auth: Convert DevAuthBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott 4968631d1b auth: Convert DevAuthBackend to use a unique argument pattern.
This helps ensure that we won't accidentally activate this backend on
other code paths.
2017-11-21 18:23:49 -08:00
Tim Abbott f2d3258a56 auth: Rewrite RemoteUserBackend to use new helper. 2017-11-21 18:23:49 -08:00
Tim Abbott 73df431b88 auth: Check for RemoteUserBackend being enabled earlier.
This is possible now that we have a realm object before fetching the
UserProfile object.
2017-11-21 18:23:49 -08:00
Tim Abbott d63e9f240c auth: Remove unnecessary remote_user=None code path.
This code path was only required because we had remote_user set as a
positional argument here, and thus we'd be running this auth backend's
code when actually using another auth backend (due to how Django auth
backends are selected based on argument signature).
2017-11-21 18:23:49 -08:00
Tim Abbott 387c9109ec auth: Convert RemoteUserBackend to accept a realm object. 2017-11-21 18:23:49 -08:00
Tim Abbott fb6abe1b1e auth: Rewrite DummyAuthBackend to not block email reuse.
This require some care to ensure we still provide the same nice error
messages for the case of a user who has an account, just not with this
organization.

Also, we fix the fact that the docstring was (and I think always has
been) at best confusing and perhaps even inaccurate.
2017-11-21 18:23:26 -08:00
Tim Abbott f17974ab32 DummyAuthBackend: Require being passed a realm object.
We should now always know the realm in our auth code paths.
2017-11-21 18:22:37 -08:00
Tim Abbott 089ff8a0ae backends: Convert authenticate methods to modern type annotations. 2017-11-21 11:47:26 -08:00
rht bff736868e Generate custom-icon-webfont on each provision or update-prod-static.
Fixes #7354.
2017-11-20 16:36:49 -08:00
Tim Abbott 80a2a36dd1 reactions: Rename legacy reactions for greater clarity. 2017-11-20 15:27:21 -08:00
Tim Abbott c8edbae21c password reset: Fix error message for invalid realm.
This is a lot cleaner than the previous model.

Basically rewritten by Vishnu Ks to actually work :).
2017-11-20 10:34:55 -08:00
Rishi Gupta f6bf11f5e6 portico: Add for/mystery-hunt. 2017-11-19 17:04:17 -08:00
Umair Khan 1e3aa55f4f github: Override get_authenticated_user.
Now we have moved the `do_auth` function to `SocialAuthMixin`. Instead
of overriding `do_auth`, derived class is now expected to override
`get_authenticated_user`.

`do_auth` now contains code which is expected by all backends.
2017-11-17 11:44:13 -08:00
derAnfaenger 395f1e9270 embedded bots: Add database config storage.
Storage limititations are only set on the value of
a config entry, since this is the only user-accessible
part of the schema. Keys are statically set by each
embedded bot.
2017-11-16 23:06:38 -08:00
Aditya Bansal 79576797b0 thumbor: Auto generate thumbor_key and add it to zulip settings. 2017-11-16 22:34:14 -08:00
Rishi Gupta f0807353ae settings: Remove ABOUT_LINK_DISABLED.
Was introduced in 0f4bbc8, I think because the /team page was unfinished at
the time.
2017-11-16 21:15:24 -08:00
Harshit Bansal d9c2f613e3 api: Add new endpoint for reactions.
This endpoint will allow us to add/delete emoji reactions whose emoji
got renamed during various emoji infra changes. This was also a
required change for realm emoji migration.

This commit was tweaked significantly by tabbott for greater clarity
(with no changes to the actual logic).
2017-11-16 20:52:15 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Brock Whittaker b76578ca13 Add an experimental dark mode stylesheet.
Note from tabbott: While this initial version is experimental and
definitely incomplete, we expect to have a solid version done over the
next few weeks (after more refactoring).  We're merging this now to
make it easy to test both versions when refactoring our CSS.

Fixes #267.
2017-11-15 16:45:34 -08:00
Vishnu Ks 8a46ddf9d7 dev_settings: Add localhost:9991 as realm host for zulip realm.
[greg: Tweaked explanatory comment.]
2017-11-14 18:34:17 -08:00
Steve Howell ae5ba7f4fd Add a couple cross-realm bots.
These are new:

    new-user-bot
    emailgateway

Our cross-realm bots are hard coded to have email addresses
in the `zulip.com` domain, and they're not part of ordinary
realms.

These have always been cross-realm, but new enforcement in the
frontend code of all messages having been sent by a known user means
that it's important to add these properly.
2017-11-14 16:02:19 -08:00
Greg Price 415da352df prod install: Cut premature suggestion to configure auth backends.
The warning here means that the admin can't really act on this yet if
they want to disable email auth, which is likely among admins that
want to make any changes here.  And for admins who don't, this is an
extra thing to read and make a decision about before they can get a
server running.  See #6985.

Conversely, we already discuss auth backends right at the top of the
`prod-customize` doc, which is linked under "Next steps" at the end of
these instructions.

The warning about EmailAuthBackend is important; but we can move it to
the config file right next to the setting, and then it's available
right when it's actionable, which is if the admin is actually thinking
about changing the setting.
2017-11-14 15:57:39 -08:00
Vishnu Ks 36f29764cb emails: Add option to forward mails send in dev env to external email.
Fixes #7085.
2017-11-14 15:27:29 -08:00
Vishnu Ks 2af249dd5d api: Add description attribute to default stream group. 2017-11-14 14:41:42 -08:00
Vishnu Ks f45ba7de93 api: Pass group id instead of name to default stream group api. 2017-11-14 14:41:42 -08:00
Umair Khan 0e21cbc5d9 user-groups: Create static/js/user_groups.js. 2017-11-13 13:08:15 +05:00
derAnfaenger d707e3bc1d bots: Add USER_STATE_SIZE_LIMIT setting. 2017-11-10 18:56:26 -08:00
Tim Abbott 54aa87fba3 api: Migrate to using the new version of the API site. 2017-11-10 17:56:22 -08:00
Vishnu Ks 4a102b248a droplets: Set EXTERNAL_HOST to username.zulipdev.org:9991. 2017-11-10 16:13:53 -08:00
Vishnu Ks f44b60a150 Implement API for default stream groups. 2017-11-10 16:05:36 -08:00
Brock Whittaker 00d11fb0af input-pill: Add styling.
This adds some styling for the default case of the input pills and some
custom styling.
2017-11-10 14:14:03 -08:00
Brock Whittaker 7b00736fa2 input-pill: Add "input_pill" class and documentation. 2017-11-10 14:14:03 -08:00
Cynthia Lin 0b800b0a7d icons: Create framework for custom icons and add new bot icon. 2017-11-10 11:18:42 -08:00
Umair Khan b1603d289c user-groups: Add update members API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:37 -08:00
Umair Khan a1b894a786 user-groups: Add delete API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:30 -08:00
Umair Khan eeaf0f1742 user-groups: Add edit API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:22 -08:00
Umair Khan 1bbe84af49 user-groups: Add create API endpoint.
Significantly modified by tabbott for better security structure.
2017-11-09 17:26:14 -08:00
Tim Abbott 900cd13865 docs: Link to postgres documentation for SSL modes.
For some historical reason we'd had the Postgres documentation on
valid SSL modes copied into the Zulip settings.py template file.  This
fixes that historical artifact.
2017-11-07 16:55:32 -08:00
rht 047eca1629 zproject: Remove unused imports (F401). 2017-11-07 16:37:03 -08:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
Steve Howell f105c480b3 Calculate gravatar hashes on the webapp client.
This change prepares us to have the server send avatar_url
of None when somebody wants a gravatar avatar (as opposed
to a user-uploaded one).

Subsequent commits will change behavior on both the server
and client to have this happen.  So this commit has no-op
code for now, but it will soon use the fallback-to-gravatar
logic.
2017-11-07 10:36:02 -08:00
rht e33c8d91e0 zproject: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
rht 26b6b893e6 zproject: Use python 3 syntax for typing. 2017-11-04 19:58:03 -07:00
neiljp (Neil Pilgrim) 1197ff9655 mypy: Explicitly return Set[Any] for empty set in backends.py. 2017-11-04 19:47:45 -07:00
Greg Price 909631b5a9 docs/oauth: Update for Google UI changes, and for zulipdev.com.
The control panel on the Google side doesn't seem to match the
instructions we have; it looks pretty 2017 to me, so I imagine
it's had a redesign since the instructions were written.

Also, in dev, EXTERNAL_HOST is now a port on zulipdev.com, not on
localhost.

Update these instructions for those developments, and edit lightly.
In dev, recommend setting in `dev_settings` instead of in
`prod_settings_template`; that feels to me a little more reflective of
the actual intent, and the effect should be equivalent.
2017-11-03 13:38:59 -07:00
Greg Price 119bc14182 alias domains: Add a v1 of this feature.
The main limitation of this version is that it's controlled entirely
from settings, with nothing in the database and no web UI or even
management command to control it.  That makes it a bit more of a
burden for the server admins than it'd ideally be, but that's fine
for now.

Relatedly, the web flow for realm creation still requires choosing a
subdomain even if the realm is destined to live at an alias domain.

Specific to the dev environment, there is an annoying quirk: the
special dev login flow doesn't work on a REALM_HOSTS realm.  Also,
in this version the `add_new_realm` and `add_new_user` management
commands, which are intended for use in development environments only,
don't support this feature.

In manual testing, I've confirmed that a REALM_HOSTS realm works for
signup and login, with email/password, Google SSO, or GitHub SSO.
Most of that was in dev; I used zulipstaging.com to also test
 * logging in with email and password;
 * logging in with Google SSO... far enough to correctly determine
   that my email address is associated with some other realm.
2017-11-03 13:38:59 -07:00
Greg Price b33d981d7b settings: Simplify setting ALLOWED_HOSTS, by handling EXTERNAL_HOST.
This means one fewer thing the admin typically needs to read, absorb,
and make a decision about at install time.

The one way this change could hypothetically cause trouble is if the
admin wants to keep subdomains of EXTERNAL_HOST out of ALLOWED_HOSTS.
But while the subdomains often won't exist as domain names, it's hard
to imagine the situation in which they would exist but be under
someone else's control, or be doing something other than serving
Zulip realms.
2017-11-03 13:38:59 -07:00
rht 38acddee99 zproject: Remove u prefix from strings. 2017-11-02 11:01:47 -07:00
Steve Howell 85f605a978 Fix bug with EmailLogBackend.send_messages.
The `logging.info` call was not properly using string interpolation.
2017-11-01 10:05:13 -07:00
Tim Abbott d27002ba21 landing-page: Replace /about with /team and /history in links. 2017-10-31 12:44:05 -07:00
Brock Whittaker 425738fa24 landing-page: Extract /team/ from /about/.
This extracts and creates a /team/ page using the existing
'team.html' template and applying the why-page styling to it.
2017-10-31 11:37:42 -07:00
Brock Whittaker e86e259cb1 /api/: Add Markdown-based /api-new documentation.
This commit allows for the /api-new/ page to rendered similarly to our
/help pages.  It's based on the old content for /api, but we're not
replacing the old content yet, to give a bit of time to restructure
things reasonably.

Tweaked by eeshangarg and tabbott.
2017-10-30 22:32:43 -07:00
Greg Price 271a9f0da7 settings: Eliminate disused variable EXTERNAL_API_URI.
This was part of the logic to handle EXTERNAL_API_PATH varying.
But also it was already no longer used -- it was only ever passed
into template contexts, as `external_api_uri`, and it'd been
overtaken there by `external_api_uri_subdomain`.

So, update our dev docs to reflect that, and eliminate the variable.
2017-10-30 18:29:29 -07:00
Greg Price 7366a4ca7a settings: Eliminate the disused EXTERNAL_API_PATH.
This setting isn't documented at all, and I believe nobody has used it
since the end of api.zulip.com in 2016.  So we get to complete the
cleanup of this logic.
2017-10-30 18:29:29 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Brock Whittaker 6933d51c0f views/integrations: Change non-generic HelpView to MarkdownDirectoryView.
The HelpView class will render a directory as markdown with an index HTML
page. This however can also be used for other generics and applied to
the API pages as well, so change the class to a generic class and
specify the path templates and names.

Tweaked by tabbott and Eeshan Garg.
2017-10-29 18:15:36 -07:00
Harshit Bansal 56f26c2d6f zproject/settings.py: Correct emoji sprite CSS files. 2017-10-28 10:38:33 -07:00