Commit Graph

595 Commits

Author SHA1 Message Date
acrefoot a36e5d4987 Add get_auth_backends endpoint to API.
We would like to know which kind of authentication backends the server
supports.

This is information you can get from /login, but not in a way easily
parseable by API apps (e.g. the Zulip mobile apps).
2016-06-27 18:30:34 -07:00
Tim Abbott e72f41cdec Remove old prototype data import/export tool.
This prototype from Dropbox Hack Week turned out to be too inefficient
to be used for realms with any significant amount of history, so we're
removing it.

It will be replaced by https://github.com/zulip/zulip/pull/673.
2016-06-27 13:47:08 -07:00
rahuldeve 674def30ee Add support for serving files using API authentication.
Also remove 'get_uploaded_file' view function and the corresponding
old '/user_upload/' url pattern.
2016-06-27 11:00:41 -07:00
rahuldeve 823bf15c6e Add API route for uploading files. 2016-06-27 10:30:58 -07:00
Tim Abbott 199e2df1e3 Split styles/settings.css out of zulip.css. 2016-06-26 08:21:39 -07:00
Vishnu Ks 7d654a26c8 Casper test for realm creation. 2016-06-25 10:50:12 -07:00
Vishnu Ks eb9798027c Move development URLs into dev_urls.py. 2016-06-25 10:50:12 -07:00
Tim Abbott 28b6f1d850 urls: Move non-legacy URLs back out of legacy_urls.py.
Only the old API endpoints were actually legacy URLs.
2016-06-25 10:49:56 -07:00
acrefoot e4ed9195dc Remove rest_dispatch hack and optimize imports.
For a long time, rest_dispatch has had this hack where we have to
create a copy of it in each views file using it, in order to directly
access the globals list in that file.  This removes that hack, instead
making rest_dispatch just use Django's import_string to access the
target method to use.

[tweaked and reorganized from acrefoot's original branch in various
ways by tabbott]
2016-06-24 16:11:03 -07:00
Tim Abbott aebd84cb1b urls: Clarify comment on mobile device credentials. 2016-06-24 16:04:14 -07:00
Tim Abbott 6f06101b73 urls: Add comments on sections of REST URLs. 2016-06-24 16:01:35 -07:00
Tim Abbott 71313fbbdf urls: Move users/me urls further down. 2016-06-24 16:00:32 -07:00
acrefoot 788853a632 urls: Fix over-broad /users/email regexs in urls.py.
/users/me could get captured as an email--our urls shouldn't be so
sentitive to ordering.
2016-06-24 15:53:12 -07:00
Tim Abbott 5235ad5416 urls: Rearrange a few REST URL entries. 2016-06-24 15:51:54 -07:00
Tim Abbott 810d392947 urls: Fix mis-indented URLs entries. 2016-06-24 15:45:26 -07:00
Tim Abbott 16d260d36a urls: Cleanup unnecessary use of 'patterns` calls. 2016-06-24 15:42:04 -07:00
Tim Abbott aa228c9719 urls: Update some comments. 2016-06-24 15:39:14 -07:00
Tim Abbott 6dc416b6c3 urls: Wrap some comments better. 2016-06-24 15:39:14 -07:00
Tim Abbott 92f5e38171 urls: Move the dual-use patterns config up. 2016-06-24 15:39:14 -07:00
Tim Abbott 169810b874 urls: Move development avatars config further down. 2016-06-24 15:39:09 -07:00
acrefoot 0c799dc6b8 Add comments to label important sections of urls.py. 2016-06-24 15:21:36 -07:00
acrefoot b36bc051af Move old-style API endpoints to legacy_urls.py. 2016-06-24 15:14:03 -07:00
acrefoot 5a49c5e280 Denormalize urls in routes file.
Currently we use the deprecated django pattern() prefix pattern.
This make it hard to read the router logic in zproject/urls.py

This commit denormalizes the urls so that they can be read
more easily, at the expense of some verbosity. This also makes it
easier to reorganize urls in that file.

We skip denomalizing rest_dispatch due to its unique complications.
2016-06-24 15:12:51 -07:00
Tim Abbott 1565ee8453 Fix using LDAP backend with Zulip mobile apps.
The recent changes to api_fetch_api_key to receive detailed data via
the "return_data" object did not properly update the LDAP backend to
accept that argument, causing mobile password authentication to not
work with the LDAP backend.
2016-06-21 14:57:54 -07:00
Vishnu Ks ad1c3894d9 Add interface for creating new realms.
This is controlled by settings.OPEN_REALM_CREATION; if that setting is
off, this feature doesn't do anything.
2016-06-17 16:15:28 -07:00
Vishnu Ks 8213ca135a Move default_stream list to settings. 2016-06-17 16:01:38 -07:00
Tim Abbott b5e92f541a Add new documentation on Zulip settings system.
Fixes: #962.
2016-06-17 12:15:15 -07:00
Tim Abbott 6d1d7471e6 Refactor out zproject/dev_settings.py. 2016-06-17 12:15:15 -07:00
kunall17 007eee6061 Add route to fetch emails for mobile passwordless login.
[Tweaked by tabbott to rename API to explicitly support not just
Android]
2016-06-17 11:03:19 -07:00
kunall17 7ea0eaed1c Add passwordless login for mobile app development.
[Tweaked by tabbott to rename API to explicitly support not just
Android].
2016-06-17 10:58:33 -07:00
Tim Abbott e109b50152 Fix alignment of airbrake integration in urls.py. 2016-06-16 15:30:45 -07:00
Tomasz Kolek 76cbe89613 Add IFTTT integration. 2016-06-16 15:30:45 -07:00
Tim Abbott 02e6d267f1 [third] Replace jquery-caret with newer implementation.
Apparently, there are like 5 independently developed jquery-caret
plugins, none of which are great.  The previous one we were using was
last modified in 2010.  This new one comes from
https://github.com/acdvorak/jquery.caret and at least doesn't use
deprecated jQuery syntax and has a repository on GitHub.

This plugin is way larger than it needs to be for what it does, but we
can deal with that later.
2016-06-14 16:36:51 -07:00
Tomasz Kolek 4e51a86ea4 Add updownio integration. 2016-06-14 12:14:07 -07:00
Tomasz Kolek 14d69348d3 Add Airbrake integration. 2016-06-13 20:36:40 -07:00
Tim Abbott 8b9c66aac5 Fix missing $ at end of webhook URL patterns. 2016-06-10 17:12:05 -07:00
rahuldeve fa13582ffb Serve uploaded files through get_uploaded_file in development.
Previously, uploaded files were served:
* With S3UploadBackend, via get_uploaded_file (redirects to S3)
* With LocalUploadBackend in production, via nginx directly
* With LocalUploadBackend in development, via Django's static file server

This changes that last case to use get_uploaded_file in development,
which is a key step towards being able to do proper access control
authorization.

Does not affect production.
2016-06-09 22:58:25 -07:00
rahuldeve e1a2660e70 Refactor realm_id to realm_id_str in url.py. 2016-06-08 09:49:14 -07:00
Tim Abbott bc2961d3ac Refactor file upload routes to their own file. 2016-06-06 16:09:05 -07:00
Umair Khan 0ab6b99cbb Fix backend i18n bug.
`makemessages` escapes the `%` sign in `.po` files, but Jinja2 does
not unescape it while replacing the tranlation strings. In Jinja2,
there is an updated implementation of gettext available called
new-style gettext which handles escaping better; this commit switches
to using that.

Fixes #906.
2016-06-06 09:34:51 -07:00
Tomasz Kolek 8411b2e574 Add Crashlytics integration. 2016-06-04 15:18:42 -07:00
Tomasz Kolek 093e5a96d4 Add Transifex integration.
Fixes: #810.
2016-06-04 14:52:57 -07:00
Vishnu Ks 4fd569f910 Change add_default_stream method from PATCH to PUT.
This is more consistent with our other routes.
2016-06-03 09:52:51 -07:00
Tim Abbott 44929523d6 Harden extraction of AUTH_LDAP_BIND_PASSWORD. 2016-05-31 15:06:43 -07:00
Tim Abbott dac8f7d923 Fix buggy extraction of EMAIL_GATEWAY_PASSWORD.
EMAIL_GATEWAY_PASSWORD has a value of None in the defaults list, so it
will never fail to be in vars().
2016-05-31 15:06:17 -07:00
Umair Khan daf3d51d4b Send 'template_rendered' signal from Jinja2.
Send the signal only under DEBUG=True just like Django.
2016-05-31 14:31:13 +05:00
Tomasz Kolek ff4e92dc3d settings: change tab indentation to space. 2016-05-30 20:16:00 -07:00
Eklavya Sharma ea52fc05ed Make zproject/urls.py pass mypy check. 2016-05-25 15:04:39 -07:00
Umair Khan 82b5d9304b [third] Integrate i18next with Handlebars 2016-05-19 22:58:25 -07:00
Umair Khan f9bbc5d6ff Enable i18n support in URL configuration.
This supports i18n using all of the following:
- I18N urls
- Session
- Cookie
- HTTP header
2016-05-19 08:33:30 -07:00
Vishnu Ks 425363ced4 Fix non determinism bug in casper tests.
This manifested as "ReferenceError: Can't find variable: message_list"
on random tests in the Casper test suite.
2016-05-19 08:27:46 -07:00
Eklavya Sharma 016a2faa23 Make zproject/local_settings.py pass mypy check. 2016-05-18 17:10:18 +05:30
Eklavya Sharma 54759be785 Make zproject/local_settings_template.py pass mypy check. 2016-05-18 17:10:18 +05:30
Umair Khan dfc58b0ed0 Upgrade digest email templates to Jinja2.
Fixes: #780
2016-05-13 01:01:28 +05:00
Tomasz Kolek eeeb4d0c92 Add CircleCI integration.
Fixes: #617.
2016-05-11 21:17:37 -07:00
Tim Abbott 92bec8cfea Merge Zulip 1.3.12 security release. 2016-05-10 11:32:26 -07:00
Tim Abbott be216506a9 Improve api_fetch_api_key error messages.
Previously, api_fetch_api_key would not give clear error messages if
password auth was disabled or the user's realm had been deactivated;
additionally, the account disabled error stopped triggering when we
moved the active account check into the auth decorators.
2016-05-10 09:50:57 -07:00
Tim Abbott 52ddd500f0 Add tests for authentication backends. 2016-05-10 09:50:57 -07:00
Tim Abbott b28b3cd65c CVE-2016-4427: Fix access by deactivated realms/users.
The security model for deactivated users (and users in deactivated
realms) being unable to access the service is intended to work via two
mechanisms:

* All active user sessions are deleted, and all login code paths
  (where a user could get a new session) check whether the user (or
  realm) is inactive before authorizing the request, preventing the
  user from accessing the website and AJAX endpoints.
* All API code paths (which don't require a session) check whether the
  user (and realm) are active.

However, this security model was not implemented correctly.  In
particular, the check for whether a user has an active account in the
login process was done inside the login form's validators, which meant
that authentication mechanisms that did not use the login form
(e.g. Google and REMOTE_USER auth) could succeed in granting a session
even with an inactive account.  The Zulip homepage would still fail to
load because the code for / includes an API call to Tornado authorized
by the user's token that would fail, but this mechanism could allow an
inactive user to access realm data or users to access data in a
deactivated realm.

This fixes the issue by adding explicit checks for inactive users and
inactive realms in all authentication backends (even those that were
already protected by the login form validator).

Mirror dummy users are already inactive, so we can remove the explicit
code around mirror dummy users.

The following commits add a complete set of tests for Zulip's inactive
user and realm security model.
2016-05-10 09:50:48 -07:00
Tim Abbott b869be9301 style: Use 'not in' consistently rather than `not foo in`. 2016-05-09 17:00:10 -07:00
Tim Abbott 9cf18f8535 settings: Fix whitespace errors. 2016-05-09 16:53:12 -07:00
Umair Khan 5359e6b0d4 Convert Zulip to use Jinja2 templates.
This results in a substantial performance improvement for all of
Zulip's backend templates.

Changes in templates:
- Change `block.super` to `super()`.
- Remove `load` tag because Jinja2 doesn't support it.
- Use `minified_js()|safe` instead of `{% minified_js %}`.
- Use `compressed_css()|safe` instead of `{% compressed_css %}`.
- `forloop.first` -> `loop.first`.
- Use `{{ csrf_input }}` instead of `{% csrf_token %}`.
- Use `{# ... #}` instead of `{% comment %}`.
- Use `url()` instead of `{% url %}`.
- Use `_()` instead of `{% trans %}` because in Jinja `trans` is a block tag.
- Use `{% trans %}` instead of `{% blocktrans %}`.
- Use `{% raw %}` instead of `{% verbatim %}`.

Changes in tools:
- Check for `trans` block in `check-templates` instead of `blocktrans`

Changes in backend:
- Create custom `render_to_response` function which takes `request` objects
  instead of `RequestContext` object. There are two reasons to do this:
    1. `RequestContext` is not compatible with Jinja2
    2. `RequestContext` in `render_to_response` is deprecated.
- Add Jinja2 related support files in zproject/jinja2 directory. It
  includes a custom backend and a template renderer, compressors for js
  and css and Jinja2 environment handler.
- Enable `slugify` and `pluralize` filters in Jinja2 environment.

Fixes #620.
2016-05-09 09:55:18 -07:00
Tim Abbott c0d38f42f1 supervisor: Use 127.0.0.1 rather than localhost for tornado.
In theory these should be the same, but in misconfigured environments
(such at Travis CI) where /etc/hosts has multiple entries for
"localhost", 127.0.0.1 is safer than "localhost".
2016-05-08 17:35:50 -07:00
Tim Abbott 73b3f7a26e settings: Fix new-style template LOADERS syntax. 2016-05-07 19:35:16 -07:00
Umair Khan 4620cd8483 settings: Migrate template settings to using TEMPLATES.
This is cleaner and also is necessary preparation for supporting
Jinja2.
2016-05-07 17:29:51 -07:00
Umair Khan a3acd5e8e9 settings: Add LocaleMiddleware to MIDDLEWARE_CLASSES.
This is needed for internationalization.
2016-05-07 17:00:10 -07:00
rahuldeve 899bfb97ee Add tests for managing uploads in S3. 2016-05-02 22:14:47 -07:00
Tim Abbott 52c1e8ac7d Run a local camo server in voyager production environments.
Camo is a caching image proxy, used in Zulip to avoid mixed-content
warnings by proxying HTTP image content over HTTPS.  We've been using
it in zulip.com production for years; this change makes it available
in standalone Zulip deployments.
2016-05-02 17:21:31 -07:00
Ashish Kumar 48be2e33f8 Delete old route for /json/get_public_streams. 2016-04-29 12:57:57 -07:00
Tim Abbott a0430c02ce Allow users who haven't set a password to set one.
Previously, if a user had only authenticated via Google auth, they
would be unable to reset their password in order to set one (which is
needed to setup the mobile apps, for example).
2016-04-28 14:27:43 -07:00
Antek Grzanka 646ea3214a Add Taiga integration. 2016-04-28 13:44:53 -07:00
Tim Abbott c83999fe52 Fix EXTRA_INSTALLED_APPS in development.
In theory, tools like populate_db should probably be in zerver, not
zilencer, but until we migrate them out, we need to include these in
EXTRA_INSTALLED_APPS in development.
2016-04-26 21:55:31 -07:00
Tim Abbott 8905216df5 Automate inclusion of urls.py files for EXTRA_INSTALLED_APPS.
By removing this hardcoding of the list of valid extra apps, we make
it a lot easier to add additional pluggable apps to the Zulip
codebase.
2016-04-26 21:39:39 -07:00
Tim Abbott bf50dd7771 Simplify zilencer urls.py configuration to a single file.
The previous separated-out configuration wasn't helping us, and this
makes it easier to make the extra installed applications pluggable in
the following commits.
2016-04-26 21:35:07 -07:00
Tim Abbott 2b30b670e0 Factor out EXTRA_INSTALLED_APPS setting. 2016-04-26 21:28:51 -07:00
Vladislav Manchev f5e6176aea Add custom realm emoji UI to administration page. 2016-04-26 13:15:54 -07:00
Tim Abbott e4aab64464 Sort webhook integration URL definitions.
This will merge conflict with every new integraiton in flight, which
is unfortunate, but will make there be fewer merge conflicts as people
add new webhooks in the future (currently, every pair of new
integrations conflict because folks are adding them all at the end,
whereas after this change, there will only be merge conflicts when
adding two integrations near each other alphabetically).
2016-04-26 11:49:33 -07:00
Tomasz Kolek 5fbda3a9c1 Add codeship integration. 2016-04-26 09:54:01 -07:00
David Payne 8c62a27769 Add teamcity webhook integration.
This integration relies on the Teamcity "tcWebHooks" plugin which is
available at
https://netwolfuk.wordpress.com/category/teamcity/tcplugins/tcwebhooks/

It posts build fail and success notifications to a stream specified in
the webhook URL.

It uses the name of the build configuration as the topic.

For personal builds, it tries to map the Teamcity username to a Zulip
username, and sends a private message to that person.
2016-04-26 09:45:26 -07:00
Tim Abbott 49799440a4 Replace use of django-guardian with fields on UserProfile.
As documented in https://github.com/zulip/zulip/issues/441, Guardian
has quite poor performance, and in fact almost 50% of the time spent
running the Zulip backend test suite on my laptop was inside Guardian.

As part of this migration, we also clean up the old API_SUPER_USERS
variable used to mark EMAIL_GATEWAY_BOT as an API super user; now that
permission is managed entirely via the database.

When rebasing past this commit, developers will need to do a
`manage.py migrate` in order to apply the migration changes before the
server will run again.

We can't yet remove Guardian from INSTALLED_APPS, requirements.txt,
etc. in this release, because otherwise the reverse migration won't
work.

Fixes #441.
2016-04-20 21:51:52 -07:00
Eklavya Sharma c75c5fb3e1 Use a different uploads directory when running tests. 2016-04-19 16:48:30 -07:00
Tim Abbott 74abd47684 Fix EMAIL_GATEWAY_BOT not being set by default in production.
Previously the DEFAULTS value of None for EMAIL_GATEWAY_BOT was
overriding the initialization code.
2016-04-13 13:19:02 -07:00
Vishnu Ks 35f70e9dac Move 5 legacy global variables to pointer.js.
Move recenter_pointer_on_display, suppress_scroll_pointer_update,
fast_forward_pointer, furthest_read, and server_furthest_read to
a new pointer module in pointer.js.
2016-04-12 10:56:54 -07:00
Ashish 1bf644369f Delete old route for json/update_active_status. 2016-04-11 21:38:23 -07:00
Ashish 78b9f45bf7 Delete old route for json/update_pointer. 2016-04-11 21:38:23 -07:00
Ashish 9429358795 Delete old route for /json/get_profile. 2016-04-11 21:38:23 -07:00
Ashish 86fb7103fa Delete old route for json/change_enter_sends. 2016-04-11 21:38:23 -07:00
Ashish 42fe918138 Delete old route for json/get_old_messages. 2016-04-11 21:38:23 -07:00
Ashish cfefc94200 Delete old route for json/set_alert_words. 2016-04-11 21:38:23 -07:00
Ashish c0a218edfc Delete old route for /json/update_message_flags. 2016-04-11 21:38:23 -07:00
Ashish 8fc8717409 Replace json/set_alert_words with REST style route. 2016-04-11 21:38:22 -07:00
Ashish dac4e58b91 Changes REST API backend route for /json/change_enter_sends. 2016-04-11 21:11:51 -07:00
Anindya Chakravarti f3d03d89b4 Add integration for Yo App.
[includes some small tweaks by tabbott]
2016-04-08 11:02:10 -07:00
Tomasz Kolek 44ed9da7f0 Add pingdom integration. 2016-04-08 10:36:29 -07:00
Ashish 909b0635c8 Add management command to test sending email.
Fixes: #622.

[With cleanups/doc tweaks by tabbott]
2016-04-03 16:50:16 -07:00
Tim Abbott e9637a545f Rename zerver.handlers to zerver.logging_handlers for clarity. 2016-03-30 21:48:38 -07:00
goelakash 25a75bcefe Change LOCAL_UPLOAD_DIR to 'uploads' in development.
Fixes #488.
2016-03-29 11:12:29 -07:00
Luke Faraone 5c28b0340a Don't show Zulip.com terms on other sites
While we already don't link to /terms anywhere on the site, they can still be
accessed if you navigate to /terms directly. Now, those routes will only be
exported on the Zulip.com service.

We should ideally provide a mechanism for deployments to specify their own
terms without modifying source code; in the interim, sites that have already
customised the provided Zulip.com terms can simply carry a patch reverting this
commit.
2016-03-21 05:46:28 +00:00
Eklavya Sharma 9e3c3e14f5 Partially apply Python 3 libmodernize.fixes.fix_dict_six.
Refer to #256
2016-03-19 15:52:58 -07:00
Tomasz Kolek 3e3a224607 Moved pagerduty webhook into its own file pagerduty.py 2016-03-14 20:44:50 -07:00