Commit Graph

2157 Commits

Author SHA1 Message Date
Oren Leaffer 2916fb30cb bugdown: add some type annotations.
Had to add some "type: ignore" because the pattern used in match
doesn't affect the type returned. A fix for this issue has been pushed
to typeshed - https://github.com/python/typeshed/pull/244
2016-06-03 17:03:52 -07:00
medullaskyline 2213a9f41f Annotate zerver/lib/cache_helpers.py 2016-06-03 16:34:30 -07:00
Conrad Dean 7f61a5e862 Add type annotations to zerver.lib.test_helpers. 2016-06-03 12:17:26 -07:00
Max a6e60419c4 Add types to confirmation/views.py and zerver/tornadoviews.py 2016-06-03 11:26:30 -07:00
Vishnu Ks 4fd569f910 Change add_default_stream method from PATCH to PUT.
This is more consistent with our other routes.
2016-06-03 09:52:51 -07:00
Nathan Florea 04c71fadc6 More removal of mutable default arguments.
I've left a few that clearly aren't being passed and aren't being mutated, but
I think I've gotten the rest of them.
2016-06-03 09:17:04 -07:00
Nathan Florea 5fe9076631 Remove some mutable default arguments.
These ones don't fix any bugs, because the mutable arg is never passed
outside of the callable or mutated.  But it's good practice to not use
them in case those invariants are changed in the future.
2016-06-03 09:16:56 -07:00
Ashish Kumar 9b990e3bd0 Type annotation of zerver/views/alert_words.
[Tweaked by tabbott to annotate the REQ variables the new way]
2016-06-03 08:07:58 -07:00
Ashish Kumar 31bf6b8259 Type annotation of zerver/models.py
[Substantially revised by tabbott]

This probably still has some bugs in it, but having mostly complete
annotations for models.py will help a lot for the annotations folks
are adding to other files.
2016-06-02 23:28:34 -07:00
Tim Abbott 37015fd7c5 Run mypy on zerver/lib/test_auth_backends.py. 2016-06-02 23:01:15 -07:00
Tim Abbott 8cef9675c8 Run mypy on zerver/lib/test_events.py. 2016-06-02 23:00:04 -07:00
Tim Abbott e6d2b0cdbc Run mypy on zerver/lib/test_unread.py. 2016-06-02 22:59:00 -07:00
Tim Abbott f3b07ee9aa Run mypy on zerver/lib/test_subs.py. 2016-06-02 22:57:07 -07:00
Oren Leaffer c2ce5119c6 Annotate zerver.views.tutorial. 2016-06-02 18:49:27 -07:00
Rachel Kelly df36216914 Change instances of 'coworkers' to 'users'.
In order to genericize use of Zulip outside companies,
    all instances of coworkers have been changed to users.
    NOTABLE EXCEPTION: When the Zulip instance is domain-
    locked, the reference to coworkers remains.  The reason
    for this is twofold: first, the majority of Zulip instances
    which require a particular domain will be locked to a
    company, and second, the template variable for the domain
    necessary should be added to the alert so it is clear
    to the user what the domain needs to be for access.

Fixes: #861.
2016-06-02 16:05:27 -07:00
Pei-Wei Wu 8d2733ae8c Add mypy type annotations to zerver/views/streams.py. 2016-06-02 15:44:43 -07:00
Ashley Dunn 7826aa7e7f Type annotation of zerver/views/realm_emoji. 2016-06-02 14:01:28 -07:00
Ashish Kumar cad342aff6 Correct annotation of generic_bulk_cached_fetch in zerver/lib/cache.py.
Previously, object_ids was tagged as an int, but it is called from
models.py with a string, so we make it an Any.
2016-06-01 14:00:49 -07:00
Umair Khan 08fbd57245 [i18n] Make error messages translatable.
Make all strings passing through `json_error` and `JsonableError`
translatable.

Fixes #727
2016-05-31 07:40:42 -07:00
Tim Abbott ab2d325a08 Update production default streams to be less engineering-centric. 2016-05-31 07:38:25 -07:00
Vishnu Ks 100d885f23 Change default announcement stream to announce.
Fixes #788.
2016-05-31 07:38:07 -07:00
Tim Abbott 960144a49e Desupport using uninstantiated REQ with has_request_variables.
This makes life difficult for doing static type annotations, and
didn't make the code look that much better anyway.
2016-05-31 07:31:15 -07:00
Tim Abbott 41336f3782 lint-all: Check for use of '== None'. 2016-05-31 07:02:04 -07:00
Umair Khan 9a57176ad6 Do shallow testing of backend templates.
Just render the templates without the actual workflow to see if they
don't return a 500 error; this lets us catch various classes of
template bugs automatically.

Fixes #784.
2016-05-31 05:42:17 -07:00
Umair Khan c884559ec6 Show templates rendered report.
Add two options to the `test-backend` script:

1. verbose
    If given the `test-backend` script will give detailed output.
2. no-shallow
    Default value is False. If given the `test-backend` script will
    fail if it finds a template which is shallow tested.
2016-05-31 16:46:11 +05:00
Tomasz Kolek 8e144a1f57 Add zip and absoulte_import to pass py3k test. 2016-05-30 22:28:05 +02:00
Tim Abbott baec0f12cf Add a proper annotation for REQ in streams.py. 2016-05-30 11:41:16 -07:00
Reid Barton 8c6afac7cd Add a stub file for request.py.
This stub file allows us to annotate view functions using the actual
types present in the bodies of the functions, rather than everything
having the type REQ.
2016-05-30 11:28:53 -07:00
Tim Abbott 572c69f3c2 Move REQ and friends to their own module. 2016-05-30 11:24:17 -07:00
Eklavya Sharma 48e7e1a2a1 zerver/lib/actions.py: Rename stream_name to stream.
In function bulk_add_subscriptions, some variables were named
`stream_name` but their type is Stream, not a string.  Rename
those variables to `stream`.
2016-05-30 09:52:59 -07:00
Eklavya Sharma 94e4b39112 Replace python2.7 by python everywhere. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 1bb6a0db4c Annotate zerver/lib/actions.py. 2016-05-29 04:26:17 -07:00
Eklavya Sharma 2308107805 zerver/lib/actions.py: Use unicode literals.
Convert some strings literals to unicode strings by prefixing with `u`.
2016-05-29 04:26:17 -07:00
Eklavya Sharma b74f603682 zerver/lib/actions.py: Rename variables and add/edit comments. 2016-05-29 04:26:17 -07:00
Eklavya Sharma efab224bd1 zerver/lib/actions.py: Remove unneeded `return {}` statements. 2016-05-29 04:26:17 -07:00
Eklavya Sharma a2b48f05e5 zerver/lib/actions.py: Fix return values. 2016-05-29 04:26:17 -07:00
Eklavya Sharma 1ea6171179 Fix an annotation in zerver/lib/cache.py.
This is done to make annotations in zerver/lib/actions.py work correctly.
2016-05-25 15:11:48 -07:00
Eklavya Sharma 30892b2f99 Make makemessages.py pass mypy check. 2016-05-25 15:04:39 -07:00
Eklavya Sharma 1c04560def Re-enable pyflakes in linter and remove python 3 pyflakes errors. 2016-05-25 19:25:13 +05:30
Eklavya Sharma 459c6640bf Fix type annotations in zerver/lib/alert_words.py. 2016-05-24 14:12:11 -07:00
Eklavya Sharma 95d059bfb3 Fix typo in zerver/lib/actions.py. 2016-05-24 14:12:11 -07:00
Eklavya Sharma 508a080e08 do_change_bot_type: Add update_fields to user_profile.save(). 2016-05-24 13:21:44 -07:00
Umair Khan 82b5d9304b [third] Integrate i18next with Handlebars 2016-05-19 22:58:25 -07:00
Tomasz Kolek 8c18b8947f Add bot_type field to UserProfile.
This is intended to support creating different types of bots with
potentially limited permissions.
2016-05-19 22:37:37 -07:00
Umair Khan f9bbc5d6ff Enable i18n support in URL configuration.
This supports i18n using all of the following:
- I18N urls
- Session
- Cookie
- HTTP header
2016-05-19 08:33:30 -07:00
Tim Abbott b01196db86 to_log_dict: Add sender_id to logged fields. 2016-05-18 23:02:43 -07:00
Aristeidis Fkiaras 3ee210d9e8 Add setting to only allow admins create new streams.
Fixes: #691.

Thanks to Preston Hansen for work on this feature!
2016-05-18 18:53:13 -07:00
Tim Abbott e781136132 Fix subscribing to existing streams when can_create_streams=False.
Previously, a user with can_create_streams=False would be incorrectly
unable to subscribe to streams, whether the streams previously existed
or not.
2016-05-18 18:47:24 -07:00
Tomasz Kolek c4254497b2 Add WebhookTestCase abstract class for writing webhook tests.
This cuts a ton of code duplication and semi-duplication between the
webhook tests, and thus should make it a lot easier to write new ones.
2016-05-18 14:37:31 -07:00
Eklavya Sharma 98afe000ee Make zerver/lib/statistics.py pass mypy check. 2016-05-18 17:10:18 +05:30
Eklavya Sharma 0dcd8b387d Make zerver/lib/bugdown/fenced_code.py pass mypy check. 2016-05-18 17:10:17 +05:30
Eklavya Sharma 46757f07bf Make zerver/lib/actions.py pass mypy check. 2016-05-18 17:10:17 +05:30
Eklavya Sharma 16067b7013 Make zerver/views/webhooks/jira.py pass mypy check. 2016-05-18 17:10:17 +05:30
Tim Abbott c3985520e5 webhooks: Remove unnecessary get_client imports. 2016-05-13 12:25:12 -07:00
Tomasz Kolek db7ea8b484 Move getting client to api_key_only_webhook_view.
This decreases the amount of convention developers need to understand
in order to write a new webhook integration.
2016-05-13 12:22:38 -07:00
Umair Khan dfc58b0ed0 Upgrade digest email templates to Jinja2.
Fixes: #780
2016-05-13 01:01:28 +05:00
Tomasz Kolek eeeb4d0c92 Add CircleCI integration.
Fixes: #617.
2016-05-11 21:17:37 -07:00
Tim Abbott 2409ac9b2f cache: Add type annotations to active_*_dict_fields. 2016-05-10 11:48:03 -07:00
Tim Abbott f2aee961e1 test_auth_backends: Fix unused variables. 2016-05-10 11:46:39 -07:00
Tim Abbott 92bec8cfea Merge Zulip 1.3.12 security release. 2016-05-10 11:32:26 -07:00
Tim Abbott 9b65464b6b logout_all_users: Add option to logout deactivated users. 2016-05-10 09:50:57 -07:00
Tim Abbott 393159bbd8 queue: Disable RabbitMQ heartbeat in BlockingConnection.
Fixes #741.
2016-05-10 09:50:57 -07:00
Tim Abbott d82e44ecd0 queue: Refactor Pika credentials code to be a bit cleaner. 2016-05-10 09:50:57 -07:00
Tim Abbott 620debc5fd Change PrincipalError to return status code 403 by default. 2016-05-10 09:50:57 -07:00
Tim Abbott 85c64c9f93 zulip_login_required: Add checks for active users and realms.
Like the recent change blocking JSON endpoints for deactivated users
and users in deactivated realms, this change is a hardening
improvement.  Those users should be unable to get an active session
anyway, but if somehow one is leaked, this means they won't be able to
access any user data.
2016-05-10 09:50:57 -07:00
Tim Abbott be216506a9 Improve api_fetch_api_key error messages.
Previously, api_fetch_api_key would not give clear error messages if
password auth was disabled or the user's realm had been deactivated;
additionally, the account disabled error stopped triggering when we
moved the active account check into the auth decorators.
2016-05-10 09:50:57 -07:00
Tim Abbott 52ddd500f0 Add tests for authentication backends. 2016-05-10 09:50:57 -07:00
Tim Abbott 38c82083de Add test suite for deactivated users. 2016-05-10 09:50:57 -07:00
Tim Abbott df7466e893 Add test suite for deactivate realms. 2016-05-10 09:50:57 -07:00
Tim Abbott 76814f37a3 decorators: Block access to JSON endpoints for deactivated users.
While in theory users should be unable to get a valid session in order
to access these endpoints in the first place, this provides an extra
layer of hardering to prevent a deactivated user with a session from
accessing data via the old-style JSON API.
2016-05-10 09:50:57 -07:00
Tim Abbott b28b3cd65c CVE-2016-4427: Fix access by deactivated realms/users.
The security model for deactivated users (and users in deactivated
realms) being unable to access the service is intended to work via two
mechanisms:

* All active user sessions are deleted, and all login code paths
  (where a user could get a new session) check whether the user (or
  realm) is inactive before authorizing the request, preventing the
  user from accessing the website and AJAX endpoints.
* All API code paths (which don't require a session) check whether the
  user (and realm) are active.

However, this security model was not implemented correctly.  In
particular, the check for whether a user has an active account in the
login process was done inside the login form's validators, which meant
that authentication mechanisms that did not use the login form
(e.g. Google and REMOTE_USER auth) could succeed in granting a session
even with an inactive account.  The Zulip homepage would still fail to
load because the code for / includes an API call to Tornado authorized
by the user's token that would fail, but this mechanism could allow an
inactive user to access realm data or users to access data in a
deactivated realm.

This fixes the issue by adding explicit checks for inactive users and
inactive realms in all authentication backends (even those that were
already protected by the login form validator).

Mirror dummy users are already inactive, so we can remove the explicit
code around mirror dummy users.

The following commits add a complete set of tests for Zulip's inactive
user and realm security model.
2016-05-10 09:50:48 -07:00
Tim Abbott b31ac1eca9 Fix users in deactivated realms sending webhook messages.
In a deactivated realm, webhooks would still successfully send
messages, since there was no check for whether the realm was active in
api_key_only_webhook_view.
2016-05-10 09:50:48 -07:00
Tim Abbott 9da73b22d3 assert_json_error_contains: Support passing a status code.
Previously this test helper function hardcoded 400.
2016-05-10 09:50:48 -07:00
Tim Abbott 3cde06ea33 Add support for setting HTTP status codes in JsonableError. 2016-05-10 09:50:48 -07:00
Tim Abbott b4ccca300b Add tests for whether API keys appear in initial state data. 2016-05-10 09:50:47 -07:00
Tim Abbott 07fc47f953 CVE-2016-4426: Fix non-admin users having access to all bot API keys.
Long ago, there was work on an experimental integration model where
every user in a realm would have administrative control over all bots,
with the goal of simplifying the process of setting up communally
administered bots for smaller teams.  While that new model was never
fully implemented (and thus never setup as an option), an error in
that original implementation meant that the data on all bots in a
realm, including their API keys, was sent to the browsers of users via
the `realm_bots` variable in `page_params`.  The data wasn't displayed
in the UI for non-admin users, but was available via e.g. the
javascript console.

This commit updates this behavior to only send sensitive bot data like
API keys to the owner of the bot (and realm admins).

We may in the future implement a model simplifying communally
administered integrations, but if we do that, those bots should be
limited in their capabilities (e.g. only able to send webhook
messages).

This bug has been present since Zulip was released as open source.
2016-05-10 09:50:02 -07:00
Tim Abbott b869be9301 style: Use 'not in' consistently rather than `not foo in`. 2016-05-09 17:00:10 -07:00
Tim Abbott 43f167849b queue: Disable RabbitMQ heartbeat in BlockingConnection.
Fixes #741.
2016-05-09 10:23:28 -07:00
Tim Abbott 0c42fc2f8f queue: Refactor Pika credentials code to be a bit cleaner. 2016-05-09 10:23:28 -07:00
Tim Abbott 0161d2fddd Cleanup guardian-based complexity in get_realm_user_dicts.
The old code for this lookup was unnecessarily complicated because we
were working around Guardian, where the `is_realm_admin` check was
extremely expensive.
2016-05-09 10:12:35 -07:00
Tim Abbott 2a2cbd60c3 cache: Fix fragile active_bot_dicts_in_realm caching model.
The issue here is similar to that in the previous commit.
2016-05-09 10:12:35 -07:00
Tim Abbott fbc7e977ac cache: Fix fragile active_user_dicts_in_realm caching model.
Previously we relied on having two matching list of fields for the
get_active_user_dicts_in_realm, one in the actual code and the other
in the caching system.  By unifying these lists to have a single
source, we eliminate a class of caching bugs we might otherwise
regularly introduce.
2016-05-09 10:12:35 -07:00
Tim Abbott f02571202a EventsRegisterTest: display full error diffs. 2016-05-09 10:12:35 -07:00
Umair Khan 5359e6b0d4 Convert Zulip to use Jinja2 templates.
This results in a substantial performance improvement for all of
Zulip's backend templates.

Changes in templates:
- Change `block.super` to `super()`.
- Remove `load` tag because Jinja2 doesn't support it.
- Use `minified_js()|safe` instead of `{% minified_js %}`.
- Use `compressed_css()|safe` instead of `{% compressed_css %}`.
- `forloop.first` -> `loop.first`.
- Use `{{ csrf_input }}` instead of `{% csrf_token %}`.
- Use `{# ... #}` instead of `{% comment %}`.
- Use `url()` instead of `{% url %}`.
- Use `_()` instead of `{% trans %}` because in Jinja `trans` is a block tag.
- Use `{% trans %}` instead of `{% blocktrans %}`.
- Use `{% raw %}` instead of `{% verbatim %}`.

Changes in tools:
- Check for `trans` block in `check-templates` instead of `blocktrans`

Changes in backend:
- Create custom `render_to_response` function which takes `request` objects
  instead of `RequestContext` object. There are two reasons to do this:
    1. `RequestContext` is not compatible with Jinja2
    2. `RequestContext` in `render_to_response` is deprecated.
- Add Jinja2 related support files in zproject/jinja2 directory. It
  includes a custom backend and a template renderer, compressors for js
  and css and Jinja2 environment handler.
- Enable `slugify` and `pluralize` filters in Jinja2 environment.

Fixes #620.
2016-05-09 09:55:18 -07:00
Preston Hansen 635828069f Add feature to mark all in stream/topic as read with mouse.
Fixes #736.
2016-05-08 09:02:46 -07:00
Tim Abbott 6139e8948a travis: User REQ framework for extracting JSON payload. 2016-05-07 11:54:14 -07:00
Tim Abbott 678adc2048 webhooks: Use REQ more consistently in stream name parsing.
To avoid the potential for introducing regressions here, we carefully
pass a default to REQ or not based on how the existing webhook's
parsing code worked.  In the longer term, we'll want to make the
behavior consistent.
2016-05-07 11:54:14 -07:00
Tomasz Kolek c2de38239e Add payload validation to has_request_variables and REQ tasks.
[with tweaks by tabbott]
2016-05-07 11:54:09 -07:00
Tim Abbott c1a680e2a9 rate_limiter: Fix misplaced type annotation and cleanup code.
You don't put type annotations on return values.
2016-05-06 13:38:12 -07:00
Tim Abbott 4219a6779f socket: Initialize client_id to None by default.
This fixes an exception where client_id was never set in an error code
path.  It shouldn't be needed, but I think this makes the code clearer
and this will help in debugging the actual problem.

Related to #753.
2016-05-05 14:49:26 -07:00
Tim Abbott 191201bd10 Fix unnecessary whitespace between % and (. 2016-05-04 14:22:52 -07:00
Tim Abbott 54022ac204 Fix unnecessary whitespace between , and ). 2016-05-04 14:16:53 -07:00
Tim Abbott ae6037668a test_runner: Add debugging tips for missing test modules. 2016-05-03 12:33:47 -07:00
Tim Abbott bab267f332 test_runner: Improve error handling when importing test files.
The error message for a test file that doesn't import properly was
previously pretty difficult to understand and it wasn't clear how to
debug the issue.
2016-05-03 12:17:10 -07:00
rahuldeve 899bfb97ee Add tests for managing uploads in S3. 2016-05-02 22:14:47 -07:00
rahuldeve 01fb6c77a2 Add test for managing files in Local Storage. 2016-05-02 22:14:47 -07:00
rahuldeve dde832b158 Add Attachment model to keep track of uploads.
This commit adds the capability to keep track and remove uploaded
files.  Unclaimed attachments are files that have been uploaded to the
server but are not referred in any messages.  A management command to
remove old unclaimed files after a week is also included.

Tests for getting the file referred in messages are also included.
2016-05-02 22:14:47 -07:00
Tim Abbott 762a3188ee Fix missing whitespace after # in comments. 2016-05-02 22:10:47 -07:00
Tim Abbott d9e4968d6f Increase maximum URL length for RealmEmoji to 1000.
The default of 200 was shorter than the Camo URLs use by Zulip.
2016-05-02 19:02:56 -07:00