Commit Graph

108 Commits

Author SHA1 Message Date
neiljp (Neil Pilgrim) 2ed6da77c7 mypy: Rewrite some middleware annotations to use ViewFuncT. 2018-03-17 23:25:05 +00:00
Greg Price 53c57cf002 errors: Include request info on error mails for JSON routes too.
When our code raises an exception and Django converts it to a 500
response (in django.core.handlers.exception.handle_uncaught_exception),
it attaches the request to the log record, and we use this in our
AdminNotifyHandler to include data like the user and the URL path
in the error email sent to admins.

On this line, when our code raises an exception but we've decided (in
`TagRequests`) to format any errors as JSON errors, we suppress the
exception so we have to generate the log record ourselves.  Attach the
request here, just like Django does when we let it do the job.

This still isn't an awesome solution, in that there are lots of other
places where we call `logging.error` or `logging.exception` while
inside a request; this just covers one of them.  This is one of the
most common, though, so it's a start.
2018-03-01 15:12:32 -08:00
Callum Fraser aa9567ce37 mypy: Use Python 3 type syntax in zerver/middleware.py. 2017-12-11 18:43:24 -08:00
rht a1cc720860 zerver: Use Python 3 syntax for typing.
Tweaked by tabbott to fix some minor whitespace errors.
2017-11-28 16:49:36 -08:00
Greg Price b6cc21b438 debug: Add facility to dump tracemalloc snapshots.
Originally this used signals, namely SIGRTMIN.  But in prod, the
signal handler never fired; I debugged fruitlessly for a while, and
suspect uwsgi was foiling it in a mysterious way (which is kind of
the only way uwsgi does anything.)

So, we listen on a socket.  Bit more code, and a bit trickier to
invoke, but it works.

This was developed for the investigation of memory-bloating on
chat.zulip.org that led to a331b4f64 "Optimize query_all_subs_by_stream()".

For usage instructions, see docstring.
2017-11-28 15:52:07 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
Tim Abbott 10ab9410c9 python: Sort imports in easy files in zerver/. 2017-11-15 15:50:28 -08:00
derAnfaenger 3ac09b3e9b queue processors: Add coverage for SlowQueryWorker. 2017-11-09 15:20:40 -08:00
rht 5ee40bf718 Remove usage of six.moves.binary_type. 2017-11-09 10:00:00 -08:00
Felix Yan aea33fc738 Fix a comment typo in zerver/middleware.py. 2017-10-30 10:36:35 -07:00
derAnfaenger 1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
Greg Price 093bae4bc5 subdomains: Fix some implicit uses of "" for the root subdomain.
These are just instances that jumped out at me while working on the
subdomains code, mostly while grepping for get_subdomain call sites.
I haven't attempted a comprehensive search, and there are likely
still others left.
2017-10-26 10:29:17 -07:00
Tim Abbott 1ab2ca5986 subdomains: Extract zerver.lib.subdomains library.
These never really belonged with the rest of zerver.lib.utils.py, and
having a separate library makes it easier to enforce full test
coverage.
2017-10-18 22:27:48 -07:00
Alena Volkova 5515a075ec urls: Move the report endpoints to be API-style routes. 2017-10-17 22:05:56 -07:00
Tim Abbott 46485322eb middleware: Remove logic for redirecting to zulipdev.com domains.
We originally wrote this because when testing subdomains, you wanted
to be sure you were actually testing subdomains.  Now that subdomains
is the default, doesn't seem to actually be a good reason why we
should need this.
2017-10-05 23:21:02 -07:00
Tim Abbott 40c59f2878 middleware: Fix losing sub-URL when pushing to zulipdev.com.
Previously, this would always send one to homepage, making visiting
the /help/ documentation in the development environment using the
localhost URL unpleasant.

While this fixes the proximal bug, it's not clear to me that we need
this redirect logic at all, so I'm going to try removing it soon.
2017-10-05 16:36:34 -07:00
Tim Abbott 1d72629dc4 subdomains: Hardcode REALMS_HAVE_SUBDOMAINS=True. 2017-10-02 16:42:43 -07:00
rht 2949d1c1e8 zerver: Remove the rest of absolute_import. 2017-09-27 10:02:39 -07:00
neiljp (Neil Pilgrim) bb83742906 mypy: Correct 2 type annotations in zerver/middleware.py. 2017-08-15 17:50:18 -07:00
neiljp (Neil Pilgrim) e772e89fe2 mypy: Add None return path for RateLimitMiddleware.process_exception(). 2017-08-03 11:03:14 -07:00
Umair Khan 9e33917d25 rate_limiter: Upgrade max_api_calls to generic API. 2017-08-02 18:01:39 -07:00
Greg Price 192ec7c0f6 middleware: Use a proper error code on CSRF failure.
This allows us to reliably parse the error in code, rather than
attempt to parse the error text.  Because the error text gets
translated into the user's language, this error-handling path
wasn't functioning at all for users using Zulip in any of the
seven non-English languages for which we had a translation for
this string.

Together with 709c3b50f which fixed a similar issue in a
different error-handling path, this fixes #5598.
2017-07-25 14:02:12 -07:00
Greg Price 5de7c4f2af JsonErrorHandler: Take advantage of the new JsonableError structured data. 2017-07-24 16:41:22 -07:00
Greg Price 9faa44af60 JsonableError: Optionally carry error codes and structured data.
This provides the main infrastructure for fixing #5598.  From here,
it's a matter of on the one hand upgrading exception handlers -- the
many except-blocks in the codebase that look for JsonableError -- to
look beyond the string `msg` and pass on the machine-readable full
error information to their various downstream recipients, and on the
other hand adjusting places where we raise errors to take advantage
of this mechanism to give the errors structured details.

In an ideal future, I think all exception handlers that look (or
should look) for a JsonableError would use its contents in structured
form, never mentioning `msg`; but the majority of error sites might
continue to just instantiate JsonableError with a string message.  The
latter is the simplest thing to do, and probably most error types will
never have code looking for them specifically.

Because the new API refactors the `to_json_error_msg` method which was
designed for subclasses to override, update the 4 subclasses that did
so to take full advantage of the new API instead.
2017-07-24 16:41:22 -07:00
Greg Price 6dfb46dc08 JsonableError: Rename `status_code` and rely more on its default.
With #5598 there will soon be an application-level error code
optionally associated with a `JsonableError`, so rename this
field to make clear that it specifically refers to an
HTTP status code.

Also take this opportunity to eliminate most of the places
that refer to it, which only do so to repeat the default value.
2017-07-24 16:41:22 -07:00
Greg Price a5597e91a1 exceptions: Move zerver/exceptions.py under zerver/lib/.
Seems like a more appropriate place for it.  Preparation for
moving a bit more into that file.
2017-07-24 16:41:22 -07:00
Greg Price ff5013c619 JsonableError: Add types, and eliminate duck-typing.
In order to benefit from the modern conveniences of type-checking,
add concrete, non-Any types to the interface for JsonableError.

Relatedly, there's no need at this point to duck-type things at
the places where we receive a JsonableError and try to use it.
Simplify those by using straightforward standard typing.
2017-07-24 16:41:22 -07:00
Umair Khan 95fc16d90d Django 1.11: MIDDLEWARE_CLASSES setting is deprecated.
Django provides MiddlewareMixin to upgrade old-style middlewares. See
https://docs.djangoproject.com/en/1.11/topics/http/middleware/#upgrading-middleware
2017-06-13 15:04:04 -07:00
Tim Abbott 8e978df957 mypy: Fix missing import from recent mypy merge. 2017-05-25 16:22:19 -07:00
Ethan d2e72b0082 mypy: correct process response type. 2017-05-25 15:41:51 -07:00
Ethan b6e7e36c86 mypy: correct from int to str in rate limiting. 2017-05-25 15:41:49 -07:00
Tim Abbott 17f87cfc9e mypy: Fix missing Optional on process_exception. 2017-05-23 17:44:30 -07:00
Lukasz Prasol 5eaccc550a rate_limit: Make retry-after data machine-readable.
Fixes #4831.
2017-05-22 17:35:12 -07:00
hollywoodno 75d9630258 Add notifications on new logins to Zulip.
This adds helpful email notifications for users who just logged into a
Zulip server, as a security protection against accounts being hacked.

Text tweaked by tabbott.

Fixes #2182.
2017-03-25 16:50:52 -07:00
Tim Abbott c2c02ea4da middleware: Fix typo in render_to_response migration.
This fixes a 500 on the invalid realm page.
2017-03-21 07:30:28 -07:00
Umair Khan 4442703011 jinja2: No need for custom render_to_response.
Django 1.10 has changed the implementation of this function to
match our custom implementation; in addition to this, we prefer
render().

Fixes #1914 via #4093.
2017-03-17 13:57:34 -07:00
Umair Khan 97639e5e48 middleware: Change render_to_response to render.
Related to #4093
2017-03-17 13:52:59 -07:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Tim Abbott 32bfebeb7a mypy: Fix inconsistencies in use of *args/**kwargs. 2017-02-18 18:39:44 -08:00
Tim Abbott b81fd407e8 mypy: Fix several Optional typing errors. 2017-02-10 23:53:44 -08:00
Tim Abbott de3e96162e middleware: Fix recursive DisallowedHost exceptions. 2017-01-29 20:26:58 -08:00
Tim Abbott 22d1aa396b lint: Clean up W503 PEP-8 warning. 2017-01-23 20:50:04 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Juan Verhook cfa9c2eaf2 mypy: Update zerver directory to use Text 2016-12-29 09:12:15 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Rafid Aslam 41bd88d5ed pep8: Fix E301 pep8 violations.
Fix "E301: expected (1 or 2) blank line" pep8 violations.
2016-11-29 08:51:44 -08:00
Rishi Gupta 4a74301a62 models.py: Replace resolve_subdomain_to_realm with get_realm_by_string_id.
No change in functionality.
2016-11-03 13:59:11 -07:00
hackerkid ea39fb2556 Add option for hosting each realm on its own subdomain.
This adds support for running a Zulip production server with each
realm on its own unique subdomain, e.g. https://realm_name.example.com.

This patch includes a ton of important features:
* Configuring the Zulip sesion middleware to issue cookier correctly
  for the subdomains case.
* Throwing an error if the user tries to visit an invalid subdomain.
* Runs a portion of the Casper tests with REALMS_HAVE_SUBDOMAINS
  enabled to test the subdomain signup process.
* Updating our integrations documentation to refer to the current subdomain.
* Enforces that users can only login to the subdomain of their realm
  (but does not restrict the API; that will be tightened in a future commit).

Note that toggling settings.REALMS_HAVE_SUBDOMAINS on a live server is
not supported without manual intervention (the main problem will be
adding "subdomain" values for all the existing realms).

[substantially modified by tabbott as part of merging]
2016-09-27 23:24:14 -07:00