Commit Graph

227 Commits

Author SHA1 Message Date
Tim Abbott 17f87cfc9e mypy: Fix missing Optional on process_exception. 2017-05-23 17:44:30 -07:00
Lukasz Prasol 5eaccc550a rate_limit: Make retry-after data machine-readable.
Fixes #4831.
2017-05-22 17:35:12 -07:00
hollywoodno 75d9630258 Add notifications on new logins to Zulip.
This adds helpful email notifications for users who just logged into a
Zulip server, as a security protection against accounts being hacked.

Text tweaked by tabbott.

Fixes #2182.
2017-03-25 16:50:52 -07:00
Tim Abbott c2c02ea4da middleware: Fix typo in render_to_response migration.
This fixes a 500 on the invalid realm page.
2017-03-21 07:30:28 -07:00
Umair Khan 4442703011 jinja2: No need for custom render_to_response.
Django 1.10 has changed the implementation of this function to
match our custom implementation; in addition to this, we prefer
render().

Fixes #1914 via #4093.
2017-03-17 13:57:34 -07:00
Umair Khan 97639e5e48 middleware: Change render_to_response to render.
Related to #4093
2017-03-17 13:52:59 -07:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Tim Abbott 32bfebeb7a mypy: Fix inconsistencies in use of *args/**kwargs. 2017-02-18 18:39:44 -08:00
Tim Abbott b81fd407e8 mypy: Fix several Optional typing errors. 2017-02-10 23:53:44 -08:00
Tim Abbott de3e96162e middleware: Fix recursive DisallowedHost exceptions. 2017-01-29 20:26:58 -08:00
Tim Abbott 22d1aa396b lint: Clean up W503 PEP-8 warning. 2017-01-23 20:50:04 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Juan Verhook cfa9c2eaf2 mypy: Update zerver directory to use Text 2016-12-29 09:12:15 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Rafid Aslam 41bd88d5ed pep8: Fix E301 pep8 violations.
Fix "E301: expected (1 or 2) blank line" pep8 violations.
2016-11-29 08:51:44 -08:00
Rishi Gupta 4a74301a62 models.py: Replace resolve_subdomain_to_realm with get_realm_by_string_id.
No change in functionality.
2016-11-03 13:59:11 -07:00
hackerkid ea39fb2556 Add option for hosting each realm on its own subdomain.
This adds support for running a Zulip production server with each
realm on its own unique subdomain, e.g. https://realm_name.example.com.

This patch includes a ton of important features:
* Configuring the Zulip sesion middleware to issue cookier correctly
  for the subdomains case.
* Throwing an error if the user tries to visit an invalid subdomain.
* Runs a portion of the Casper tests with REALMS_HAVE_SUBDOMAINS
  enabled to test the subdomain signup process.
* Updating our integrations documentation to refer to the current subdomain.
* Enforces that users can only login to the subdomain of their realm
  (but does not restrict the API; that will be tightened in a future commit).

Note that toggling settings.REALMS_HAVE_SUBDOMAINS on a live server is
not supported without manual intervention (the main problem will be
adding "subdomain" values for all the existing realms).

[substantially modified by tabbott as part of merging]
2016-09-27 23:24:14 -07:00
Tim Abbott 647cead0d1 slow queries: Include full log line in slow query log.
The extra data is useful, and I think this won't make the lines annoying long.
2016-07-12 19:12:49 -07:00
Eklavya Sharma 9161ddaee0 zerver/middleware.py: Handle binary data in errors.
In write_log_line, error_content can be binary_type and
error_content_iter can be a Sequence of binary_type.  Handle
this this in a python 3 compatible way.  Also change annotations
to reflect this fact.
2016-07-10 11:30:13 -07:00
Taranjeet a8a4caf2c0 zerver: Fix lines with length greater than 120. 2016-07-08 11:41:43 -07:00
Eklavya Sharma 4761cc27dd zerver/middleware.py: Fix annotations.
* Use abstract types where relevant.
* Fix string types.
* Fix annotation of args and kwargs.
2016-07-04 02:14:42 +05:30
medullaskyline c5f0d5b40a Annotate zerver.middleware. 2016-06-04 18:32:06 -07:00
Umair Khan 08fbd57245 [i18n] Make error messages translatable.
Make all strings passing through `json_error` and `JsonableError`
translatable.

Fixes #727
2016-05-31 07:40:42 -07:00
Tim Abbott 92bec8cfea Merge Zulip 1.3.12 security release. 2016-05-10 11:32:26 -07:00
Tim Abbott 3cde06ea33 Add support for setting HTTP status codes in JsonableError. 2016-05-10 09:50:48 -07:00
Tim Abbott 54022ac204 Fix unnecessary whitespace between , and ). 2016-05-04 14:16:53 -07:00
Ryan Moore beac606ce6 switch output stats memcached -> remote_cache 2016-03-31 12:54:29 -07:00
Ryan Moore 85b05d4e2b s/memcached_output/remote_cache_output/g 2016-03-31 12:54:29 -07:00
Ryan Moore 5346e2ac23 s/memcached_count_delta/remote_cache_count_delta/g 2016-03-31 12:54:29 -07:00
Ryan Moore 1a2117292f s/memcached_requests/remote_cache_requests/g 2016-03-31 12:54:28 -07:00
Ryan Moore 16c936f638 s/memcached_time/remote_cache_time/g 2016-03-31 12:54:28 -07:00
Tim Abbott df0d2a726d python3: Add missing utf-8 encoding/decoding in various places. 2016-03-08 09:14:15 -08:00
Tim Abbott 10f15a2d00 middleware: Fix str/unicode type mismatch in statsd_path. 2016-02-03 19:29:07 -08:00
Tim Abbott b879b7ff42 Use logger.debug when logging 200/304 output on static assets. 2015-12-25 16:23:57 -08:00
Tim Abbott 06f6ee6566 Apply Python 3 futurize transform lib2to3.fixes.fix_idioms. 2015-11-01 09:25:47 -08:00
Tim Abbott 1f2aa2fcab Fix write_log_line for real.
(imported from commit cbb5c38b8e6c31822b28c478463978aa6cab33d4)
2015-08-22 14:40:47 -07:00
Tim Abbott f1bf5ba24f Fix write_log_line breakage for websockets.
(imported from commit 43bf24822329cf9729654ba58e9ffb0bff3403da)
2015-08-22 14:19:35 -07:00
Reid Barton ab9539cffe Remove OpenID authentication
(imported from commit 70a859041a851ed10dc40cfc068330e472d2ed09)
2015-08-20 23:52:48 -07:00
Reid Barton dfdc34603e Django 1.7 compatibility: handle both response.content and response.streaming_content
(imported from commit faaaff96819731a334d52b7d715c8ddb7c0d4293)
2015-08-20 23:01:26 -07:00
Tim Abbott eb1631f78d Set session cookie domain for *.e.zulip.com hostnames.
(imported from commit 42b15de3b4576341304041588ffaceac6f40baaf)
2015-01-15 21:09:52 -08:00
Tim Abbott 7e786d5426 Import default session middleware as start for custom session middleware.
(imported from commit 76aae367ab6ea5c2a7b0d98368482a3cb312b217)
2015-01-15 21:09:52 -08:00
Luke Faraone 2d3a7e5418 Use a different status code and include seconds remaining header in ratelimits
This will make it slightly easier to consume the data from our clients.

Ref:
    RFC 6585 §4

(imported from commit 6d323dc25db78a6d84a163add950f039e03e73d3)
2014-03-11 13:06:19 -04:00
Leo Franchi c504435bc3 Blacklist more paths, and fix paths with / to use . instead
(imported from commit 7e1840b7efb5d4f6e27307c3f7c95a9c822c8086)
2014-02-03 14:06:58 -05:00
Leo Franchi 30ae1c3463 Blacklist a few more statsd paths
(imported from commit 893b3d6c25e3a626b2948e69566fe5bd0db59813)
2014-01-22 10:49:49 -05:00
Zev Benjamin db23674749 Do query time tracking at the psycopg2 level instead of the Django level
This allows us to track the query time of SQLAlchemy and raw queries.

(imported from commit 818a4ee41786ffc57b80d7ed1cfba075f29b6ee5)
2014-01-14 11:47:12 -05:00
Steve Howell 5dc3d9abce Log internal queries if they are >= 5s.
(imported from commit ee88fcd6292a177e02bfe5e5bca5480b0e474030)
2013-12-26 09:23:18 -05:00
Steve Howell eb6868704f Give higher threshold for webathena kerberos queries.
These are mostly out of our control, so they are not very actionable.

(imported from commit ef342ec1edbff0fa1a934413a7f19ed14817a502)
2013-12-26 09:20:59 -05:00
Steve Howell f61740551c Bump slow query threshold to 1.2s
(imported from commit 8d97fc22d208274bc57b884828957dacf396348a)
2013-12-26 09:16:49 -05:00
Steve Howell 89f3a7c72f Break up conditional in is_slow_query().
(imported from commit c7ca42965e917a0386069c915c0225cefc218c3e)
2013-12-26 09:13:00 -05:00
Steve Howell e0a1841b1c Extract middleware.is_slow_query() and add tests.
(imported from commit 60902244a420800f558fdf2f1c38b4ed736c1286)
2013-12-26 09:09:15 -05:00
Tim Abbott b5754758a7 Don't alert on slow queries for /user_activity and /realm_activity.
(imported from commit 8b08ad47138477068f432ff15c56b2ba93ac2db6)
2013-12-19 17:41:36 -05:00
Tim Abbott 66e72d4705 Ignore slow queries on certain non-customer-facing URLs.
(imported from commit c3f53883d93a75d805b15f617f391ccd1a492ce8)
2013-12-19 17:20:43 -05:00
Tim Abbott 2ca5f43f05 Report json format 500 errors from all json format views.
Previously, we only did this via rest_dispatch.

(imported from commit b0edfdccea294378292b64677a64d5b01f936b08)
2013-12-19 16:48:51 -05:00
Tim Abbott b30afe432e Return a nice JSON error when CSRF errors happen in JSON views.
(imported from commit 916166c115f9b3ba0fdc93f8d917ff37ae22c2ae)
2013-12-19 16:48:51 -05:00
Tim Abbott c91415f318 Improve names for per-request display recipient cache.
It incorrectly advertises itself as per-process.

(imported from commit faf7ca7374d020058e80249bb16a4c6afbcb3e44)
2013-12-19 14:04:55 -05:00
Tim Abbott cc0ce6b437 Finish event handlers when disconnecting from an event queue.
This should help prevent timeouts from clients whose requests have
been supplanted.

(imported from commit fdb3a89c4ec02bb23d0fba50ea558d48cb786916)
2013-12-12 16:52:25 -05:00
Tim Abbott 878e07a5d8 Fix doubled query count logging when DEBUG=True.
(imported from commit 32dfc4c4e366ca80e659ca3d6ab4959b0235e861)
2013-12-03 12:21:06 -05:00
Tim Abbott 26f7b783a2 logging: Fix request processing time to not count initialization.
Previously, we counted not just the time required to process a
particular request, but also the time required to import+find the view
function via urls.py.  The latter is usually fast, but when a new
Django thread starts up, it can take significant time, resulting in us
flagging slow requests on each server restart and also when a new
Django thread starts up on prod to handle requests.

This change means that we no longer include that startup time as part
of request processing time -- but we still log it in the case that it
was more than 5ms, so that we can identify why a particular request
was slower than expected if high startup times become a problem.  We
annotate the time in log lines as "+start" rather than just "start" to
make clear that it's not counted in the total.

(imported from commit c677682e23b26005060390d85d386234f4f463ad)
2013-11-18 18:05:19 -05:00
Tim Abbott d44c6636c6 Add setting to enable profiling of all requests.
This is useful for the occasional case where we cannot figure out what
is causing a particular problem, but it can be easily reproduced on
staging.

(imported from commit 8b51184a8b686814f2c6ff103ba355538463ceb0)
2013-11-18 18:05:19 -05:00
Zev Benjamin eec195f421 socket: Use our full logging infrastructure
We also now separate out the times for the socket overhead, the
request service time, and the queuing delays.

(imported from commit e1683f7f28b968b86ebb701b0ac29b00ac6d67c3)
2013-11-12 15:24:30 -05:00
Zev Benjamin 6ff8ff0f3f Make all the work of the logging middleware not depend on Request or Response objects
(imported from commit ce13909d338230a931cb09405d54bb872b2ff0a6)
2013-11-12 15:24:30 -05:00
Zev Benjamin 32ed5f9f42 Move flushing the display recipient cache to its own middleware
(imported from commit 27a6935a5830ef986b18de169d66dd86d273d064)
2013-11-12 15:24:30 -05:00
Zev Benjamin 53ec292022 Store logging data in a dict instead of individual attributes
(imported from commit f7d76670428d5d8298c572a23cbfffc1d9695f23)
2013-11-12 15:24:30 -05:00
Tim Abbott 2e833613c3 [Django 1.6] update monkeypatching for CursorDebugWrapper.
(imported from commit cb4b44a2bb6ba6efbd1fb5710da2df7c1dec7f81)
2013-11-08 08:22:04 -05:00
Leo Franchi 980dc8345a Strip unicode chars from statsd path
(imported from commit 1c5829be7f89ad1e26be52b416a51da63e2a94cf)
2013-11-05 17:47:15 -05:00
Tim Abbott 5806a6e508 logging: Log the type of the narrow for get_old_messages.
(imported from commit 9e6471c10602242c924f29d29bc780667ac17672)
2013-10-21 14:33:24 -04:00
Tim Abbott 1d4f39f55a logging: Update slow query logger to not display client string.
I believe with this change the log lines will fit much better into
Zulip, and the Client string was I suspect rarely important for
responding to slow queries (and is always available in the main log
anyway).

(imported from commit ad56f446bf3fb96a14a56b825f46c1dad9b6babe)
2013-10-21 14:33:24 -04:00
Tim Abbott f3fee40d1d logging: Don't log query parameters for GET requests.
(imported from commit 4f764ae0168f6b0ad73d13a15ca7e0895514951c)
2013-10-21 14:33:24 -04:00
Leo Franchi 724f7e54e7 Clean up slow log displays
(imported from commit cab99eeaef14e1f24a6b09b0cdbcdcc45bcb620a)
2013-10-02 11:26:58 -04:00
Leo Franchi 2614716fca Log slow queries to zulip so we notice them
(imported from commit 23f311ad881edda4c4495089ea3b55213470a059)
2013-09-30 17:41:56 -04:00
Leo Franchi 0a1a63c87f Blacklist some webreq urls that we don't want to store for metrics
(imported from commit 60def99010e5c55e77acc65aa516459c6e4ffaae)
2013-09-18 11:56:25 -04:00
Jessica McKellar 2be988e6b7 Give recipient_cache and associated functions a clearer name.
(imported from commit 14b12b196e33dfc4eeca5ab0f12c130ddaef1065)
2013-08-28 10:23:40 -04:00
Jessica McKellar 1f4f799030 Flush the per-process recipient cache after every request.
This cache was created to make recipient lookups within a single
request (e.g. when fetching old messages) cheaper. To support stream
name changes, we need to invalidate this cache on every request so
that users get a consistent view of the name change.

(imported from commit 801051b9f6a108c1f50be7eca9a1242d661919b1)
2013-08-28 10:23:39 -04:00
Tim Abbott 7b9305b06f Rename Django project to zproject.
This includes a hack to preserve humbug/backends.py as a symlink, so
that we don't need to regenerate all our old sessions.

(imported from commit b7918988b31c71ec01bbdc270db7017d4069221d)
2013-08-07 11:04:03 -04:00
Tim Abbott e111a2f9a5 [manual] Rename Django app from zephyr to zerver.
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).

At the time it is deployed, we need to make a few changes directly in
the database:

(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';

(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
2013-08-06 07:39:36 -04:00