Commit Graph

44 Commits

Author SHA1 Message Date
Anders Kaseorg bdc365d0fe logging: Pass format arguments to logging.
https://docs.python.org/3/howto/logging.html#optimization

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 10:18:02 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Anders Kaseorg f8c95cda51 mypy: Add specific codes to type: ignore annotations.
https://mypy.readthedocs.io/en/stable/error_codes.html

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 10:46:33 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Anders Kaseorg 453919bbc7 version: Only let `git describe` match tags beginning with a digit.
This will let us use other tags for things like `@zulip/shared`
versions.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-24 14:54:45 -07:00
Anders Kaseorg e0cf3d0e76 zerver: Accept HEAD requests wherever GET requests are accepted.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-12 16:47:41 -07:00
Wyatt Hoodes e331a758c3 python: Migrate open statements to use with.
This is low priority, but it's nice to be consistently using the best
practice pattern.

Fixes: #12419.
2019-07-20 15:48:52 -07:00
Anders Kaseorg 88e90d34b9 try_git_describe: Set cwd, not --git-dir.
git describe --dirty requires a working tree, not just a repository.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-28 14:59:33 -08:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Shubham Dhama 50ed378dd6 bugdown: Improve exception handler for BugdownRenderingException.
This has the benefit that we now get the usual data about the
user/request/etc. in error emails related to bugdown exceptions;
previously we were just getting the traceback in the emails (since our
`mail_admins` template was very simple) and no other debugging
details.

Comments tweaked by tabbott to help make clear exactly what's going on
here, since it's a little subtle and a little hacky.

Fixes #8843.
2018-07-26 09:27:55 -07:00
Aayush Agrawal d32d7a9b4d test_logging_handlers: Mock out `git describe` because can be slow.
This cuts out about 11 calls to `git describe`.  In a nice fast LXC
container following our instructions for development on a Linux host,
this might save "only" about 1.5s; in a dev environment on a Windows
host, the savings have been clocked at 49s, presumably due to an
extremely slow filesystem in the VM.

The tests weren't doing much with this codepath as they were, and
there isn't a lot of value to be gained by testing it anyway; it's
totally non-critical and rarely changes.

[Commit message rewritten by greg.]
2018-03-23 18:24:51 -07:00
Greg Price e44a8b8de9 logging: Add a setting to aid manual testing of error-notif changes.
This is nicer than the "For manual testing ..." comment. :-)
Also as a proper setting we can have it control some logging I
added locally while testing my recent changes to pika logging.
2018-03-21 18:03:05 -07:00
Steve Howell c33102f692 Fix mypy error in AdminNotifyHandler.emit(). 2017-12-22 15:02:18 -05:00
Greg Price a34c01780f errors: Compute deployment metadata on the right deployment.
When I added this "Deployed code" feature to the error reporting,
I apparently hadn't worked out enough of how this code works to
realize that `notify_server_error` may be in a different process,
at a different time and potentially even on a different machine
from the actual error being reported.

Given that architecture, all the data about the error must be computed
in `AdminNotifyHandler`, before sending the report through the queue,
or else it risks being wrong.  The job of `notify_server_error` and
friends is only to format the data and send it off.  So, move the
implementation of this feature in order to do that.

(@showell added some "nocoverage" directives here for code that
is hard to test (exceptions being thrown, deployment files not
existing) and that was originally part of a file that didn't
require 100% coverage)
2017-12-22 11:52:13 -05:00
Greg Price eea1ceb0db errors: Show what logger and source code the message comes from. 2017-12-22 11:52:13 -05:00
Greg Price 77e47900dd errors: Stop showing a long list of Nones when there's no request.
This had contributed to making the emails look broken and of uncertain
accuracy.
2017-12-22 11:52:13 -05:00
Greg Price 68fa29c3ec errors: Make Zulip and email paths more parallel.
This helps prevent them from diverging and getting different sets of
features and fixes.  As a bonus, the email path gets a nice tweak that
the Zulip path has had for years, since f7f2ec0ac, which makes the
emails clearer and less broken-looking when logging a message with no
stack trace.
2017-12-22 11:52:13 -05:00
Greg Price 8c0fd5beaf errors: Clean up the logic slightly.
This deduplicates a little bit of logic, and also has us always put
things into `report` the same way.

Empirically an exception in this codepath is very rare, so we won't
complicate the code by trying to salvage a lot of partial information
if it happens -- just log the traceback, and try to get a minimal
notification sent of the bare fact this happened.
2017-12-22 11:52:13 -05:00
Greg Price b15231dfc2 logging: Rename AdminZulipHandler to AdminNotifyHandler.
This name hasn't been right since f7f2ec0ac back in 2013; this handler
sends the log record to a queue, whose consumer will not only maybe
send a Zulip message but definitely send an email.  I found this
pretty confusing when I first worked on this logging code and was
looking for how exception emails got sent; so now that I see exactly
what's actually happening here, fix it.
2017-12-22 11:52:13 -05:00
rht a1cc720860 zerver: Use Python 3 syntax for typing.
Tweaked by tabbott to fix some minor whitespace errors.
2017-11-28 16:49:36 -08:00
Robert Hönig 0e0a8a2b14 queue processor tests: Call consume by default.
This significantly improves the API for queue_json_publish to not be
overly focused on what the behavior of this function should be in our
unit tests.
2017-11-26 11:45:34 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Tim Abbott 10ab9410c9 python: Sort imports in easy files in zerver/. 2017-11-15 15:50:28 -08:00
derAnfaenger 1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
Tim Abbott 0744961afe docs: Add more comprehensive documentation on error reporting.
Fixes #2232.
2017-09-27 19:47:08 -07:00
rht 2949d1c1e8 zerver: Remove the rest of absolute_import. 2017-09-27 10:02:39 -07:00
Tim Abbott cb6373210b logging_handlers: Fix tracebacks being emailed in subject lines.
Some exceptions happening in queue processors were being incorrectly
emailed out in the subject line, rather than the body, of the error
report.
2017-04-25 18:55:11 -07:00
Tim Abbott b6094b8e91 logging_handlers: Fix missing host key in queue errors.
This fixes exceptions thrown when trying to report errors from queue
processors to email.
2017-04-25 13:48:44 -07:00
Tim Abbott b5cf0067d5 logging_handlers: Avoid super long subject lines. 2017-03-26 13:10:43 -07:00
Tim Abbott 36bb2bf54a AdminZulipHandler: Bring logging_handlers.py to 100% test coverage.
This involved fixing some unupdated code in one code path.
2017-03-26 13:10:43 -07:00
Tim Abbott 267346f5fb AdminZulipHandler: Support passing a record without a request. 2017-03-26 13:10:43 -07:00
Tim Abbott efa151b488 AdminZulipHandler: Extract add_request_metadata. 2017-03-26 13:10:43 -07:00
Tim Abbott 4276face7f AdminZulipHandler: Construct report more dynamically. 2017-03-26 13:10:43 -07:00
Tim Abbott 6e442902e1 AdminZulipHandler: Don't print extra traceback for logged-out users.
This is not an interesting condition worth highlighting for sysadmins.

Fixes #3963.
2017-03-07 21:23:10 -08:00
Tim Abbott b81fd407e8 mypy: Fix several Optional typing errors. 2017-02-10 23:53:44 -08:00
Tim Abbott 20096dfe0d lint: fix PEP8 issue with logging_handlers.py. 2017-02-07 13:22:01 -08:00
Tim Abbott f79bced276 logging: Fix recursive exceptions with too large files.
This resolves another new exception in Django 1.10 that now can happen
in our error reporting code path.
2017-02-07 12:28:54 -08:00
Tim Abbott 20f3705c00 logging_handlers: Add error handling for invalid hosts. 2017-01-29 19:57:09 -08:00
Tim Abbott e69c4458c6 errors: Use hostname to report deployment. 2017-01-28 13:07:09 -08:00
Tommy Ip 89a126b93f zerver: Fix bare except clause. 2017-01-08 16:22:21 -08:00
Alex Huang 007b693cc7 pep8: Fix E131. 2016-12-01 23:16:47 -08:00
Tim Abbott 522ccf8eb2 settings: Extract settings.STAGING_ERROR_NOTIFICATIONS. 2016-07-19 15:28:41 -07:00
Tim Abbott 157a3efb78 Annotate zerver.exceptions, zerver.filters, zerver.logging_handlers. 2016-06-04 22:51:18 -07:00
Tim Abbott e9637a545f Rename zerver.handlers to zerver.logging_handlers for clarity. 2016-03-30 21:48:38 -07:00