queue_processors: Defer initial email connection creation.

We previously created the connection to the outgoing email server when
the EmailSendingWorker was first created.  Since creating the
connection can fail (e.g. because of firewalls or typos in the
hostname), this can cause the `QueueProcessingWorker` creation to
raise an exception.  In multi-threaded mode, exceptions in the worker
threads which are _not_ during the handling of a specific event
percolate out to `log_and_exit_if_exception` and trigger the
termination of the entire process -- stopping all worker threads from
making forward progress.

Contain the blast radius of misconfigured email servers by deferring
the opening of the connection until it is first needed.  This will not
cause any overall performance change, since it only affects the
latency of the very first email after startup.

(cherry picked from commit c618c42729)
This commit is contained in:
Alex Vandiver 2024-01-11 21:59:01 +00:00 committed by Tim Abbott
parent f298889cbf
commit 5f5ad2f0b1
1 changed files with 3 additions and 2 deletions

View File

@ -790,7 +790,7 @@ class MissedMessageWorker(QueueProcessingWorker):
class EmailSendingWorker(LoopQueueProcessingWorker):
def __init__(self, threaded: bool = False, disable_timeout: bool = False) -> None:
super().__init__(threaded, disable_timeout)
self.connection: BaseEmailBackend = initialize_connection(None)
self.connection: Optional[BaseEmailBackend] = None
@retry_send_email_failures
def send_email(self, event: Dict[str, Any]) -> None:
@ -812,7 +812,8 @@ class EmailSendingWorker(LoopQueueProcessingWorker):
@override
def stop(self) -> None:
try:
self.connection.close()
if self.connection is not None:
self.connection.close()
finally:
super().stop()