queue: Prevent an AttributeError rather than swallowing it.

When the RabbitMQ server disappears, we log errors like these:

```
Traceback (most recent call last):
  File "./zerver/lib/queue.py", line 114, in json_publish
    self.publish(queue_name, ujson.dumps(body))
  File "./zerver/lib/queue.py", line 108, in publish
    self.ensure_queue(queue_name, do_publish)
  File "./zerver/lib/queue.py", line 88, in ensure_queue
    if not self.connection.is_open:
AttributeError: 'NoneType' object has no attribute 'is_open'

During handling of the above exception, another exception occurred:
[... traceback of connection failure inside the retried self.publish()]
```

That's a type error -- a programming error, not an exceptional
condition from outside the program.  Fix the programming error.

Also move the retry out of the `except:` block, so that if it also
fails we don't get the exceptions stacked on each other.  This is a
new feature of Python 3 which is sometimes indispensable for
debugging, and which surfaced this nit in the logs (on Python 2 we'd
never see the AttributeError part), but in some cases it can cause a
lot of spew if care isn't taken.
This commit is contained in:
Greg Price 2017-10-18 15:11:55 -07:00 committed by Greg Price
parent 16d06c6a78
commit 4b5c52fc99
1 changed files with 5 additions and 4 deletions

View File

@ -85,7 +85,7 @@ class SimpleQueueClient:
# type: (str, Callable[[], None]) -> None
'''Ensure that a given queue has been declared, and then call
the callback with no arguments.'''
if not self.connection.is_open:
if self.connection is None or not self.connection.is_open:
self._connect()
if queue_name not in self.queues:
@ -112,11 +112,12 @@ class SimpleQueueClient:
# Union because of zerver.middleware.write_log_line uses a str
try:
self.publish(queue_name, ujson.dumps(body))
except (AttributeError, pika.exceptions.AMQPConnectionError):
return
except pika.exceptions.AMQPConnectionError:
self.log.warning("Failed to send to rabbitmq, trying to reconnect and send again")
self._reconnect()
self.publish(queue_name, ujson.dumps(body))
self._reconnect()
self.publish(queue_name, ujson.dumps(body))
def register_consumer(self, queue_name, consumer):
# type: (str, Consumer) -> None