Commit Graph

17 Commits

Author SHA1 Message Date
Mateusz Mandera 122d0bca83 check-rabbitmq-queue: Add a simple algorithm to analyze queue stats.
This new algorithm is designed to avoid monitoring paging when a queue
simply has bursty behavior.
2020-04-09 13:41:01 -07:00
Mateusz Mandera ea93810d9a check-rabbitmq-queue: Put user check before rabbitmqctl call. 2020-03-22 18:46:28 -07:00
Tim Abbott bbc1484253 check-rabbitmq-queue: Adjust threshholds for paging.
Ultimately, this isn't an effective way to monitor this queue; we want
time-based monitoring, not count-based monitoring.  Doing that
properly will likely involve modifying the queue processor to write
something about its status.

But until we add the monitoring we want, it makes sense to leave this
active with low limits.
2019-10-13 22:39:52 -07:00
Tim Abbott 1c73ce2450 user_activity: Use LoopQueueProcessingWorker strategy.
This should dramatically improve the queue processor's performance in
cases where there's a very high volume of requests on a given endpoint
by a given user, as described in the new docstring.

Until we test this more broadly in production, we won't know if this
is a full solution to the problem, but I think it's likely.  We've
never seen the UserActivityInterval worker end up backlogged without a
total queue processor outage, and it should have a similar workload.

Fixes #13180.
2019-09-21 11:48:24 -07:00
Tim Abbott ad81f700a1 scripts: Remove nagios overrides for missedmessage_emails.
Since 5cec566cb9, the
missedmessage_emails queue no longer is expected to grow a backlog
over time.
2019-04-13 20:43:07 -07:00
Puneeth Chaganti 9876f1b14e check_rabbitmq_queue: Fix the time period when we ignore long queues.
The commit 87d1809657 changed the time when
digests are sent by 3 hours to account for moving from the US East Coast to the
West Coast, but didn't change the time period exception in the
`check-rabbitmq-queue` script.

Closes #5415
2019-04-13 20:43:07 -07:00
Anders Kaseorg e984107966 scripts: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:02:58 -08:00
Tim Abbott 999f264ad3 check_rabbitmq_queue: Exclude slow_queries queue from alerting.
Structurally, this queue has the same property as the missed_message
one, namely that it accumulates things and processes them only every
few minutes.

This should stop Zulip from paging in response to slow queries
accumulating when a server restart happens.
2018-05-25 13:06:50 -07:00
rht 53e37aa511 scripts: Text-wrap long lines exceeding 110. 2017-11-10 16:22:26 -08:00
rht 71188d7b0a scripts: Remove import print_function. 2017-09-29 15:43:30 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00
Tommy Ip e4091c6413 pep8: Fix E222 violations. 2016-11-30 21:49:02 +00:00
Eklavya Sharma 51ea5c1602 scripts/: Make subprocess calls unicode-aware. 2016-07-26 12:06:41 -07:00
Eklavya Sharma 11732f9ab0 Make all scripts in scripts/ pass mypy check. 2016-07-24 00:17:21 +05:30
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Tim Abbott 2761c012e5 Move rabbitmq consumer checks from bots/ to scripts/nagios/. 2016-05-07 19:37:06 -07:00