Commit Graph

283 Commits

Author SHA1 Message Date
Luke Faraone f76ea63b77 Document humbug-send and humbugrc in README for API users
(imported from commit 9ca282d4d768f2d2a474da1209621f03147954fe)
2012-12-12 16:48:00 -05:00
Luke Faraone 2da0a2275c Switch to the option group pattern for API examples.
(imported from commit 67d0164df822a56f06d5f959297cc2efa9706001)
2012-12-12 16:48:00 -05:00
Luke Faraone 5a0f1897bc Add humbug-send, a more full-featured sending tool.
(imported from commit 631f3c7320192a502b60bc4c4838d068ed60ee76)
2012-12-12 16:48:00 -05:00
Luke Faraone 0057c04e13 Update copyright notices for consistency, use the copyright symbol.
No textual changes were made besides reflowing text and subbing © for
(C).

(imported from commit e36108ad31be6ec65325e4ac48e9fb7d17e409ff)
2012-12-12 16:48:00 -05:00
Luke Faraone d69d935b7c Use a configuration file for API variables.
(imported from commit 784483949f29c865ad1eac522c61e2ee42f406e8)
2012-12-12 16:48:00 -05:00
Jessica McKellar 1ab592f81a feedback-bot: fix display_recipient typo that crashes the bot.
(imported from commit 79ec561fe60074b0eec23780c6d6977a64c86a00)
2012-12-11 10:22:52 -05:00
Tim Abbott dd42d844aa check-mirroring: 'Detected server failure' notices go to stdout, not stderr.
(imported from commit 9118b59130aaad274180c3c639ec897bc5634b9f)
2012-12-11 08:55:17 -05:00
Tim Abbott dbaeb9b1d5 zephyr_mirror: Add comment explaining why we drop the @ATHENA.MIT.EDU in zwrite -C.
(imported from commit b591e2e0d6c607ef04e3c0005091078cfe23e3ff)
2012-12-10 17:05:04 -05:00
Tim Abbott fb574966e1 zephyr_mirror: Improve debugging output on outgoing messages.
Previously all that we logged was the sender, which turns out to be a
constant for humbug=>zephyr mirroring.

(imported from commit 527a3ac4b9b815a2b4d6b63c3ad92d9d5ad99a6e)
2012-12-10 17:05:04 -05:00
Tim Abbott 0532d4b49e check_mirroring: Retry automatically on 'Detected server failure' errors.
(imported from commit 2f834218bd492876749dd8953e522725935a7a1d)
2012-12-10 17:05:04 -05:00
Tim Abbott 635c5f3d1f check-mirroring: Use zwrite -n to not send PINGs.
(imported from commit e345f1b86c7f8850ac4eb4ae092cc83717e2cbe8)
2012-12-10 17:05:04 -05:00
Tim Abbott 2d1e072896 zephyr_mirror: Fix handling of Humbug=>Zephyr personals.
(imported from commit 149f8aaf9076583499392bcb09a0250b05c3dea6)
2012-12-08 13:05:44 -05:00
Tim Abbott 17a0246b0b zephyr_mirror: Use a variable for humbug_account_email.
(imported from commit 90da406a5eab67d20b42243e96b8afb3e5b5f342)
2012-12-08 13:00:22 -05:00
Tim Abbott c241713332 zephyr_mirror: Fix detection of personals.
(imported from commit 1fc9050bad36dc15e76f65bdb6532e3e7e601a40)
2012-12-05 12:50:14 -05:00
Tim Abbott b00f23e21f Don't send zwrite pings.
(imported from commit 8dd7894eda6315ab52e61489e21ec1d201e86cc9)
2012-12-05 12:04:18 -05:00
Zev Benjamin 7e4776322d [manual] Unify huddles and personals into private messages on the receive path
feedback-bot and zephyr_mirror will need to be updated and restarted
when this is deployed to prod.

(imported from commit fe2b524424c174bcb1b717a851a5d3815fda3f69)
2012-12-04 18:01:51 -05:00
Keegan McAllister 62fff3b900 Add a Tddium post-build notification task for CUSTOMER1
Resolves #502.

(imported from commit b88fa55ea6561b601a3d6519c84d56ba3098c3fe)
2012-12-04 14:25:44 -05:00
Tim Abbott ef992fc8ba zephyr-mirror-crontab: Fix truncation races.
(imported from commit f50f34b3a739e67f2dffb442ac01e50eb7c9e4b8)
2012-12-03 17:56:38 -05:00
Keegan McAllister ae0d007e56 mirroring: Fix import by putting 'api' path first
(imported from commit b35d962d6456f01b97deeeaa328faa03b65494f8)
2012-12-03 17:50:29 -05:00
Keegan McAllister 0f0ff53527 zephyr_mirror_backend: Rename variable 'humbug'
(imported from commit 2e278e4d2452f750cb874510a762fc9616e7dfb3)
2012-12-03 17:49:27 -05:00
Keegan McAllister 25f0eb67bf gcal-bot: Rename variable 'humbug'
It's weird to clobber the module, although it works.

(imported from commit 160d6e1c596ba6618c3da9c8cdc2f5cd3ca43dc6)
2012-12-03 17:49:27 -05:00
Tim Abbott 021a897786 Add zmirror's crontab to git.
(imported from commit 37c96933a507c661c2ca252c78e5cd9e8a699ba2)
2012-12-03 16:20:19 -05:00
Tim Abbott 0fe3d93fbc zephyr_mirror: Fix case issue in subscriptions syncing.
(imported from commit e87ddab3684f4acae940ee831caadbc346458a11)
2012-12-03 16:20:19 -05:00
Keegan McAllister 82475afe82 Fix module and class name in api/README
(imported from commit c5157d52479ef0e1f26c471df79ffb9c5ea7f51e)
2012-12-03 15:54:54 -05:00
Keegan McAllister 7a080c7911 gcal-bot: Remove .humbug-api-key reading
This behavior is part of api/humbug.py now.

(imported from commit 8a9722c8ad05391160cbb98387f34ef9924d81de)
2012-12-03 15:54:54 -05:00
Keegan McAllister 9125489c20 Rename class HumbugAPI to Client
(imported from commit 4270f31fc5febcd9c444d0d133a1dad3860618f0)
2012-12-03 15:54:54 -05:00
Keegan McAllister bd7b7adb6b Rename api.common to humbug
Fixes #482.

(imported from commit 1bd6a7fd993d8d5e225e0311c288dbce0c369a40)
2012-12-03 15:54:54 -05:00
Tim Abbott 70a8e51351 Rewrite check-mirroring to be more robust and flexible.
Features include:
* Not forking into two processes (shells out to zwrite to send
  instead).  This makes life easier since we're not doing concurrent
  programming.
* Eliminated a lot of hard-to-read or unnecessary debugging output.
* Adding explanatory test suggesting the likely problem for some
  common sets of received messages.
* Much less code duplication.
* Support for testing a sharded zephyr_mirror script (--sharded).
* Use of the logging module to print timestamps -- makes debugging
  some issues a lot.
* Only one sleep, and for only 10 seconds, between sending the
  outgoing messages and checking that they were received.
* Support for running two copies of this script at the same time, so that
  running it manually doesn't screw up Nagios.
* Passed running 100 tests run in a row.

(imported from commit a3ec02ac1d1a04972e469ca30fec1790c4fb53bc)
2012-11-30 17:57:52 -05:00
Tim Abbott 4dc47bcd33 zephyr_mirror: Move class name canonicalization earlier.
We should be canonicalizing stream names to class names in
update_subscriptions_from_humbug, before we even decide which classes
to subscribe to; otherwise deduplication and tracking of which classes
we're already subscribed to won't work.

(imported from commit a751b6fca1022390a087516a0730ff77f13d7edf)
2012-11-29 13:57:06 -05:00
Tim Abbott 1bc98a6f8a call_on_each_message: Remove dead 'first' parameter.
(imported from commit 06885bf8049238184226d9fc7b64345e6848931a)
2012-11-29 13:57:06 -05:00
Tim Abbott 735857008a api: Don't mutate the original request.
This causes e.g. call_on_each_message to switch to dont_block mode
after the first error.

(imported from commit b6a5a10970c987faf8017f0ddae4e0b64a513c6f)
2012-11-29 13:57:06 -05:00
Tim Abbott 7914bb78c6 do_api_query: Reduce error handling code duplication.
Also fixes bugs where the retry code wouldn't work correctly if
verbose wasn't set, prints out which API query had the error, and is
more consistent about printing something to end the "..." sequences.

(imported from commit 15c1e0e4a14c5c5559b43bafe4ec268451ee04f5)
2012-11-29 13:57:06 -05:00
Tim Abbott 0a278b55b9 api: Only retry up to 10 times on connection errors.
(imported from commit f395370120d5e7a635eec7b27e4d5ed0c48b774d)
2012-11-29 13:57:06 -05:00
Tim Abbott b699ad6cde zephyr_mirror: Add options to run class mirror without sharding for debugging.
(imported from commit 4fcc403504d93e30aca6046cba9f1fe5342faac6)
2012-11-29 13:57:06 -05:00
Tim Abbott b3c1f08b3b zephyr_mirror: Don't leak zephyr_mirror processes on send_humbug failures.
(imported from commit be005018672369683d9e3bdbf144f96f58af6b6c)
2012-11-29 13:57:06 -05:00
Tim Abbott b54e5eb3c0 zephyr_mirror: Replace bare excepts with 'except Exception'.
(imported from commit 857212e0e5116471e418d37845550108c0a47d2b)
2012-11-29 13:57:05 -05:00
Tim Abbott de3799589b zephyr_mirror: Compare zephyr subs from server after normalization.
(imported from commit eea290698e83c7d26f23ab3bb0201914417f0e09)
2012-11-28 22:26:10 -05:00
Tim Abbott 18aace499c zephyr_mirror: Add two more restricted access classes.
(imported from commit 06ff0e19c1522ade018aea66f152fdfa55c18e1d)
2012-11-28 22:26:10 -05:00
Tim Abbott d23a83ee81 get_updates: Replace "failures" with the new dont_block option.
(imported from commit 2b4ecb35c4f14b6c408323662ef6f39c6485c62d)
2012-11-28 17:41:38 -05:00
Tim Abbott a1d8e9d30f get_updates: Replace the "mirror" argument with checking "client".
(imported from commit bd84c2c7a456de8ea7ec35eaf76b5bf8647ca61a)
2012-11-28 17:41:38 -05:00
Tim Abbott f58898fba7 zephyr_mirror: Pass an encoded stream to hashlib for sharding.
(imported from commit cc57b3956198b66c9cfc84423434a71b8200d819)
2012-11-28 15:44:35 -05:00
Tim Abbott 9505b38004 zephyr_mirror: Validate zephyr subscriptions happened successfully.
(imported from commit 494895b87eb9b555175c9890c5c2046cf4ce40ac)
2012-11-28 15:44:35 -05:00
Tim Abbott 84ab466ff3 zephyr_mirror: Fix display of personals on non-message classes.
(imported from commit 42ff030d2d78c530428dcede406d46fc71521fef)
2012-11-27 14:12:19 -05:00
Keegan McAllister e4b7fcc2d9 send-nagios-notification: Expand "\n" escape sequences that we get from Nagios
(imported from commit 3e8efbfe8bbd5a2f13a8e02d9f44442b035113ba)
2012-11-27 14:08:49 -05:00
Tim Abbott 2f198a50c1 send_message: Accept subject and message arguments.
(imported from commit 93aca5e750fa74d7ee95d8a00adbf6d4e61b2fc8)
2012-11-27 14:02:24 -05:00
Tim Abbott 8cf9012977 zephyr_mirror: Run the class mirror in 16 parallel shards.
(imported from commit cb17ef999e94cc57d5b01114479728b04475c9c4)
2012-11-27 12:09:57 -05:00
Tim Abbott af40020ff9 zephyr_mirror: Add a --no-forward-from-humbug option.
We'll use this to only have one of our shards forward from Humbug.

(imported from commit a6f3c2180b49c1fa826baf1ae9e7e3ba252a5bfa)
2012-11-27 12:09:57 -05:00
Tim Abbott cdabd2b90a zephyr_mirror: Clean up unnecessary logic in process_notice.
(imported from commit 28599789016fd1006c2de4d5ddc3cf1f10baa47f)
2012-11-27 12:09:57 -05:00
Tim Abbott 4b6f771be1 zephyr_mirror: Fix the check for whether a Zephyr is a personal.
(imported from commit 609b6339fa82ecbd6435a386d24bf555e75a8120)
2012-11-27 12:09:57 -05:00
Tim Abbott f05967034b zephyr_mirror: Reorder process_notice to log fewer "Skipping" messages.
Previously, we were sending "Skipping message we got from Humbug!"
for messages we wouldn't have forwarded anyway.

(imported from commit 36df85a61336ac00e3d7913d5a417d6b42764350)
2012-11-27 12:09:57 -05:00