zulip/tools/documentation_crawler
Adam Birds 4d9f161e0f tools: Suppress errors for github.com links.
I have suppressed errors for github.com by adding an function to
exclude domains as well as urls; this is necessary because GitHub has
marked this tool's User-Agent as a blocker crawler.

I have also suppressed reoccurring url errors that do definitely exist.

Fixes #17928.
2021-04-09 12:25:23 -07:00
..
documentation_crawler tools: Suppress errors for github.com links. 2021-04-09 12:25:23 -07:00
__init__.py
scrapy.cfg cleanup: Delete leading newlines. 2019-08-06 23:29:11 -07:00