Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/zerver/lib
History
Mateusz Mandera 1901775383 email_mirror: Add realm-based rate limiting. 
Closes #2420

We add rate limiting (max X emails withing Y seconds per realm) to the
email mirror. By creating RateLimitedRealmMirror class, inheriting from
RateLimitedObject, and rate_limit_mirror_by_realm function, following a
mechanism used by rate_limit_user, we're able to have this
implementation mostly rely on the already existing, and proven over
time, rate_limiter.py code. The rules are configurable in settings.py in
RATE_LIMITING_MIRROR_REALM_RULES, analogically to RATE_LIMITING_RULES.

Rate limit verification happens in the MirrorWorker in
queue_processors.py. We don't rate limit missed message emails, as due
to using one time addresses, they're not a spam threat.

test_mirror_worker is adapted to the altered MirrorWorker code and a new
test - test_mirror_worker_rate_limiting is added in test_queue_worker.py
to provide coverage for these changes.
 		2019-03-18 11:16:58 -07:00
..
bugdown markdown: Improve handling of broken img urls. 2019-03-08 13:39:34 -08:00
url_preview python: Remove unused imports. 2019-02-22 16:54:36 -08:00
webhooks zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
__init__.py
…
actions.py email_mirror: Change stream name encoding in mirror addresses. 2019-03-18 11:14:22 -07:00
addressee.py internal_prep_stream_message: Support accepting a Stream object. 2019-02-08 08:53:16 -08:00
alert_words.py alert_words: Performance improvements in looking for alert_words. 2019-03-01 15:36:39 -08:00
api_test_helpers.py api_tests: Test zulip.Client.add_reaction. 2019-02-18 10:24:13 -08:00
attachments.py
…
avatar.py python: Clean up various if False blocks. 2018-12-17 11:14:47 -08:00
avatar_hash.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
bot_config.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
bot_lib.py streams: Refactor multi-option helpers into separate functions. 2019-02-12 11:10:26 -08:00
bot_storage.py python: Stop importing Length from the wrong file. 2019-02-02 17:09:10 -08:00
bulk_create.py stream-descriptions: Create wrapper for rendering stream descriptions. 2019-03-06 17:16:14 -08:00
cache.py cache: Add missing : in test-backend key prefixes. 2019-03-18 10:56:50 -07:00
cache_helpers.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
camo.py camo: Add endpoint to handle camo requests. 2019-01-04 10:27:04 -08:00
ccache.py str_utils: Move force_bytes into ccache.py. 2018-05-15 19:07:32 -07:00
context_managers.py
…
create_user.py email: Set email based on realm email_address_visibility. 2019-02-04 18:45:10 -08:00
db.py python: Remove unnecessary NonBinaryStr TypeVar. 2018-11-27 11:44:09 -08:00
debug.py mypy: Set local_partial_types = True. 2018-05-21 22:41:00 -07:00
dev_ldap_directory.py dev_ldap: Add custom profile data. 2019-01-29 16:01:30 -08:00
digest.py notifications: Rename notifications.py to email_notifications.py. 2019-03-15 11:02:17 -07:00
domains.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
email_mirror.py email_mirror: Add realm-based rate limiting. 2019-03-18 11:16:58 -07:00
email_notifications.py notifications: Rename notifications.py to email_notifications.py. 2019-03-15 11:02:17 -07:00
emoji.py emoji: Move `EMOTICON_CONVERSIONS` mapping to build_emoji infra. 2018-07-23 12:35:08 -07:00
error_notify.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
events.py soft-deactivation: Rename maybe_catch_up_soft_deactivated_user. 2019-03-13 17:16:22 -07:00
exceptions.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
export.py send_email: Add ScheduledEmail support for multiple recipients. 2019-03-15 11:02:12 -07:00
feedback.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
fix_unreads.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
generate_test_data.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
hotspots.py docs: Mention bug in ALWAYS_SEND_ALL_HOTSPOTS feature. 2019-02-27 10:21:30 -08:00
html_diff.py
…
i18n.py i18n: Pass translation data in page_params. 2018-05-30 08:57:26 -07:00
import_realm.py streams: Add first_message_id tracking first message in stream. 2019-03-11 13:30:49 -07:00
initial_password.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
integrations.py integrations: Rename social media logos to bypass Adblock Plus. 2019-02-23 10:56:43 -08:00
json_encoder_for_html.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
logging_util.py
…
management.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
mdiff.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
mention.py markdown: Change syntax of silent mentions ( _@person -> @_person). 2019-02-20 10:41:42 -08:00
message.py alert_words: Performance improvements in looking for alert_words. 2019-03-01 15:36:39 -08:00
migrate.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
mobile_auth_otp.py mobile: Make otp_encrypt_api_key accept API keys. 2018-08-08 16:45:40 -07:00
name_restrictions.py signup: Add opayq.com to disposable domain whitelist. 2018-08-13 10:56:47 -07:00
narrow.py search: Add stop words to page_params. 2019-02-05 10:23:28 -08:00
onboarding.py streams: Refactor multi-option helpers into separate functions. 2019-02-12 11:10:26 -08:00
openapi.py Minimize race conditions for reading zulip.yaml. 2018-09-12 11:07:20 -07:00
outgoing_webhook.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
parallel.py
…
profile.py
…
push_notifications.py push notif: Make O(1) database queries in handle_remove_... 2019-02-26 16:41:55 -08:00
queue.py requirements: Upgrade pika from 0.12.0 to 0.13.0. 2019-01-31 10:04:07 -08:00
rate_limiter.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
realm_icon.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
realm_logo.py settings: Add support for uploading logo for night mode. 2019-02-18 15:15:57 -08:00
redis_utils.py
…
remote_server.py production: Add optional support for submitting usage statistics. 2019-02-26 17:35:10 -08:00
request.py requirements: Upgrade pyflakes. 2019-02-01 07:26:13 -08:00
request.pyi extract_recipients: Support user IDs. 2019-01-08 19:08:49 -03:30
response.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
rest.py report: Allow error-reporting views from unauthed users. 2018-12-16 15:44:48 -08:00
retention.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
send_email.py emails: Move clear_scheduled_*emails to send_email.py. 2019-03-15 11:02:17 -07:00
sessions.py sessions: Suppress coverage error until we can debug. 2018-12-01 10:08:57 -08:00
soft_deactivation.py soft-deactivation: Run catch-up when "auto" deactivate is run. 2019-03-14 11:53:15 -07:00
sqlalchemy_utils.py mypy: Set local_partial_types = True. 2018-05-21 22:41:00 -07:00
statistics.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
storage.py python: Remove now-unnecessary str_utils library. 2018-11-27 11:57:54 -08:00
stream_recipient.py
…
stream_subscription.py
…
stream_topic.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
streams.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
subdomains.py subdomains: Extract get_subdomain_from_hostname function. 2019-03-11 12:01:11 -07:00
test_classes.py models: Fix performance of supported_auth_backends with caching. 2019-03-17 15:14:05 -07:00
test_data.source.txt
…
test_fixtures.py test_fixtures: Fix bug with run_db_migrations for test platform. 2018-07-09 14:17:53 +05:30
test_helpers.py tests: Optimize test-backend performance. 2019-02-19 14:50:41 -08:00
test_runner.py tests: Optimize test-backend performance. 2019-02-19 14:50:41 -08:00
tex.py Enable pycodestyle W605 (invalid escape sequence). 2018-07-03 16:54:46 +02:00
thumbnail.py camo: Add endpoint to handle camo requests. 2019-01-04 10:27:04 -08:00
timeout.py
…
timestamp.py
…
timezone.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
topic.py topics: Fix messages_for_topic to use iexact matching. 2019-01-24 17:55:15 -08:00
topic_mutes.py Extract topic_match_sa() helper. 2018-11-07 10:03:53 -08:00
transfer.py python: Remove unused imports. 2019-02-22 16:54:36 -08:00
type_debug.py zerver/lib: Change use of typing.Text to str. 2018-05-10 14:19:49 -07:00
types.py extract_recipients: Support user IDs. 2019-01-08 19:08:49 -03:30
unminify.py unminify: Clean up unnecessary repetition of webpack:/// URLs. 2018-08-03 15:56:40 -07:00
upload.py models: Move currently_used_upload_space function to Realm model. 2019-03-04 18:46:13 -08:00
url_encoding.py topic -> subject: Extract get_topic_from_message_info(). 2018-11-14 23:24:06 -08:00
user_agent.py user_agent: Use re.X to make the regex a little more readable. 2018-12-05 14:58:13 -08:00
user_groups.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
user_status.py user status: Add "status_text" support in the backend. 2019-01-24 09:46:35 -08:00
users.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
utils.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
validator.py stream: Fix validator for stream colors. 2019-03-11 11:43:39 -07:00
video_calls.py zoom: Fix handling of 201 status codes from zoom API. 2019-01-16 15:37:06 -08:00
widget.py zerver core: Remove unused imports. 2019-02-02 17:41:24 -08:00
zcommand.py zcommand: Fix broken switch mode message. 2018-08-08 11:29:49 -07:00
zephyr.py python: Remove now-unnecessary str_utils library. 2018-11-27 11:57:54 -08:00