Commit Graph

194 Commits

Author SHA1 Message Date
Umair Khan 68513952fb email-worker: Create EmailSendingWorker.
This commit just copies all the code from MissedMessageSendingWorker
class to a new EmailSendingWorker class. All the logic to send an email
through a queue was already there. This commit only makes the logic
generic. It does so by creating a special purpose queue called
'email_senders' to send any type of email. To make
MissedMessageSendingWorker still work we derive it from
EmailSendingWorker. All the tests that were testing
MissedMessageSendingWorker now run against EmailSendingWorker.
2017-12-20 19:36:27 -08:00
Vishnu Ks 16d8244c0a tests: Eliminate 'Sending invitation' output spam in test_signup.
Fixes #7563
2017-12-20 18:50:01 -08:00
Rishi Gupta 869b4d41ef models: Add ScheduledEmail.realm.
The two extra queries in the test are due to the assert in
send_future_email.
2017-12-19 17:46:36 -08:00
Shreyansh Dwivedi 47fcb27e39 invitations: Remove custom_body.
Fixes #7672
2017-12-11 19:23:54 -08:00
Rishi Gupta 968aae167b invitations: Remove get_prereg_user_by_email.
The original logic is buggy now that emails can belong to (and be
invited to) multiple realms.

The new logic in the `invites` queue worker also avoids the bug where
when the PreregistrationUser was gone by the time the queue worker got
to the invite (e.g., because it'd been revoked), we threw an exception.

[greg: fix upgrade-compatibility logic; add test; explain
revoked-invite race above]
2017-12-06 20:35:50 -08:00
Tim Abbott b2cb443d24 subs: Fix clearing unread counts when leaving private streams.
Because we use access_stream_by_id here, and that checks for an active
subscription to interact with a private stream, this didn't work.

The correct fix to add an option to active_stream_by_id to accept an
argument indicating whether we need an active subscription; for this
use case, we definitely do not.
2017-11-29 14:40:08 -08:00
Eeshan Garg c45517f544 python-zulip-api: Upgrade to PyPI package release 0.3.8.
There's one migration required by this release:

* queue_processors: Stop passing state_handler to handle_message.

  state_handler is now a property of bot_handler and thus, does
  not need to be passed to bot_handler.handle_message().

  The commit responsible is:
  2a74ad11c5
2017-11-27 20:31:37 -08:00
Vishnu Ks 766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Tim Abbott 638eb7a8e4 docs: Update links to ReadTheDocs to always use https.
This is better security practice.

We also add a lint rule to enforce this for the future.
2017-11-16 10:59:24 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Steve Howell dbcc76b996 Fix MissedMessageWorker.
This fixes a regression in 25c669df52.

We were draining the queue in both the superclass and the subclass,
so by the time the subclass started processing events, there were
no events to process.  Now the subclass properly uses the events
passed in from the superclass.
2017-11-15 10:26:28 -08:00
derAnfaenger d3bee4fcc5 queue_processors: Exclude various lines from coverage.
The os.mkdir call is straightforward and doesn't testing.
Workers relying on LoopQueueProcessingWorker are tested
with its consume method that exists solely for this purpose.
2017-11-10 13:23:53 -08:00
Tim Abbott 25c669df52 queue_processors: Extract LoopQueueProcessingWorker class.
This should help make it easier to add tests coverage for these queue
processors, since they now use a system more similar to the other
queue processors.
2017-11-09 15:20:40 -08:00
rht de319b4558 refactor: Remove six.moves.StringIO import. 2017-11-07 10:51:44 -08:00
derAnfaenger c8e87a322d queue processors: Exclude TestWorker from coverage. 2017-11-07 09:38:58 -08:00
rht 8990b1046d zerver: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
Tim Abbott eade4d0052 tornado: Use call_consume_in_tests for message_sender queue.
This increases test coverage of queue_processors.py significantly,
with no real cost, achieving valuable progress torwards #6542.
2017-11-03 14:09:48 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Tim Abbott b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
derAnfaenger 1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
Tim Abbott ad165a6f8f settings: Remove remaining DEPLOYMENT_ROLE_* code remnants.
These should have been removed when we removed Zilencer.
2017-10-23 21:15:03 -07:00
derAnfaenger c8a2702b9a embedded bots: Add functional state handler.
This replaces the former non-functional StateHandler
stub with a dictionary-like state object. Accessing it will
will read and store strings in the BotUserStateData model.

Each bot has a limited state size. To enforce this limit while
keeping data updates efficient, StateHandler caches the expensive
query for getting a bot's total state size. Assignments to a key
then only need to fetch that entry's previous size, if any, and
compare it to the new entry's size.
2017-10-19 13:09:23 -07:00
Tim Abbott 6176d0fbca json: Replace most use of simplejson with json.
This is progress towards removing simplejson as a dependency.
2017-10-11 22:55:35 -07:00
Umair Khan 435fe40199 worker: Retry signups queue event on 400. 2017-10-05 23:14:19 -07:00
Umair Khan 19e2551e82 mypy: Change type to Dict for SignupWorker. 2017-10-05 23:14:19 -07:00
Umair Khan d95d34a66a Retry email failures in missed-message emails queue.
Fixes #6518.
2017-10-03 10:35:07 -07:00
Umair Khan 7d6ddaad26 mypy: Use Dict instead of Mapping in queues. 2017-10-03 10:35:07 -07:00
Greg Price 2a832386dc queue: Move more received-event logs to DEBUG level.
These can spew a lot of noise when starting the dev server, and aren't
in a super helpful form for normal server administration anyway.
2017-09-28 18:32:42 -07:00
Tim Abbott 630ecf9669 queue: Move user presence log events to DEBUG log level.
These are pretty noisy, and generally of little interest, since
presence requests come with a server log entry anyway.
2017-09-28 14:35:55 -07:00
rht 2ae47525d3 zerver/worker: Remove `import six`. 2017-09-27 17:06:16 -07:00
rht 41d8db373a zerver/worker: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott 84926ff321 slow_queries: Log slow queries even if not reported via Zulip.
This should help make these logs more useful for debugging on an
arbitary Zulip system.
2017-09-22 14:11:07 -07:00
Tim Abbott ec61a070b4 signups: Add basic logging for new signup processing. 2017-09-22 14:11:07 -07:00
Tim Abbott f78dd33037 invites: Add logging for newly sent invitation emails. 2017-09-22 14:11:07 -07:00
Rishi Gupta a7c8770f97 emails: Move enqueue_welcome_emails outside of signups queue.
The only thing this queue should do is sign you up for the newsletter, since
it is only populated if newsletter_data is not None.
2017-09-22 06:20:33 -07:00
Tim Abbott f706f657c0 signup: Fix invitation emails not being cleared properly.
Previously, invitation reminder emails were only being cleared after a
successful signup if newsletter_data was available, since that was the
circumstance in which we were calling the relevant queue processor
code.  Now, we (1) clear them when a human user finishes signing up
and (2) correctly clear them using the 'address' field of
ScheduleEmail, not user_id.
2017-09-21 06:15:11 -07:00
Steve Howell ba397b5109 Use user_ids, not full objects, in render path.
There is no reason for either render_incoming_message() or
render_markdown() to require full UserProfile objects just to
triage alert words.

By only asking for user_ids, we save extra queries in two
callpaths and we make it easier to start using user_ids in
do_send_messages().
2017-09-12 04:22:55 -07:00
neiljp (Neil Pilgrim) 116b8dd9c9 mypy: Set assign_queue() parameter queue_type to not be Optional. 2017-08-07 21:27:50 -07:00
Abhijeet Kaur 1deb58b178 bots: Add complete test-coverage for bot_lib.py file.
Also, add error handling for get_bot_handler instead of
throwing an assertion error.
2017-07-27 15:50:29 -07:00
Abhijeet Kaur 6f60c65a65 embedded bots: Add tests for verification of embedded bot services.
Add test to check if the embedded bot service being used is in the
registry or not.
Add test to check if the bot being added to the registry has a valid
bot corresponding to it.
Move 'get_bot_handler' to 'zerver/lib/bot_lib.py' as it is an independent
function, not related to the 'EmbeddedBotWorker' class that it was
previously a part of.
2017-07-24 17:14:14 -07:00
Elliott Jin 6a61a8a431 outgoing webhooks: Consolidate interfaces into lib/outgoing_webhook.py 2017-07-24 14:10:14 -07:00
vaibhav 87dcd5442a outgoing webhook system: Minor fixes.
This fixes some error message strings and skips converting request_data
into json. From now, conversion would be the responsibility of interface.
Also, base_url is now not passed into event structure.
2017-07-24 14:10:14 -07:00
Abhijeet Kaur 5980d420a8 Embedded bots: Fix minor errors to make embedded bots/service run.
Splitting bot_lib.py file into 2 files led to unnecessary
redirection of the code workflow. For an embedded bot/service to
send a reply, it was being redirected 3 times.

First, the code flow comes to "EmbeddedBotHandler" class to send
reply, then it goes to the common function in "zulip_bots/lib.py",
then it would come back to "EmbeddedBotHandler". Later on, if we
create an abstract class, from where the bot work flow would
directly hit and then from there it is classified into
EmbeddedBotHandler or ExternalBotHandler and accordingly it would
get redirected.

Now, first the bot flow goes to it's handler class External or
Embedded (where we pass that this is External or Embedded bot as
parameter) and then goes to a common point and then comes back to
the same class.
2017-07-20 10:22:52 -07:00
Eeshan Garg 148bb4db09 requirements: Update requirements/ to install bots/API packages.
This is required, since we just reorganized the python-zulip-api
repository into 3 packages.

A nice side effect is that we get to eliminate some now-unnecessary
code for editing sys.path.
2017-07-18 00:10:30 -07:00
Rishi Gupta 0f4b71b766 confirmation: Liberate get_link_for_object from ConfirmationManager. 2017-07-17 23:18:47 -07:00
Rishi Gupta 36dbb76516 emails: Rename clear_followup_emails_queue. 2017-07-17 16:05:38 -07:00
Rishi Gupta 5b3e6af2e5 emails: Remove only emails of the correct type when clearing queue. 2017-07-17 16:05:38 -07:00
Rishi Gupta f51bd898dc notifications: Change clear_followup_emails_queue to take a user_id. 2017-07-17 16:05:38 -07:00
Rishi Gupta 898269bbac email: Change send_email to raise exception on failure.
More in line with how we do error handling in the rest of Zulip.
2017-07-16 16:56:39 -07:00
Rishi Gupta eacdb0b302 emails: Change welcome emails to use to_user_id. 2017-07-16 16:56:39 -07:00