Commit Graph

188 Commits

Author SHA1 Message Date
Eeshan Garg c45517f544 python-zulip-api: Upgrade to PyPI package release 0.3.8.
There's one migration required by this release:

* queue_processors: Stop passing state_handler to handle_message.

  state_handler is now a property of bot_handler and thus, does
  not need to be passed to bot_handler.handle_message().

  The commit responsible is:
  2a74ad11c5
2017-11-27 20:31:37 -08:00
Vishnu Ks 766511e519 actions: Mark all messages as read when user unsubscribes from stream.
This fixes a bug where, when a user is unsubscribed from a stream,
they might have unread messages on that stream leak.  While it might
seem to be a minor problem, it can cause significant problems for
computing the `unread_msgs` data structures, since it means we need to
add an extra filter for whether the user is still subscribed, either
in the backend or in the UI.

Fixes #7095.
2017-11-21 20:09:17 -08:00
Tim Abbott 638eb7a8e4 docs: Update links to ReadTheDocs to always use https.
This is better security practice.

We also add a lint rule to enforce this for the future.
2017-11-16 10:59:24 -08:00
Tim Abbott 054952a44a docs: Update links from codebase to point to ReadTheDocs. 2017-11-16 10:53:49 -08:00
Steve Howell dbcc76b996 Fix MissedMessageWorker.
This fixes a regression in 25c669df52.

We were draining the queue in both the superclass and the subclass,
so by the time the subclass started processing events, there were
no events to process.  Now the subclass properly uses the events
passed in from the superclass.
2017-11-15 10:26:28 -08:00
derAnfaenger d3bee4fcc5 queue_processors: Exclude various lines from coverage.
The os.mkdir call is straightforward and doesn't testing.
Workers relying on LoopQueueProcessingWorker are tested
with its consume method that exists solely for this purpose.
2017-11-10 13:23:53 -08:00
Tim Abbott 25c669df52 queue_processors: Extract LoopQueueProcessingWorker class.
This should help make it easier to add tests coverage for these queue
processors, since they now use a system more similar to the other
queue processors.
2017-11-09 15:20:40 -08:00
rht de319b4558 refactor: Remove six.moves.StringIO import. 2017-11-07 10:51:44 -08:00
derAnfaenger c8e87a322d queue processors: Exclude TestWorker from coverage. 2017-11-07 09:38:58 -08:00
rht 8990b1046d zerver: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
Tim Abbott eade4d0052 tornado: Use call_consume_in_tests for message_sender queue.
This increases test coverage of queue_processors.py significantly,
with no real cost, achieving valuable progress torwards #6542.
2017-11-03 14:09:48 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Tim Abbott b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
derAnfaenger 1792dcbd09 tests: Call real consume method of queue processors.
This switches to more real tests for a first batch of
queue_json_publish() calls that don't cause trouble when
used with call_consume_tests=True.
2017-10-26 14:58:03 -07:00
Tim Abbott ad165a6f8f settings: Remove remaining DEPLOYMENT_ROLE_* code remnants.
These should have been removed when we removed Zilencer.
2017-10-23 21:15:03 -07:00
derAnfaenger c8a2702b9a embedded bots: Add functional state handler.
This replaces the former non-functional StateHandler
stub with a dictionary-like state object. Accessing it will
will read and store strings in the BotUserStateData model.

Each bot has a limited state size. To enforce this limit while
keeping data updates efficient, StateHandler caches the expensive
query for getting a bot's total state size. Assignments to a key
then only need to fetch that entry's previous size, if any, and
compare it to the new entry's size.
2017-10-19 13:09:23 -07:00
Tim Abbott 6176d0fbca json: Replace most use of simplejson with json.
This is progress towards removing simplejson as a dependency.
2017-10-11 22:55:35 -07:00
Umair Khan 435fe40199 worker: Retry signups queue event on 400. 2017-10-05 23:14:19 -07:00
Umair Khan 19e2551e82 mypy: Change type to Dict for SignupWorker. 2017-10-05 23:14:19 -07:00
Umair Khan d95d34a66a Retry email failures in missed-message emails queue.
Fixes #6518.
2017-10-03 10:35:07 -07:00
Umair Khan 7d6ddaad26 mypy: Use Dict instead of Mapping in queues. 2017-10-03 10:35:07 -07:00
Greg Price 2a832386dc queue: Move more received-event logs to DEBUG level.
These can spew a lot of noise when starting the dev server, and aren't
in a super helpful form for normal server administration anyway.
2017-09-28 18:32:42 -07:00
Tim Abbott 630ecf9669 queue: Move user presence log events to DEBUG log level.
These are pretty noisy, and generally of little interest, since
presence requests come with a server log entry anyway.
2017-09-28 14:35:55 -07:00
rht 2ae47525d3 zerver/worker: Remove `import six`. 2017-09-27 17:06:16 -07:00
rht 41d8db373a zerver/worker: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Tim Abbott 84926ff321 slow_queries: Log slow queries even if not reported via Zulip.
This should help make these logs more useful for debugging on an
arbitary Zulip system.
2017-09-22 14:11:07 -07:00
Tim Abbott ec61a070b4 signups: Add basic logging for new signup processing. 2017-09-22 14:11:07 -07:00
Tim Abbott f78dd33037 invites: Add logging for newly sent invitation emails. 2017-09-22 14:11:07 -07:00
Rishi Gupta a7c8770f97 emails: Move enqueue_welcome_emails outside of signups queue.
The only thing this queue should do is sign you up for the newsletter, since
it is only populated if newsletter_data is not None.
2017-09-22 06:20:33 -07:00
Tim Abbott f706f657c0 signup: Fix invitation emails not being cleared properly.
Previously, invitation reminder emails were only being cleared after a
successful signup if newsletter_data was available, since that was the
circumstance in which we were calling the relevant queue processor
code.  Now, we (1) clear them when a human user finishes signing up
and (2) correctly clear them using the 'address' field of
ScheduleEmail, not user_id.
2017-09-21 06:15:11 -07:00
Steve Howell ba397b5109 Use user_ids, not full objects, in render path.
There is no reason for either render_incoming_message() or
render_markdown() to require full UserProfile objects just to
triage alert words.

By only asking for user_ids, we save extra queries in two
callpaths and we make it easier to start using user_ids in
do_send_messages().
2017-09-12 04:22:55 -07:00
neiljp (Neil Pilgrim) 116b8dd9c9 mypy: Set assign_queue() parameter queue_type to not be Optional. 2017-08-07 21:27:50 -07:00
Abhijeet Kaur 1deb58b178 bots: Add complete test-coverage for bot_lib.py file.
Also, add error handling for get_bot_handler instead of
throwing an assertion error.
2017-07-27 15:50:29 -07:00
Abhijeet Kaur 6f60c65a65 embedded bots: Add tests for verification of embedded bot services.
Add test to check if the embedded bot service being used is in the
registry or not.
Add test to check if the bot being added to the registry has a valid
bot corresponding to it.
Move 'get_bot_handler' to 'zerver/lib/bot_lib.py' as it is an independent
function, not related to the 'EmbeddedBotWorker' class that it was
previously a part of.
2017-07-24 17:14:14 -07:00
Elliott Jin 6a61a8a431 outgoing webhooks: Consolidate interfaces into lib/outgoing_webhook.py 2017-07-24 14:10:14 -07:00
vaibhav 87dcd5442a outgoing webhook system: Minor fixes.
This fixes some error message strings and skips converting request_data
into json. From now, conversion would be the responsibility of interface.
Also, base_url is now not passed into event structure.
2017-07-24 14:10:14 -07:00
Abhijeet Kaur 5980d420a8 Embedded bots: Fix minor errors to make embedded bots/service run.
Splitting bot_lib.py file into 2 files led to unnecessary
redirection of the code workflow. For an embedded bot/service to
send a reply, it was being redirected 3 times.

First, the code flow comes to "EmbeddedBotHandler" class to send
reply, then it goes to the common function in "zulip_bots/lib.py",
then it would come back to "EmbeddedBotHandler". Later on, if we
create an abstract class, from where the bot work flow would
directly hit and then from there it is classified into
EmbeddedBotHandler or ExternalBotHandler and accordingly it would
get redirected.

Now, first the bot flow goes to it's handler class External or
Embedded (where we pass that this is External or Embedded bot as
parameter) and then goes to a common point and then comes back to
the same class.
2017-07-20 10:22:52 -07:00
Eeshan Garg 148bb4db09 requirements: Update requirements/ to install bots/API packages.
This is required, since we just reorganized the python-zulip-api
repository into 3 packages.

A nice side effect is that we get to eliminate some now-unnecessary
code for editing sys.path.
2017-07-18 00:10:30 -07:00
Rishi Gupta 0f4b71b766 confirmation: Liberate get_link_for_object from ConfirmationManager. 2017-07-17 23:18:47 -07:00
Rishi Gupta 36dbb76516 emails: Rename clear_followup_emails_queue. 2017-07-17 16:05:38 -07:00
Rishi Gupta 5b3e6af2e5 emails: Remove only emails of the correct type when clearing queue. 2017-07-17 16:05:38 -07:00
Rishi Gupta f51bd898dc notifications: Change clear_followup_emails_queue to take a user_id. 2017-07-17 16:05:38 -07:00
Rishi Gupta 898269bbac email: Change send_email to raise exception on failure.
More in line with how we do error handling in the rest of Zulip.
2017-07-16 16:56:39 -07:00
Rishi Gupta eacdb0b302 emails: Change welcome emails to use to_user_id. 2017-07-16 16:56:39 -07:00
Rishi Gupta b0d325b8c5 emails: Change send_future_email to accept a to_user_id.
Also changes digest emails to use a to_user_id instead of a to_email.
2017-07-16 16:56:39 -07:00
Vishnu Ks ba4ea7dd8a notifications.py: Replace get_user_profile_by_email.
Replace with get_user.
2017-07-13 00:45:24 +05:30
Aditya Bansal d46cf59b0d pep8: Add compliance with rule E261 to worker/queue_processors.py. 2017-07-11 11:55:02 -07:00
Rishi Gupta 8fed9eeb75 confirmation: Make host a required argument in get_link_for_object.
Removes some lines of test from test_email_change.py. The relevant code path
was never utilized by the code itself, just by the tests.
2017-07-07 18:53:00 -07:00
Umair Khan ee6e07d13f message_sender: Use response_data variable.
response_data already contains the result of ujson.loads.
2017-07-07 16:33:36 -07:00
Umair Khan 0e8231d0f1 process_queue: Recover gracefully after PostgreSQL restart.
- For threaded workers:
Django's autoreloader catches SIGQUIT(3) to reload the program. If
a process being watched by autoreloader exits with status code 3,
reloader will restart the process. To reload, we send SIGUSR1(10)
signal from consumers to a handler in process_queue which then
exits with status code 3.

- For single worker per process:
Catch the SIGUSR1 and quit; supervisorctl will restart the worker
automatically.

Fixes #5512
2017-07-07 16:33:15 -07:00