Commit Graph

105 Commits

Author SHA1 Message Date
Tim Abbott 8da7976058 queues: Add new system for managing rabbitmq per-queue work.
Our lists of rabbitmq queues was likely to end up out of date, since
there was nothing enforcing that the various lists of queues were
correct or the same as each other.
2017-02-19 16:18:37 -08:00
Tim Abbott 6ad883fb61 docs: Update queuing documentation for new templates. 2017-02-19 13:12:34 -08:00
Tim Abbott 9072d3c350 socket: Transmit HTTP_USER_AGENT for websockets sending.
This significantly simplify the logic for our logging process, making
it the case that websockets message sending requests always are logged
as having the exact same client as a normal AJAX request from that
server.
2017-02-17 15:19:21 -08:00
Tim Abbott 5c911c44d6 MessageSenderWorker: Reformat environment setup. 2017-02-17 15:00:57 -08:00
Tim Abbott e69c4458c6 errors: Use hostname to report deployment. 2017-01-28 13:07:09 -08:00
Tim Abbott ae35bde3a6 errors: Enable error reports without zilencer enabled. 2017-01-27 10:27:41 -08:00
Tim Abbott 04ae2977fa errors: Add logging for consuming processed error reports.
This should make it a big easier to debug issues with this system.
2017-01-27 10:27:41 -08:00
Tim Abbott dd9e0b8463 errors: Move do_report_error into zerver/lib/. 2017-01-23 23:18:28 -08:00
Tim Abbott d6e38e2a5c lint: Clean up E123 PEP-8 rule. 2017-01-23 21:34:26 -08:00
Tim Abbott 01e0acb282 render_incoming_message: Require a realm object. 2017-01-21 21:37:57 -08:00
Tim Abbott 99c5563bc6 internal_send_message: Make realm argument mandatory.
A lot of care has been taken to ensure we're using the realm that the
message is being sent into, not the realm of the sender, to correctly
handle the logic for cross-realm bot users such as the notifications
bot.
2017-01-21 21:37:30 -08:00
Tim Abbott 46fc136943 feedback-bot: Add logging for successfully submitted feedback. 2017-01-06 21:42:16 -08:00
Rishi Gupta 13d1a636d5 queue_processors.SignupWorker: Upgrade to MailChimp API v3.0.
Removes the dependence on postmonkey, which is a wrapper around
MailChimp API v1.3. MailChimp recommends using their v3.0 API directly,
rather than through a wrapper library.
2016-12-29 16:16:18 -08:00
Rishi Gupta c61938f719 SignupWorker: Use dictionary indexing instead of get for name and email.
We always pass in EMAIL in data, and NAME in merge_vars.
2016-12-29 16:16:18 -08:00
Steve Howell 3d804de962 Fix FeedbackBot queue processor.
Recent changes to the API removed Client.register(), and
this change restores the correct API call, although the
codepath this affects is probably ready for eventual
deprecation.
2016-12-28 12:12:34 -08:00
Igor Tokarev c93f1d4eda Add oembed/Open Graph/Meta tags data retrieval from inline links.
This change adds support for displaying inline open graph previews for
links posted into Zulip.

It is designed to interact correctly with message editing.

This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.

By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.

Eventually, we may want to make this manageable via a (set of?)
per-realm settings.  E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
2016-12-07 17:40:18 -08:00
AZtheAsian 1ba150fa85 pep8: Fix E203 violations 2016-12-01 20:37:57 -08:00
Rafid Aslam 7a2282986a pep8: Fix E225 pep8 violations. 2016-11-28 15:21:15 -08:00
Tim Abbott 282f74609c tornado: Move socket code to zerver/tornado/. 2016-11-26 22:29:27 -08:00
paxapy 6d93b3b60c lint: Fix E703 pep8 violations. 2016-11-09 15:18:35 -08:00
paxapy ff1e97603d context_processors: use a common context for emails.
Fixes #1611.
2016-11-08 11:37:16 -08:00
Tim Abbott be3e9e3a05 Fix regression breaking websocket message sending.
In fe1ba6f3eb, we change our auth
decorators etc. to use request.POST/request.GET rather than the (now
removed request.REQUEST).  This broke sending messages via our
websockets codepath, because we failed to update the artificial
requests we generate to use request._post as well.
2016-11-05 11:33:04 -07:00
Tim Abbott 1a88ca4da1 queue_processors: Fix feedback bot sender.
This fixes an issue where emails sent by feedback-bot had a forged
sender and thus sometimes had delivery issues.
2016-10-28 11:04:08 -07:00
Tim Abbott 65321b77ac socket: Switch localhost to 127.0.0.1.
It does the same thing, but is a bit more robust to weird DNS
configurations.
2016-09-13 10:01:57 -07:00
Rishi Gupta 64179b2fd2 confirmation: Use realm host in activation URLs.
This is preparation for merging the subdomains feature.
2016-08-18 20:12:36 -07:00
Tim Abbott 4fbb8c3eee templates: Add new context variables to all templates.
This adds a few new helpful context variables that we can use to
compute URLs in all of our templates:
* external_uri_scheme: http(s)://
* server_uri: The base URL for the server's canonical name
* realm_uri: The base URL for the user's realm

This is preparatory work for making realm_uri != server_uri when we
add support for subdomains.
2016-08-13 16:27:35 -07:00
Tim Abbott 95aee366c8 Enable local browser error reporting if ZILENCER_ENABLED. 2016-08-11 13:49:18 -07:00
Tim Abbott 88550edd9e Annotate queue_processors.py. 2016-08-04 15:57:03 -07:00
Eklavya Sharma b98b049377 Fix string handling in email_mirror queue_processor. 2016-07-28 14:52:09 -07:00
Tim Abbott 3ea4cbb5c3 settings: Extract settings.VERBOSE_SUPPORT_OFFERS. 2016-07-19 15:28:41 -07:00
Eklavya Sharma 2080ff6c2a Decode response.content everywhere except in tests.
response.content is binary data, but code usually assumes it to
be text.  Fix this by decoding response.content where required.
Don't do this in tests yet.
2016-07-12 14:06:29 -07:00
Eklavya Sharma 1f3ce7cf38 zerver/worker/queue_processors.py: Open file in binary.
Open a file in binary mode instead of text mode.
2016-07-09 17:54:54 -07:00
Varshit 4e1060076d Purge 'from typing import *' from zerver/.
This is a partial implementation of #636.
2016-04-07 14:07:07 -07:00
Tim Abbott ee8be22160 Add PEP-484 type annotations to zerver/worker/. 2016-04-03 15:40:23 -07:00
Tim Abbott 2059f650ab Add PEP-484 type annotations to zerver/lib/. 2016-04-03 15:40:23 -07:00
Tim Abbott cd2348e9ae Run queue processers multithreaded in development.
This change drops the memory used for Python processes run by Zulip in
development from about 1GB to 300MB on my laptop.

On the front of safety, http://pika.readthedocs.org/en/latest/faq.html
explains "Pika does not have any notion of threading in the code. If
you want to use Pika with threading, make sure you have a Pika
connection per thread, created in that thread. It is not safe to share
one Pika connection across threads.".  Since this code only connects
to rabbitmq inside the individual threads, I believe this should be
safe.

Progress towards #32.
2016-03-20 18:04:24 -07:00
Tim Abbott a46647a87a python3: Mark certain strings as unicode strings.
This is required in Python 3 since these strings are combined with
other unicode strings.
2016-03-08 09:14:11 -08:00
Tim Abbott 1d008576f2 QueueProcessingWorker: Add stub consume function and queue_name.
QueueProcessingWorker will now throw errors if there's a misconfigured
queue processor.
2016-02-03 19:47:14 -08:00
Tim Abbott 1ee0706511 Fix missing UTF-8 encoding. 2016-02-03 19:29:30 -08:00
Tim Abbott 5bacda3662 python3: Fix usage of .keys()/.values() to handle iterators.
This fixes the places where we use the result of .keys(), .items(),
and .values() that wouldn't work with an iterator to wrap them with
list().
2016-01-26 21:11:25 -08:00
Tim Abbott 757e89260e Migrate use of StringIO to Python 2+3 compatible six.moves.cStringIO.
And add a check for additional usage of the original StringIO module.
2016-01-26 21:09:43 -08:00
Allie Jones 1bd1291f3c Replace json/send_message endpoint usage with json/message. 2015-12-12 18:14:08 -08:00
Tim Abbott be112d2c9d SlowQueryWorker: Don't reference potentially unset STATSD_PREFIX. 2015-11-16 21:10:06 -08:00
Tim Abbott 8c34c40924 Apply Python 3 futurize transform lib2to3.fixes.fix_except. 2015-11-01 08:08:33 -08:00
Tim Abbott 1156a82297 Make ErrorReporter run successfully on Zulip voyager.
(imported from commit 8ab2fb57fb3e580775cb8959e3b8b5a1277ea9d3)
2015-09-25 04:09:16 -07:00
Tim Abbott ae4cb6df35 Make python-postmonkey not a required package.
This is in some ways a regression, but because we don't have
python-postmonkey packaged right now, this is required to make the
Zulip production installation process work on Trusty.

(imported from commit 539d253eb7fedc20bf02cc1f0674e9345beebf48)
2015-09-25 00:22:14 -07:00
David Roe 3f7cb34b00 enterprise => voyager
(imported from commit 04be792bb480d5e5db1c91d296d1000cf1682571)
2015-08-21 10:33:35 -07:00
David Roe edf7e732a2 ENTERPRISE => VOYAGER.
(imported from commit 4f8080b9f506a87ca40bef32e39de5218cba916a)
2015-08-21 10:33:35 -07:00
David Roe d702ddb3cf DEPLOYED => PRODUCTION
(imported from commit a7d365c07916c5103fc721c712db94f9af977640)
2015-08-21 10:33:35 -07:00
Tim Abbott 3e61714035 Make DEPLOYMENT_ROLE_KEY no longer required.
(imported from commit 046ce610d847cc987d3f6d584ccc0bc84b55df84)
2015-08-21 09:01:58 -07:00