Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
11,812 Commits 28 Branches 143 Tags 627 MiB
Commit Graph

262 Commits

Author SHA1 Message Date
Varshit 4e1060076d Purge 'from typing import *' from zerver/. 
This is a partial implementation of #636.
 2016-04-07 14:07:07 -07:00
Ashish 909b0635c8 Add management command to test sending email. 
Fixes: #622.

[With cleanups/doc tweaks by tabbott]
 2016-04-03 16:50:16 -07:00
Tim Abbott b7dcf2181f Add PEP-484 type annotations to management commands. 2016-04-03 15:40:23 -07:00
Ryan Moore 2fe0700f55 Update memcache -> remote cache in inline documentation. 2016-03-31 12:56:42 -07:00
Ryan Moore 1489f0992c s/fill_memcached_cache/fill_remote_cache/g 2016-03-31 12:54:29 -07:00
Tim Abbott 06b33da709 process_queue: Fix missing worker.setup() in single-threaded codepath. 2016-03-27 23:17:16 -07:00
Tim Abbott cd2348e9ae Run queue processers multithreaded in development. 
This change drops the memory used for Python processes run by Zulip in
development from about 1GB to 300MB on my laptop.

On the front of safety, http://pika.readthedocs.org/en/latest/faq.html
explains "Pika does not have any notion of threading in the code. If
you want to use Pika with threading, make sure you have a Pika
connection per thread, created in that thread. It is not safe to share
one Pika connection across threads.".  Since this code only connects
to rabbitmq inside the individual threads, I believe this should be
safe.

Progress towards #32.
 2016-03-20 18:04:24 -07:00
Tim Abbott 06e68d52ce Remove dead AsyncDjangoHandler field. 2016-03-20 16:53:13 -07:00
Tim Abbott 7fabfe9cb9 Add __repr__s for ClientDescriptor and AsyncDjangoHandler. 2016-03-20 16:53:13 -07:00
Tim Abbott 3f55e26a9f Fix Tornado memory leak with synchronously handled requests. 
The new Tornado handler tracking logic properly handled requests that
threw an exception or followed the RespondAsynchronously code path,
but did not properly de-allocated the handler in the syncronous case.

An easy reproducer for this is to load a new Zulip browser window;
that will leak 2 handler objects for the 2 synchronous requests made
from Django to Tornado as part of initial state fetching.
 2016-03-20 16:53:13 -07:00
Tim Abbott 320428052a Fix AsyncDjangoHandler view exception code path. 
The recent Tornado memory leak fix
(1396eb7022) didn't use the correct
variable name for the current handler ID, causing this cleanup code to
fail in the event that a view raised an exception.
 2016-03-19 22:50:35 -07:00
Eklavya Sharma 9e3c3e14f5 Partially apply Python 3 libmodernize.fixes.fix_dict_six. 
Refer to #256
 2016-03-19 15:52:58 -07:00
Tim Abbott 1396eb7022 Fix Tornado memory leak of handler objects. 
In 2ea0daab19, handlers were moved to
being tracked via the handlers_by_id dict, but nothing cleared this
dict, resulting in every handler object being leaked.  Since a Tornado
process uses a different handler object for every request, this
resulted in a significant memory leak.  We fix this by clearing the
handlers_by_id dict in the two code paths that would result in a
Tornado handler being de-allocated: the exception codepath and the
handler disconnect codepath.

Fixes #463.
 2016-03-17 18:33:59 -07:00
Reid Barton 9735025167 Refactor logic around restricted_to_domain. 
Add a function email_allowed_for_realm that checks whether a user with
given email is allowed to join a given realm (either because the email
has the right domain, or because the realm is open), and use it
whenever deciding whether to allow adding a user to a realm.

This commit is not intended to change any behavior, except in one case
where the Zulip realm's domain was not being converted to lowercase.
 2016-02-07 10:54:52 -05:00
Tim Abbott 1af7cbfd64 runtornado: Move more imports to the top of the file. 
This is needed for adding more specific type annotations but is
otherwise counterproductive since it increases the diff from the
original.
 2016-02-03 19:47:14 -08:00
Tim Abbott 2259ce62f8 tornado: Fix AsyncDjangoHandler get() and friends missing args/kwargs. 2016-02-03 19:47:14 -08:00
Tim Abbott d7b7ae2d0f rundjango: Fix confusing return line in log_message_monkey. 2016-02-03 19:31:45 -08:00
Tim Abbott 05a40f11b3 runtornado: Add explicit return None. 2016-02-03 19:31:45 -08:00
Tim Abbott 37e987e250 Make realm_filters --show's empty list output more consistent. 2016-02-03 19:29:44 -08:00
Tim Abbott fcc32b1093 Fix check_redis abuse of setting self.trim global. 2016-02-03 19:29:44 -08:00
Tim Abbott 7595e4b05f process_queue: Fix worker variable being accessed before initialization. 2016-02-03 19:29:44 -08:00
Tim Abbott aad3bff193 Harden style rule for % comprehensions and fix existing errors. 2016-02-02 23:08:19 -08:00
Tim Abbott 6528b18ad3 Switch all urllib/urlparse usage to six.moves.urllib. 
This provides Python 2+3 compatibility for our use of urllib.

Also add a test to avoid future regressions.
 2016-01-26 21:09:43 -08:00
Tim Abbott c7e3c3ce38 Look up client descriptors by handler_id. 
Previously, client descriptors were referenced directly from the
handler object.  Once we split the Tornado process into separate queue
and connection servers, these will no longer be in the same process,
so we need to reference them by ID instead.
 2016-01-26 20:57:25 -08:00
Tim Abbott ea6211c041 Allocate handler ids in AsyncDjangoHandler setup process. 2016-01-26 20:56:53 -08:00
Tim Abbott dfaf45b2b6 Wrap Django runserver to prevent spammy logging. 
Django's `manage.py runserver` prints a relatively low-information log
line for every request of the form:

[14/Dec/2015 00:43:06]"GET /static/js/message_list.js HTTP/1.0" 200 21969

This is pretty spammy, especially given that we already have our own
middleware printing a more detailed version of the same log lines:

2015-12-14 00:43:06,935 INFO     127.0.0.1       GET     200   0ms /static/js/message_list.js (unauth via ?)

Since runserver doesn't have support controlling whether these log
lines are printed, we wrap it with a small bit of code that silences
the log lines for 200/304 requests (aka the uninteresting ones).
 2015-12-25 16:23:57 -08:00
Tim Abbott a79e89b28f Cleanup remaining usage of % comprehensions without explicit tuples. 2015-12-05 15:29:42 -08:00
Tim Abbott 3e794351ab Remove expunge_* management commands. 
These management commands never worked and given that we removed the
retention_policy code that they call, it makes sense to remove them as
well.
 2015-11-11 21:34:39 -08:00
Walter Heck f145b01d91 Update create_user management command. 
notify_new_user was recently moved to zerver.lib.actions from
zerver.views and this wasn't properly updated. This would give an
error when doing a `manage.py create_user` from the command line.
 2015-11-08 19:41:00 -08:00
Tim Abbott 3b185ad4de Apply Python 3 futurize transform libmodernize.fixes.fix_input_six. 2015-11-01 09:26:17 -08:00
Tim Abbott f3783fb4a1 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import. 2015-11-01 09:26:16 -08:00
Tim Abbott 43abd83d1c Apply Python 3 futurize transform lib2to3.fixes.fix_ws_comma. 2015-11-01 09:26:14 -08:00
Tim Abbott 06f6ee6566 Apply Python 3 futurize transform lib2to3.fixes.fix_idioms. 2015-11-01 09:25:47 -08:00
Tim Abbott 5ce6a3c8f9 Apply Python 3 futurize transform lib2to3.fixes.fix_funcattrs. 2015-11-01 08:09:54 -08:00
Tim Abbott 8c34c40924 Apply Python 3 futurize transform lib2to3.fixes.fix_except. 2015-11-01 08:08:33 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott 71a06d58de Convert uses of Realm.objects.get() to get_realm(). 
get_realm is better in two key ways:
* It uses memcached to fetch the data from the cache and thus is faster.
* It does a case-insensitive query and thus is more safe.
 2015-10-15 09:16:58 -04:00
Tim Abbott 419d31a007 Expand documentation for the LDAP auth integration. 
Fixes #134, #173.
 2015-10-15 09:16:58 -04:00
Tim Abbott 858d0a984b Move API super users configuration into the database. 
(imported from commit 3cc702f93e7252b42930dba4bde93a915b6dbf44)
 2015-09-20 11:13:20 -07:00
Tim Abbott 3b84a9a010 Modernize management command for deactivating a realm. 
(imported from commit 9ef681193ff04e447c0afbba66187357624d98f3)
 2015-09-19 19:10:27 -07:00
Tim Abbott a5c1fc6d76 Remove unnecessary import_realm_filters.py management command. 
It's superceded by realm_filters.py.

(imported from commit b53c8bc4daf965387f5ba88711dd7ece2db1dfd9)
 2015-09-19 19:05:16 -07:00
Tim Abbott 930c64df8a Move zulip.com-related statsd configuration out of main settings.py. 
This also removes the convenient way to run statsd in the Dev VM,
because we don't anticipate anyone doing that.  It's just 2 lines of
config to configure it anyway:

STATSD_HOST = 'localhost'
STATSD_PREFIX = 'user'

(imported from commit 5b09422ee0e956bc7f336dd1e575634380b8bfa2)
 2015-08-22 13:49:43 -07:00
Tim Abbott 83a653efce Change semantics of ZULIP_COM to include ZULIP_COM_STAGING machines. 
(imported from commit 2e84c756b4ac41c13d619ab6ceb99e347d1f48cb)
 2015-08-21 10:41:14 -07:00
Tim Abbott a8261dd6ac Remove Deployment initialization from Development. 
(imported from commit 4596fcf480c10c88496cd6506edb0a6e702d823c)
 2015-08-21 10:40:57 -07:00
Tim Abbott 16e0f21687 Fix a bunch of Zulip Voyager related comments. 
(imported from commit 176e875cbc310ba00d6137930ddb718bd68f1866)
 2015-08-21 10:40:54 -07:00
David Roe e3f38acbce Enterprise => Voyager. 
(imported from commit 41b9a67301aeaf5fd40bbbb8f34a326ca98431fd)
 2015-08-21 10:33:35 -07:00
David Roe 3f7cb34b00 enterprise => voyager 
(imported from commit 04be792bb480d5e5db1c91d296d1000cf1682571)
 2015-08-21 10:33:35 -07:00
David Roe edf7e732a2 ENTERPRISE => VOYAGER. 
(imported from commit 4f8080b9f506a87ca40bef32e39de5218cba916a)
 2015-08-21 10:33:35 -07:00
David Roe d702ddb3cf DEPLOYED => PRODUCTION 
(imported from commit a7d365c07916c5103fc721c712db94f9af977640)
 2015-08-21 10:33:35 -07:00
Reid Barton ae0ae3dde8 Django 1.8: declare positional arguments in management commands 
(imported from commit d9efca1376de92c8187d25f546c79fece8d2d8c6)
 2015-08-20 23:35:40 -07:00
Reid Barton 9ded218950 Django 1.7 compatibility: miscellaneous changes 
(imported from commit 1f63d936c3d195acd033bb42fddacd240af3889b)
 2015-08-20 23:01:26 -07:00
Nemanja Stanarevic d9f111ec28 Added import_dump command to manage.py to import JSON database dumps 
(imported from commit abd410e5299f15c1a5851e87d18664b59837fbb0)
 2015-08-20 21:19:57 -07:00
David Roe 809efc4f2b Add realm option to show or not show digest e-mail. 
(imported from commit e9cfe519aa9f2857a1dcc7c75f5e1889834fcb86)
 2015-08-20 15:29:45 -07:00
Tim Abbott 4f1388706c Disable digests and Missed Message notifications for internal realm. 
(imported from commit 24a25bcf6653c674ac8a34324a4674e438f0af38)
 2015-02-05 18:40:13 -08:00
Jason Michalski 2a5ec20295 Add support for one time use email address 
The one time use address are a unique token which maps to stored stated
in redis. We store the user_id, recipient_id, and subject. When an email
is received at this address it is sent to the stored recipient by the
stored user. Anyone with this address can send a single message as this
user.

(imported from commit 4219417bdc30c033a6cf7a0c7c0939f7d0308144)
 2014-10-06 15:27:54 -07:00
Tim Abbott 0494e40c39 Merge zerver/tornado_callbacks.py into zerver/lib/event_queue.py. 
It had stopped being a coherently distinct component a while ago.

(imported from commit 0617957bcfe8dcaf69143c88a96ddd51ecb31a98)
 2014-04-23 17:22:31 -07:00
Jessica McKellar f9d5a02710 Fix bug that hid digest email settings box for customers. 
(imported from commit 6c1e3f53998eca0232fbfe66c62ab87b009bccd8)
 2014-02-27 20:25:26 -05:00
acrefoot 233cfc2593 revert changes to deliver_email.py introduced in dc5e78cccb 
I'm pretty sure this was changed for purposes of testing

(imported from commit b4b1224a5812ceb248ded3755c0b1801797c1beb)
 2014-02-10 15:22:20 -05:00
Jessica McKellar bb60e96910 Enable digest emails for CUSTOMER4. 
(imported from commit 08b4adb4fd95fdfd661ccbd9cdb5782bf0d743f0)
 2014-02-10 13:53:39 -05:00
Luke Faraone 0f96d2229e email-mirror: Also accept original recipient via argv 
(imported from commit 9a29fe6906713e75ee208c924b67211575c011f2)
 2014-01-31 15:33:15 -05:00
Steve Howell aca5586cc0 Fix command line help for ./manage.py show_admins 
(imported from commit d5de8832214db7b8a754f155b225e706b655260b)
 2014-01-28 17:40:00 -05:00
Leo Franchi 0917a27b79 Document realm filter regexp oddities 
(imported from commit 987d8ea679f40c31f2fe2280b6d585f82b20e606)
 2014-01-27 13:43:55 -05:00
Tim Abbott 6d64427c46 Fix email mirror. 
(imported from commit 73c109db6e821b139203f662023ed780fcee0f24)
 2014-01-24 18:11:20 -05:00
Steve Howell d431206a3f Have the "knight" command use actions.do_change_is_admin. 
Using the higher level function has the benefit of sending
notifications to clients.

(imported from commit d680e3305b216fec3462479c696e572d008ee4c2)
 2014-01-22 14:43:28 -05:00
Luke Faraone b1cf9c3e25 Rewrite email-mirror documentation. 
(imported from commit 4af076b3063d7b82657765c12cf4dba0248cbf9f)
 2014-01-15 16:07:19 -05:00
Luke Faraone 895b7dc9b1 Accept emails on stdin in email-mirror and queue for later processing 
(imported from commit 7efccc294f87d3661e50f10962a80071b91e3879)
 2014-01-15 16:07:18 -05:00
Luke Faraone 1005e2ae7c Use module docstring for usage information 
(imported from commit 7c1cbf68eca4e3f31c75bfe13b50654e76be0df3)
 2014-01-15 16:07:18 -05:00
Jessica McKellar cf5b7dc0e4 Don't send digests to CUSTOMER29 teams. 
(imported from commit c1dbb6607647e65082535d4e37d50d7842f8758f)
 2014-01-15 16:07:17 -05:00
Jessica McKellar f85d45a781 Add a management command to bulk turn off digests. 
(imported from commit 0ffb565ecc9be219807ae9a45abb7b0e3e940204)
 2014-01-15 16:07:17 -05:00
Jessica McKellar d3f6c4ef36 Log the actual sending of digest emails in addition to queuing possible ones. 
(imported from commit e43ac59e71620fe715db462347db228d5a950aed)
 2014-01-10 21:38:59 -05:00
Jessica McKellar 33e8822165 Don't send digest e-mails to users in deactivated realms. 
(imported from commit dca53af9beac80242a5a96857595e99ccb75cf77)
 2014-01-07 20:24:21 -05:00
Leo Franchi 76892c52aa Send realm filters to clients 
(imported from commit a3ba31701baa4d515ea3bf36c39d7255f1dd20d6)
 2014-01-07 17:33:33 -05:00
Jessica McKellar 5f7c3a959e Enable digest emails for all but special-case domains. 
(imported from commit 6f5c8c6c1dcd0f7ba040e86edd7b1bd41ac0dc1e)
 2014-01-06 17:32:56 -05:00
Jessica McKellar b082c99267 digest: send e-mails on Fridays. 
(imported from commit 2080b98b4d67cfdefa124e2115b2c67b8ef975ce)
 2014-01-06 17:32:55 -05:00
Luke Faraone df4b2d9f85 Break out non-Twisted email mirror code into separate library 
(imported from commit 45c3e70ba81cdb44f1e9db4f162e5f2d2fe5ead6)
 2014-01-06 13:36:53 -05:00
Luke Faraone 266c0f4886 Remove duplicate validation in email_mirror 
We already have a try-except earlier in the file about email_gateway_user, so we don't
need to check for it again.

(imported from commit 2d9fa357fab2605916c5c5cb61961c0a121b1211)
 2014-01-06 13:36:53 -05:00
Jessica McKellar a20894d847 digest: fix generator bug not producing correct deployment comparison. 
(imported from commit 33c34d71320fa1f8cd095a6d9f10b2397eeeda2f)
 2013-12-31 16:34:52 -05:00
Jessica McKellar 61103d8349 digest: On an app server, only send digest emails for domains it manages. 
(imported from commit 9aea636a19eb6fdefacdc7dc393bf075a05862c5)
 2013-12-20 12:50:23 -05:00
Jessica McKellar a8cd91d6c7 digest: send emails if you've been active for slightly less than a day. 
That way if all you do is briefly check Zulip because you got the
email, we'll send you another one tomorrow.

(imported from commit fcbbd264c5e5fea7352f0fee6989e000af7b7bed)
 2013-12-17 10:47:17 -05:00
Jessica McKellar 6b64ef6be3 digest: log when we enqueue emails. 
(imported from commit 1b34e05bc2eebedb818673a4c9a3944e1db15c6d)
 2013-12-17 10:47:17 -05:00
Jessica McKellar 71f5b4e0ae Enable digest emails for zulip.com. 
(imported from commit f676f96c2b516b33269b8f5c72deaf5c985e8ccd)
 2013-12-17 10:47:17 -05:00
Tim Abbott b8579c6848 Disconnect handlers if the client closes a connection. 
Tested using the following procedure (run-dev.py won't pass through
the client connect closing)

tabbott@monastery:~/zulip$ curl http://localhost:9991/api/v1/register  -u email:key  -d 'event_types=["message"]'
{"msg":"","max_message_id":2259,"last_event_id":-1,"result":"success","queue_id":"1386884005:0"}
tabbott@monastery:~/zulip$ curl -G http://localhost:9993/api/v1/events  -u email:key -d "last_event_id=0" -d "queue_id=1386884005:0"
(then hit ctrl-C)

(imported from commit 3c4f3d5caac97b3de53da994ff9cd9ef67b2b9ea)
 2013-12-12 16:52:25 -05:00
Leo Franchi 4d30756e72 [manual] Add a manage.py command to import realm filters 
This must be run manually on staging after deployment. Once it has been run,
it can be deleted. It only needs to be run on staging, not prod.

(imported from commit 79252c23ba8cda93500a18aa7b02575f406dd379)
 2013-12-11 14:39:10 -05:00
Leo Franchi e377be96d4 Add a management command to set realm filters. 
(imported from commit 6590d7eb58ea6e6afc15104dd75ad56b832858fa)
 2013-12-11 14:39:10 -05:00
Luke Faraone 1b5c1ac021 Update style of client strings. 
(imported from commit 1516461cf53b2715de68e01f16bb8a8cc33c48ad)
 2013-12-09 11:47:52 -05:00
Tim Abbott e5be713103 Clean up EXTERNAL_API_HOST usage and defaults. 
We now have 2 variablse:
EXTERNAL_API_PATH: e.g. staging.zulip.com/api
EXTERNAL_API_URI: e.g. https://staging.zulip.com/api

The former is primarily needed for certain integrations.

(imported from commit 3878b99a4d835c5fcc2a2c6001bc7eeeaf4c9363)
 2013-12-04 15:10:54 -05:00
Waseem Daher 89844c6ced Revise day1/day2 emails. 
(imported from commit dc5e78cccb22b024be46b3066d44f09ea1b53345)
 2013-12-02 11:15:13 -05:00
Luke Faraone 0c142015ea Tool to test LDAP functionality 
(imported from commit c4b4d8b25ba3afd333faf09bc232da2f92a57b7e)
 2013-11-25 17:44:47 -05:00
Luke Faraone 4a8ec7b357 Management command to update names from the LDAP database 
(imported from commit 1e1b12c055926899fdca3e484df6c9437c800c6c)
 2013-11-25 17:44:47 -05:00
Tim Abbott 7cb261a0ed Set default streams to engineering,social when creating realms. 
(imported from commit 4e174fec162be4138da099edfc4cfb8d5662b7f1)
 2013-11-25 14:00:28 -05:00
Tim Abbott ca8225cf47 [manual] Add endpoint to cleanup a finished events queue. 
This requires a puppet apply on each of staging and prod0 to update
the nginx configuration to support the new URL when it is deployed.

(imported from commit a35a71a563fd1daca0d3ea4ec6874c5719a8564f)
 2013-11-20 18:34:15 -05:00
Tim Abbott f3fa78d30c Rename do_[rd]activate to do_[rd]activate_user. 
For similarity with do_activate_user.

(imported from commit 2874af84dbe1ce666de10e0d8ffe2daa8bad9731)
 2013-11-16 11:35:26 -05:00
acrefoot 9616ea7afd remove enqueue_followup_emails management command 
(imported from commit 20ce0b698cf795a6e1c7ff5ba7b546901e1fd8a4)
 2013-11-15 21:31:37 -05:00
acrefoot e77d534e78 remove ALLOW_ARBITRARY_SENDERS_LOCAL_EMAIL 
(imported from commit 7a374959516d8505685466293f146d95094a7ea7)
 2013-11-15 21:31:37 -05:00
Tim Abbott c24addf2d5 Build internal bots in the zulip.com domain. 
Otherwise do_create_realm can't actually send the notifications.

(imported from commit 4fa9a53f1d3d8a2e26d7b89401e7dfa77f2f7533)
 2013-11-14 11:16:16 -05:00
Tim Abbott e2537ade44 Create the admin realm using do_create_realm. 
That way, we get the first-user message sent automatically.

(imported from commit 70be26a8152676055a94c363706c35964284a3de)
 2013-11-14 11:16:16 -05:00
Tim Abbott af2da58fd0 Set default default streams for enterprise installations. 
(imported from commit 87b9fdc6ef19a88a2abda4ba5cdfe2a5b35d581d)
 2013-11-13 21:42:21 -05:00
Tim Abbott 4078b8bb6c Move set_default_streams from zilencer to zerver. 
We'll want to use it for setting up local server instances.

(imported from commit 47c60c349ba3cebad58506f439013def4c40708b)
 2013-11-13 21:08:14 -05:00
Tim Abbott bf991dac94 enterprise: Create feedback_bot as a bot. 
(imported from commit 8bd985f82aa80380188a644869c55bc54b6f7977)
 2013-11-13 17:26:30 -05:00
acrefoot 6d38285a2e fixup supervisor oops related to postmaster config 
(imported from commit 8b5c39f0d13abb5e1def9f88a2ab82cfa67b42f6)
 2013-11-13 17:15:55 -05:00