Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
20,323 Commits 28 Branches 143 Tags 605 MiB
Commit Graph

497 Commits

Author SHA1 Message Date
Tim Abbott 2259ce62f8 tornado: Fix AsyncDjangoHandler get() and friends missing args/kwargs. 2016-02-03 19:47:14 -08:00
Tim Abbott d7b7ae2d0f rundjango: Fix confusing return line in log_message_monkey. 2016-02-03 19:31:45 -08:00
Tim Abbott 05a40f11b3 runtornado: Add explicit return None. 2016-02-03 19:31:45 -08:00
Tim Abbott 37e987e250 Make realm_filters --show's empty list output more consistent. 2016-02-03 19:29:44 -08:00
Tim Abbott fcc32b1093 Fix check_redis abuse of setting self.trim global. 2016-02-03 19:29:44 -08:00
Tim Abbott 7595e4b05f process_queue: Fix worker variable being accessed before initialization. 2016-02-03 19:29:44 -08:00
Tim Abbott aad3bff193 Harden style rule for % comprehensions and fix existing errors. 2016-02-02 23:08:19 -08:00
Tim Abbott 6528b18ad3 Switch all urllib/urlparse usage to six.moves.urllib. 
This provides Python 2+3 compatibility for our use of urllib.

Also add a test to avoid future regressions.
 2016-01-26 21:09:43 -08:00
Tim Abbott c7e3c3ce38 Look up client descriptors by handler_id. 
Previously, client descriptors were referenced directly from the
handler object.  Once we split the Tornado process into separate queue
and connection servers, these will no longer be in the same process,
so we need to reference them by ID instead.
 2016-01-26 20:57:25 -08:00
Tim Abbott ea6211c041 Allocate handler ids in AsyncDjangoHandler setup process. 2016-01-26 20:56:53 -08:00
Tim Abbott dfaf45b2b6 Wrap Django runserver to prevent spammy logging. 
Django's `manage.py runserver` prints a relatively low-information log
line for every request of the form:

[14/Dec/2015 00:43:06]"GET /static/js/message_list.js HTTP/1.0" 200 21969

This is pretty spammy, especially given that we already have our own
middleware printing a more detailed version of the same log lines:

2015-12-14 00:43:06,935 INFO     127.0.0.1       GET     200   0ms /static/js/message_list.js (unauth via ?)

Since runserver doesn't have support controlling whether these log
lines are printed, we wrap it with a small bit of code that silences
the log lines for 200/304 requests (aka the uninteresting ones).
 2015-12-25 16:23:57 -08:00
Tim Abbott a79e89b28f Cleanup remaining usage of % comprehensions without explicit tuples. 2015-12-05 15:29:42 -08:00
Tim Abbott 3e794351ab Remove expunge_* management commands. 
These management commands never worked and given that we removed the
retention_policy code that they call, it makes sense to remove them as
well.
 2015-11-11 21:34:39 -08:00
Walter Heck f145b01d91 Update create_user management command. 
notify_new_user was recently moved to zerver.lib.actions from
zerver.views and this wasn't properly updated. This would give an
error when doing a `manage.py create_user` from the command line.
 2015-11-08 19:41:00 -08:00
Tim Abbott 3b185ad4de Apply Python 3 futurize transform libmodernize.fixes.fix_input_six. 2015-11-01 09:26:17 -08:00
Tim Abbott f3783fb4a1 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import. 2015-11-01 09:26:16 -08:00
Tim Abbott 43abd83d1c Apply Python 3 futurize transform lib2to3.fixes.fix_ws_comma. 2015-11-01 09:26:14 -08:00
Tim Abbott 06f6ee6566 Apply Python 3 futurize transform lib2to3.fixes.fix_idioms. 2015-11-01 09:25:47 -08:00
Tim Abbott 5ce6a3c8f9 Apply Python 3 futurize transform lib2to3.fixes.fix_funcattrs. 2015-11-01 08:09:54 -08:00
Tim Abbott 8c34c40924 Apply Python 3 futurize transform lib2to3.fixes.fix_except. 2015-11-01 08:08:33 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott 71a06d58de Convert uses of Realm.objects.get() to get_realm(). 
get_realm is better in two key ways:
* It uses memcached to fetch the data from the cache and thus is faster.
* It does a case-insensitive query and thus is more safe.
 2015-10-15 09:16:58 -04:00
Tim Abbott 419d31a007 Expand documentation for the LDAP auth integration. 
Fixes #134, #173.
 2015-10-15 09:16:58 -04:00
Tim Abbott 858d0a984b Move API super users configuration into the database. 
(imported from commit 3cc702f93e7252b42930dba4bde93a915b6dbf44)
 2015-09-20 11:13:20 -07:00
Tim Abbott 3b84a9a010 Modernize management command for deactivating a realm. 
(imported from commit 9ef681193ff04e447c0afbba66187357624d98f3)
 2015-09-19 19:10:27 -07:00
Tim Abbott a5c1fc6d76 Remove unnecessary import_realm_filters.py management command. 
It's superceded by realm_filters.py.

(imported from commit b53c8bc4daf965387f5ba88711dd7ece2db1dfd9)
 2015-09-19 19:05:16 -07:00
Tim Abbott 930c64df8a Move zulip.com-related statsd configuration out of main settings.py. 
This also removes the convenient way to run statsd in the Dev VM,
because we don't anticipate anyone doing that.  It's just 2 lines of
config to configure it anyway:

STATSD_HOST = 'localhost'
STATSD_PREFIX = 'user'

(imported from commit 5b09422ee0e956bc7f336dd1e575634380b8bfa2)
 2015-08-22 13:49:43 -07:00
Tim Abbott 83a653efce Change semantics of ZULIP_COM to include ZULIP_COM_STAGING machines. 
(imported from commit 2e84c756b4ac41c13d619ab6ceb99e347d1f48cb)
 2015-08-21 10:41:14 -07:00
Tim Abbott a8261dd6ac Remove Deployment initialization from Development. 
(imported from commit 4596fcf480c10c88496cd6506edb0a6e702d823c)
 2015-08-21 10:40:57 -07:00
Tim Abbott 16e0f21687 Fix a bunch of Zulip Voyager related comments. 
(imported from commit 176e875cbc310ba00d6137930ddb718bd68f1866)
 2015-08-21 10:40:54 -07:00
David Roe e3f38acbce Enterprise => Voyager. 
(imported from commit 41b9a67301aeaf5fd40bbbb8f34a326ca98431fd)
 2015-08-21 10:33:35 -07:00
David Roe 3f7cb34b00 enterprise => voyager 
(imported from commit 04be792bb480d5e5db1c91d296d1000cf1682571)
 2015-08-21 10:33:35 -07:00
David Roe edf7e732a2 ENTERPRISE => VOYAGER. 
(imported from commit 4f8080b9f506a87ca40bef32e39de5218cba916a)
 2015-08-21 10:33:35 -07:00
David Roe d702ddb3cf DEPLOYED => PRODUCTION 
(imported from commit a7d365c07916c5103fc721c712db94f9af977640)
 2015-08-21 10:33:35 -07:00
Reid Barton ae0ae3dde8 Django 1.8: declare positional arguments in management commands 
(imported from commit d9efca1376de92c8187d25f546c79fece8d2d8c6)
 2015-08-20 23:35:40 -07:00
Reid Barton 9ded218950 Django 1.7 compatibility: miscellaneous changes 
(imported from commit 1f63d936c3d195acd033bb42fddacd240af3889b)
 2015-08-20 23:01:26 -07:00
Nemanja Stanarevic d9f111ec28 Added import_dump command to manage.py to import JSON database dumps 
(imported from commit abd410e5299f15c1a5851e87d18664b59837fbb0)
 2015-08-20 21:19:57 -07:00
David Roe 809efc4f2b Add realm option to show or not show digest e-mail. 
(imported from commit e9cfe519aa9f2857a1dcc7c75f5e1889834fcb86)
 2015-08-20 15:29:45 -07:00
Tim Abbott 4f1388706c Disable digests and Missed Message notifications for internal realm. 
(imported from commit 24a25bcf6653c674ac8a34324a4674e438f0af38)
 2015-02-05 18:40:13 -08:00
Jason Michalski 2a5ec20295 Add support for one time use email address 
The one time use address are a unique token which maps to stored stated
in redis. We store the user_id, recipient_id, and subject. When an email
is received at this address it is sent to the stored recipient by the
stored user. Anyone with this address can send a single message as this
user.

(imported from commit 4219417bdc30c033a6cf7a0c7c0939f7d0308144)
 2014-10-06 15:27:54 -07:00
Tim Abbott 0494e40c39 Merge zerver/tornado_callbacks.py into zerver/lib/event_queue.py. 
It had stopped being a coherently distinct component a while ago.

(imported from commit 0617957bcfe8dcaf69143c88a96ddd51ecb31a98)
 2014-04-23 17:22:31 -07:00
Jessica McKellar f9d5a02710 Fix bug that hid digest email settings box for customers. 
(imported from commit 6c1e3f53998eca0232fbfe66c62ab87b009bccd8)
 2014-02-27 20:25:26 -05:00
acrefoot 233cfc2593 revert changes to deliver_email.py introduced in dc5e78cccb 
I'm pretty sure this was changed for purposes of testing

(imported from commit b4b1224a5812ceb248ded3755c0b1801797c1beb)
 2014-02-10 15:22:20 -05:00
Jessica McKellar bb60e96910 Enable digest emails for CUSTOMER4. 
(imported from commit 08b4adb4fd95fdfd661ccbd9cdb5782bf0d743f0)
 2014-02-10 13:53:39 -05:00
Luke Faraone 0f96d2229e email-mirror: Also accept original recipient via argv 
(imported from commit 9a29fe6906713e75ee208c924b67211575c011f2)
 2014-01-31 15:33:15 -05:00
Steve Howell aca5586cc0 Fix command line help for ./manage.py show_admins 
(imported from commit d5de8832214db7b8a754f155b225e706b655260b)
 2014-01-28 17:40:00 -05:00
Leo Franchi 0917a27b79 Document realm filter regexp oddities 
(imported from commit 987d8ea679f40c31f2fe2280b6d585f82b20e606)
 2014-01-27 13:43:55 -05:00
Tim Abbott 6d64427c46 Fix email mirror. 
(imported from commit 73c109db6e821b139203f662023ed780fcee0f24)
 2014-01-24 18:11:20 -05:00
Steve Howell d431206a3f Have the "knight" command use actions.do_change_is_admin. 
Using the higher level function has the benefit of sending
notifications to clients.

(imported from commit d680e3305b216fec3462479c696e572d008ee4c2)
 2014-01-22 14:43:28 -05:00
Luke Faraone b1cf9c3e25 Rewrite email-mirror documentation. 
(imported from commit 4af076b3063d7b82657765c12cf4dba0248cbf9f)
 2014-01-15 16:07:19 -05:00
Luke Faraone 895b7dc9b1 Accept emails on stdin in email-mirror and queue for later processing 
(imported from commit 7efccc294f87d3661e50f10962a80071b91e3879)
 2014-01-15 16:07:18 -05:00
Luke Faraone 1005e2ae7c Use module docstring for usage information 
(imported from commit 7c1cbf68eca4e3f31c75bfe13b50654e76be0df3)
 2014-01-15 16:07:18 -05:00
Jessica McKellar cf5b7dc0e4 Don't send digests to CUSTOMER29 teams. 
(imported from commit c1dbb6607647e65082535d4e37d50d7842f8758f)
 2014-01-15 16:07:17 -05:00
Jessica McKellar f85d45a781 Add a management command to bulk turn off digests. 
(imported from commit 0ffb565ecc9be219807ae9a45abb7b0e3e940204)
 2014-01-15 16:07:17 -05:00
Jessica McKellar d3f6c4ef36 Log the actual sending of digest emails in addition to queuing possible ones. 
(imported from commit e43ac59e71620fe715db462347db228d5a950aed)
 2014-01-10 21:38:59 -05:00
Jessica McKellar 33e8822165 Don't send digest e-mails to users in deactivated realms. 
(imported from commit dca53af9beac80242a5a96857595e99ccb75cf77)
 2014-01-07 20:24:21 -05:00
Leo Franchi 76892c52aa Send realm filters to clients 
(imported from commit a3ba31701baa4d515ea3bf36c39d7255f1dd20d6)
 2014-01-07 17:33:33 -05:00
Jessica McKellar 5f7c3a959e Enable digest emails for all but special-case domains. 
(imported from commit 6f5c8c6c1dcd0f7ba040e86edd7b1bd41ac0dc1e)
 2014-01-06 17:32:56 -05:00
Jessica McKellar b082c99267 digest: send e-mails on Fridays. 
(imported from commit 2080b98b4d67cfdefa124e2115b2c67b8ef975ce)
 2014-01-06 17:32:55 -05:00
Luke Faraone df4b2d9f85 Break out non-Twisted email mirror code into separate library 
(imported from commit 45c3e70ba81cdb44f1e9db4f162e5f2d2fe5ead6)
 2014-01-06 13:36:53 -05:00
Luke Faraone 266c0f4886 Remove duplicate validation in email_mirror 
We already have a try-except earlier in the file about email_gateway_user, so we don't
need to check for it again.

(imported from commit 2d9fa357fab2605916c5c5cb61961c0a121b1211)
 2014-01-06 13:36:53 -05:00
Jessica McKellar a20894d847 digest: fix generator bug not producing correct deployment comparison. 
(imported from commit 33c34d71320fa1f8cd095a6d9f10b2397eeeda2f)
 2013-12-31 16:34:52 -05:00
Jessica McKellar 61103d8349 digest: On an app server, only send digest emails for domains it manages. 
(imported from commit 9aea636a19eb6fdefacdc7dc393bf075a05862c5)
 2013-12-20 12:50:23 -05:00
Jessica McKellar a8cd91d6c7 digest: send emails if you've been active for slightly less than a day. 
That way if all you do is briefly check Zulip because you got the
email, we'll send you another one tomorrow.

(imported from commit fcbbd264c5e5fea7352f0fee6989e000af7b7bed)
 2013-12-17 10:47:17 -05:00
Jessica McKellar 6b64ef6be3 digest: log when we enqueue emails. 
(imported from commit 1b34e05bc2eebedb818673a4c9a3944e1db15c6d)
 2013-12-17 10:47:17 -05:00
Jessica McKellar 71f5b4e0ae Enable digest emails for zulip.com. 
(imported from commit f676f96c2b516b33269b8f5c72deaf5c985e8ccd)
 2013-12-17 10:47:17 -05:00
Tim Abbott b8579c6848 Disconnect handlers if the client closes a connection. 
Tested using the following procedure (run-dev.py won't pass through
the client connect closing)

tabbott@monastery:~/zulip$ curl http://localhost:9991/api/v1/register  -u email:key  -d 'event_types=["message"]'
{"msg":"","max_message_id":2259,"last_event_id":-1,"result":"success","queue_id":"1386884005:0"}
tabbott@monastery:~/zulip$ curl -G http://localhost:9993/api/v1/events  -u email:key -d "last_event_id=0" -d "queue_id=1386884005:0"
(then hit ctrl-C)

(imported from commit 3c4f3d5caac97b3de53da994ff9cd9ef67b2b9ea)
 2013-12-12 16:52:25 -05:00
Leo Franchi 4d30756e72 [manual] Add a manage.py command to import realm filters 
This must be run manually on staging after deployment. Once it has been run,
it can be deleted. It only needs to be run on staging, not prod.

(imported from commit 79252c23ba8cda93500a18aa7b02575f406dd379)
 2013-12-11 14:39:10 -05:00
Leo Franchi e377be96d4 Add a management command to set realm filters. 
(imported from commit 6590d7eb58ea6e6afc15104dd75ad56b832858fa)
 2013-12-11 14:39:10 -05:00
Luke Faraone 1b5c1ac021 Update style of client strings. 
(imported from commit 1516461cf53b2715de68e01f16bb8a8cc33c48ad)
 2013-12-09 11:47:52 -05:00
Tim Abbott e5be713103 Clean up EXTERNAL_API_HOST usage and defaults. 
We now have 2 variablse:
EXTERNAL_API_PATH: e.g. staging.zulip.com/api
EXTERNAL_API_URI: e.g. https://staging.zulip.com/api

The former is primarily needed for certain integrations.

(imported from commit 3878b99a4d835c5fcc2a2c6001bc7eeeaf4c9363)
 2013-12-04 15:10:54 -05:00
Waseem Daher 89844c6ced Revise day1/day2 emails. 
(imported from commit dc5e78cccb22b024be46b3066d44f09ea1b53345)
 2013-12-02 11:15:13 -05:00
Luke Faraone 0c142015ea Tool to test LDAP functionality 
(imported from commit c4b4d8b25ba3afd333faf09bc232da2f92a57b7e)
 2013-11-25 17:44:47 -05:00
Luke Faraone 4a8ec7b357 Management command to update names from the LDAP database 
(imported from commit 1e1b12c055926899fdca3e484df6c9437c800c6c)
 2013-11-25 17:44:47 -05:00
Tim Abbott 7cb261a0ed Set default streams to engineering,social when creating realms. 
(imported from commit 4e174fec162be4138da099edfc4cfb8d5662b7f1)
 2013-11-25 14:00:28 -05:00
Tim Abbott ca8225cf47 [manual] Add endpoint to cleanup a finished events queue. 
This requires a puppet apply on each of staging and prod0 to update
the nginx configuration to support the new URL when it is deployed.

(imported from commit a35a71a563fd1daca0d3ea4ec6874c5719a8564f)
 2013-11-20 18:34:15 -05:00
Tim Abbott f3fa78d30c Rename do_[rd]activate to do_[rd]activate_user. 
For similarity with do_activate_user.

(imported from commit 2874af84dbe1ce666de10e0d8ffe2daa8bad9731)
 2013-11-16 11:35:26 -05:00
acrefoot 9616ea7afd remove enqueue_followup_emails management command 
(imported from commit 20ce0b698cf795a6e1c7ff5ba7b546901e1fd8a4)
 2013-11-15 21:31:37 -05:00
acrefoot e77d534e78 remove ALLOW_ARBITRARY_SENDERS_LOCAL_EMAIL 
(imported from commit 7a374959516d8505685466293f146d95094a7ea7)
 2013-11-15 21:31:37 -05:00
Tim Abbott c24addf2d5 Build internal bots in the zulip.com domain. 
Otherwise do_create_realm can't actually send the notifications.

(imported from commit 4fa9a53f1d3d8a2e26d7b89401e7dfa77f2f7533)
 2013-11-14 11:16:16 -05:00
Tim Abbott e2537ade44 Create the admin realm using do_create_realm. 
That way, we get the first-user message sent automatically.

(imported from commit 70be26a8152676055a94c363706c35964284a3de)
 2013-11-14 11:16:16 -05:00
Tim Abbott af2da58fd0 Set default default streams for enterprise installations. 
(imported from commit 87b9fdc6ef19a88a2abda4ba5cdfe2a5b35d581d)
 2013-11-13 21:42:21 -05:00
Tim Abbott 4078b8bb6c Move set_default_streams from zilencer to zerver. 
We'll want to use it for setting up local server instances.

(imported from commit 47c60c349ba3cebad58506f439013def4c40708b)
 2013-11-13 21:08:14 -05:00
Tim Abbott bf991dac94 enterprise: Create feedback_bot as a bot. 
(imported from commit 8bd985f82aa80380188a644869c55bc54b6f7977)
 2013-11-13 17:26:30 -05:00
acrefoot 6d38285a2e fixup supervisor oops related to postmaster config 
(imported from commit 8b5c39f0d13abb5e1def9f88a2ab82cfa67b42f6)
 2013-11-13 17:15:55 -05:00
Tim Abbott 81156cf2f5 Clean up initialize_enterprise_db and fix domain. 
(imported from commit af6f20c7705720c0043817516fdb3c21777796bb)
 2013-11-13 16:47:37 -05:00
acrefoot b134ecc102 add print_email_delivery_backlog for nagios reasons 
(imported from commit c1c1edb8614913c7fff63e175423ee5701465223)
 2013-11-13 16:41:36 -05:00
acrefoot f5199081d5 Add the deliver_email management command 
This command should be run continuously via supervisor. It periodically
checks for new email messages to send, and then sends them. This is for
sending email that you've queued via the Email table, instead of mandrill
(as is the case for our localserver/development deploys).

(imported from commit a2295e97b70a54ba99d145d79333ec76b050b291)
 2013-11-13 16:41:36 -05:00
Jessica McKellar f5fc59e3e0 email mirror: remove some TODOs enterprise users don't need to see. 
These should live in tickets anyway.

(imported from commit 9c2a9445640278d251a99c68c0074c138b832c60)
 2013-11-13 16:28:30 -05:00
Jessica McKellar 3b4303598a email mirror: use the correct API host locally. 
In particular, EXTERNAL_HOST doesn't specify the protocol, which gets
coerced to HTTPS.

(imported from commit 53f2e8106cf33114dcdd2ad17e09b41609641e71)
 2013-11-13 16:28:29 -05:00
Jessica McKellar 8382e074fe Move use of html2text to a subprocess call. 
(imported from commit 36e8a6f030d75196c28fbdc0e58c6968952d95ff)
 2013-11-13 16:28:29 -05:00
Leo Franchi 691c861c85 Guard against creating a realm that is already an alias to another realm 
(imported from commit 365c79148a1a3af190d2eab449d735525554657b)
 2013-11-13 14:17:44 -05:00
Leo Franchi 62ba16a829 [schema] Allow a realm to have aliases for multiple other domains 
(imported from commit 5d0340afd4c77f50db63550a3ec8d81e885c285c)
 2013-11-13 14:17:43 -05:00
Tim Abbott 6bcd5a7ffc Update installation instructions to include configuring the app. 
(imported from commit 89395815c32416ec0636efcff12eb76f5d890bbd)
 2013-11-13 12:02:50 -05:00
Leo Franchi 300cceb3e4 Create feedback bot in enterprise installs 
(imported from commit 0c0a6919bd29e12484d520f80789b39b203099b0)
 2013-11-13 11:18:03 -05:00
Leo Franchi d7b823f017 Point email mirror to staging for zulip.com in non-localserver, and prod otherwise 
(imported from commit 3076df1e68e0200af06b79a8cdeb9822bbf4c10c)
 2013-11-12 22:22:13 -05:00
Luke Faraone fafa14db9a Patch out references to zilencer so the app runs without it on ENTERPRISE 
(imported from commit bd4fe1ff47e1b4dc9038f800ee8aa2980bd00fdf)
 2013-11-12 18:43:06 -05:00
Tim Abbott 3dadb5b76a Update comments to rename Local Server => Enterprise. 
(imported from commit 82bf89aa1148a78ce11ead203a0bc16243d1fca8)
 2013-11-12 15:57:02 -05:00
Tim Abbott 21f1058fd2 Rename local_server=>enterprise in DB initialization script. 
(imported from commit 76e26f43858e3baa6ed1b38d67973ed41acf04f3)
 2013-11-12 15:57:02 -05:00
Tim Abbott 5293cdebe8 Rename LOCAL_SERVER to ENTERPRISE. 
(imported from commit 7edf353eefe6c9e7aac74b7bbc37b923cac1b913)
 2013-11-12 15:57:01 -05:00
Kevin Mehall a593a798f8 Move send_stats management command back to zerver/ 
It's not analytics, and it's needed for restart-server.

(imported from commit 979fa15715ea437cbbc5d986c859ee4d6c668da8)
 2013-11-12 15:50:08 -05:00
Tim Abbott 64c3a9a0bd initialize_local_server_db: Improve success output. 
(imported from commit 7ac45876e639b7af5a283fb60b134ec13438a313)
 2013-11-12 09:38:43 -05:00
Tim Abbott 96918e30f9 initialize_local_serverd_db: Create the realm for ADMIN_DOMAIN. 
(imported from commit e13e73e54871c106cd4fd465b74c645657899dce)
 2013-11-12 09:38:42 -05:00
Tim Abbott e06722657a [manual] Remove /messages/latest API and related legacy code. 
This requires doing a puppet apply on our servers to take effect
properly.

(imported from commit 19dc56f071f07a5d2571eef49dd835121b2e82b6)
 2013-11-05 14:19:40 -05:00
Leo Franchi 0bcc8db924 Give the initial bots themselves as owners 
(imported from commit aca5565a3705aa6c6d55eb8afb5ca84f0efc9a0c)
 2013-11-04 17:36:14 -05:00
Tim Abbott 97e9615921 Add script to replace populate_db for local server. 
(imported from commit ca51391fc1d8ed13c0fcd36e17c875a085c78eb0)
 2013-11-04 17:01:41 -05:00
Leo Franchi fe1313ad76 Split management commands for internal use into zilencer/ 
(imported from commit 3fd81bfc7e987fc88bd1d632546850eebbb92234)
 2013-11-04 11:58:52 -05:00
Tim Abbott 68dcc760c3 Clean up some unused imports. 
(imported from commit 0c5d8e2a55ba1b8909ba807fee3afe863dcdc226)
 2013-11-04 11:51:17 -05:00
Steve Howell 7a67fb34b0 Rename: queue_followup_emails -> enqueue_followup_emails. 
(imported from commit c44e790de5b694f8d98b03608231f7a284856254)
 2013-11-04 10:20:59 -05:00
Steve Howell c4141eeeb0 Rename: queue_digest_emails -> enqueue_digest_emails. 
(imported from commit a72fdf849814ff974117c55bf1b155c494a1b160)
 2013-11-04 10:20:59 -05:00
Tim Abbott 03c5ae44a4 Remove unnecessary transaction in message flag updating. 
We only needed a transaction here to workaround problems associated
with not having database-level autocommit.

(imported from commit 240ba05a4a4a846a7ff62e6e59e403ab0d78ab11)
 2013-11-04 07:31:39 -05:00
Steve Howell 54a25b065a Add show_admins management command. 
(imported from commit f16e118fa5203408b99f0a41ff7cecbffb821fa7)
 2013-11-02 11:37:27 -04:00
Leo Franchi ca8187b8cf Email gateway bot variable name changed to EMAIL_GATEWAY_BOT 
(imported from commit 01c430bd9b916c9bb152ee821214495333f1d37e)
 2013-11-01 16:46:17 -04:00
Leo Franchi db6550e99a Make built-in bots come from settings.py, and allow localserver-specific ones 
(imported from commit e21933e37487314ac986147562817a19227e8960)
 2013-11-01 14:13:05 -04:00
Tim Abbott ddbfe31449 Remove obsolete management command for mit.edufullname transition. 
(imported from commit aed2a2d7bfedd98e2406bb18bf54f953a5ae7ef3)
 2013-11-01 10:30:17 -04:00
Tim Abbott 11b92534e8 Remove obsolete management command for unread counts transition. 
(imported from commit 52c6acbf4e8e53f86e880e475396d824c729405f)
 2013-11-01 10:30:17 -04:00
Tim Abbott 429c41da79 Rename client-activity to client_activity to match other management commands. 
(imported from commit b0b2cd372810923b3c808142f7b7eeef285d132d)
 2013-10-31 15:29:50 -04:00
Tim Abbott dbd4c187e3 Rename create-stream to create_stream to match other management commands. 
(imported from commit 28daad12943ab5eb245d7737934b08f5b29e8f90)
 2013-10-31 15:29:50 -04:00
Tim Abbott 5b1e504e13 Rename rename-stream to rename_stream to match other management commands. 
(imported from commit 28522ac669a38e3de3f25b28c82ab0c787aecd0b)
 2013-10-31 15:29:50 -04:00
Tim Abbott 013e84ef86 Remove obsolete explain_js_error management command. 
(imported from commit 92f23e3bb7e6dbbad5162446b5f19e74728cf608)
 2013-10-31 15:29:50 -04:00
Tim Abbott 217478e988 Remove obsolete clear_db management command. 
do_destroy_rebuild_database is the "new" way to reset one's dev
database.

(imported from commit 8edb707ecd5e6e24b62eef324915cc3a4a5b8eda)
 2013-10-31 15:29:50 -04:00
Tim Abbott 384f64d38d Remove obsolete management commands from various field additions. 
(imported from commit b44a2b683768c790839e82e2e9c9a18827670113)
 2013-10-31 15:29:50 -04:00
Tim Abbott f0580dd28c Remove obsolete management command for activating MIT users. 
(imported from commit a965eea8edfc9d73a00770d85cd517343f607338)
 2013-10-31 15:29:50 -04:00
Tim Abbott 06d43a6f97 Remove obsolete management commands from the stream color transition. 
(imported from commit d6d508d952714a1865ca4fddcb48d0ccedb6f4dd)
 2013-10-31 15:29:50 -04:00
Tim Abbott 6d8fc371fb Remove obsolete command to delete tutorial streams. 
We don't have tutorial-bot anymore, so this is no longer needed.

(imported from commit b8a1bc9c2379a451e8cc1ed43b15f373bb3f2b06)
 2013-10-31 15:29:50 -04:00
Tim Abbott c8d07cc64c Remove management commands from the mysql=>postgres migration. 
(imported from commit 636f71b435a50e83fd8a9e40685d7c52b3b01f30)
 2013-10-31 15:29:49 -04:00
Waseem Daher 1d784801cf More settings.LOCALSERVER -> settings.LOCAL_SERVER. 
(imported from commit 2b5045d302b8ebb2a5c7175264b4ab9cfef2844d)
 2013-10-31 13:31:03 -04:00
Steve Howell 70c1c31b3a Simplify callback protocol for register_json_consumer(). 
The register_json_consumer() function now expects its callback
function to accept a single argument, which is the payload, as
none of the callbacks cared about channel, method, and properties.

This change breaks down as follows:

    * A couple test stubs and subclasses were simplified.
    * All the consume() and consume_wrapper() functions in
      queue_processors.py were simplified.
    * Two callbacks via runtornado.py were simplified.  One
      of the callbacks was socket.respond_send_message, which
      had an additional caller, i.e. not register_json_consumer()
      calling back to it, and the caller was simplified not
      to pass None for the three removed arguments.

(imported from commit 792316e20be619458dd5036745233f37e6ffcf43)
 2013-10-31 13:18:11 -04:00
Steve Howell ddc7608afe Add enqueue_file management command. 
This reads lines from a file and pushes them on to one of our
worker queues.

(imported from commit 2ed3338c5065819f8bafb247a15799e2bc9e7d39)
 2013-10-30 16:47:16 -04:00
Luke Faraone ab9b1be82c Rework management commands to handle deployment assignment 
We now ensure `create_realm` adds you to a default deployment and that
`create_deployment` removes the old deployment association when
performed.

(imported from commit 5b94fb07b8e11332765b057dc640a5ed873ec99e)
 2013-10-30 12:00:13 -04:00
Tim Abbott 901a5c342c Use os.path directly rather than sometimes importing it separately. 
(imported from commit 48486c4ea64d02a15faeebb0f490d31e9b113d62)
 2013-10-28 15:44:13 -04:00
Zev Benjamin 807116e955 Delay connecting to RabbitMQ until it's necessary 
Previously we were connecting at import-time.

(imported from commit 9a638f0d238f3b6b00feb4aa524098a64953cb92)
 2013-10-28 15:27:25 -04:00
Zev Benjamin ffb6266319 Refuse to run a queue processor if USING_RABBITMQ is False 
(imported from commit 39beff47cdbb18ba39756989e6f07facbd16864f)
 2013-10-28 14:30:53 -04:00
Zev Benjamin 8aa2c7b547 Make the purge_queue management command more efficient 
Before we were removing items individually from the queue.  We now
directly use RabbitMQ's queue purging mechanism.

(imported from commit 62ab52c724c5a221b4c81a967154a4046a579f84)
 2013-10-28 14:30:53 -04:00
Jessica McKellar fac9d34d6c digest: Add a management command to enqueue digest email recipients. 
(imported from commit 70ff2c7a4dae654f4077041c45e2154b3ac7afb7)
 2013-10-25 19:08:47 -04:00
Waseem Daher 66f48288b4 Rename LOCALSERVER -> LOCAL_SERVER. 
(imported from commit b3abdd10d54d2ad7a9c463af9a291d2e2127707f)
 2013-10-25 17:37:06 -04:00
Luke Faraone a1b44986e0 [schema] Add a base_{api,site}_url property to our deployments. 
This will allow us to redirect clients to the correct local site.

To apply this migration, just run:
     python manage.py migrate zilencer 0002

(imported from commit 7bd39b5f035145b6b52e1b2cb2ad5f6720d598ce)
 2013-10-25 15:18:37 -04:00
Luke Faraone 81d7dd1fda [schema] Support for authenticating Deployments via the API. 
Here we introduce a new Django app, zilencer. The intent is to not have
this app enabled on LOCALSERVER instances, and for it to grow to include
all the functionality we want to have in our central server that isn't
relevant for local deployments.

Currently we have to modify functions in zerver/* to match; in the
future, it would be cool to have the relevant shared code broken out
into a separate library.

This commit inclues both the migration to create the models as well as a
data migration that (for non-LOCALSERVER) creates a single default
Deployment for zulip.com.

To apply this migration to your system, run:
   ./manage.py migrate zilencer

(imported from commit 86d5497ac120e03fa7f298a9cc08b192d5939b43)
 2013-10-25 14:13:30 -04:00
Luke Faraone 52309b5789 Enable absolute imports for non-third-party code where it was omitted previously 
(imported from commit 34856ac32c6b94b614273a6fe46c87a314058d9b)
 2013-10-25 14:13:30 -04:00
Leo Franchi a9b9c0ec7b Add Email Mirror default settings and error out with readable warning if not configured 
(imported from commit adac321d09561b51277ec1e314a1d480dc01e0c8)
 2013-10-25 11:51:13 -04:00
Leo Franchi 2d276179d0 Refactor email-mirror to handle running on any machine 
(imported from commit 2971449ceaacb564770e66874fc095f77e68d445)
 2013-10-25 11:37:34 -04:00
Leo Franchi dc8bfde0b4 Add a check_apns_tokens manage.py command to flush old tokens 
(imported from commit 15dcfaa1967bd749b582882c4a816a457ca6cc12)
 2013-10-24 14:54:31 -04:00
Leo Franchi 05564ec0e2 [schema] Add a config option for enabling/disabling push notifications 
(imported from commit 5649188cd5e0e4edb6e71559ec7d7fc5e92efaf8)
 2013-10-24 14:54:30 -04:00
Zev Benjamin 5979af3a45 [manual] Add asynchronous message sender via sockjs-tornado 
New dependency: sockjs-tornado

One known limitation is that we don't clean up sessions for
non-websockets transports.  This is a bug in Tornado so I'm going to
look at upgrading us to the latest version:
https://github.com/mrjoes/sockjs-tornado/issues/47

(imported from commit 31cdb7596dd5ee094ab006c31757db17dca8899b)
 2013-10-22 18:45:11 -04:00
Steve Howell 03c856ce10 Add create-stream management command. 
The current version should only be used for testing; for example,
if you want to create a bunch of streams for stress testing, you
can run this in a loop.

(imported from commit ec51a431fb9679fc18379e4c6ecdba66bc75a395)
 2013-10-19 09:28:59 -04:00
Jessica McKellar b7419d50b4 create_realm: add a reminder to run set_default_streams. 
(imported from commit 59cf9378f8f271a88a6fbda591814fd9142a8d93)
 2013-10-17 13:47:51 -04:00
Jessica McKellar 6dbf7613a1 Supply a name when creating realms. 
For our populate_db bulk creation, just use the domain.

(imported from commit 4fb756f6dfa2d8f90e55822e27891e84168d5d1c)
 2013-10-17 13:47:51 -04:00
Jessica McKellar 648b3bed74 create_realm: don't let us create realms with invalid domains. 
(imported from commit 95a75a941b18d0857027084f5f9ea4c011805c9d)
 2013-10-17 13:47:51 -04:00
Leo Franchi 753aace8f8 Add a basic statistics module to analyze active user engagement 
(imported from commit 4d7a21b9a09b97c88a80d0446fb16e53012507c9)
 2013-10-16 18:39:44 -04:00
acrefoot 98897fe5e3 Queue the day 1 and day 2 Zulip followup emails 
Handled by the queue processor for signups. Added a management command
that accomplishes the same task, in case it's needed for manually added users,
or in case we goof and need to remove queued emails for a given user.

This addresses Trac #1807

(imported from commit 6727b82a07fa6a3ea3d827860c9e60fd0602297a)
 2013-10-10 19:32:21 -04:00
Jessica McKellar 6afdfeb1dc Filter obvious footers from email mirror messages. 
(imported from commit 7e22b6823b5d0a07f7f350e4af65be66728a8f88)
 2013-10-02 14:39:22 -04:00
Jessica McKellar f0911f51a0 email mirror: ignore decoding errors. 
Empirically, we only get these for malformed emails where the charset
specified in a message part header does not match the true encoding of
the part. I checked what the resulting Zulip looked like for the
original offender, and it looked find with ignoring errors.

(imported from commit ac6ba65b611cb22d4ec547b75a585abce6fc50b0)
 2013-09-27 11:34:57 -04:00
Tim Abbott e4496521e9 Update logging path for email-mirror. 
(imported from commit 75f36dbb963b2b77ecda7ba138023f0f774047e0)
 2013-09-25 16:52:41 -04:00
Jessica McKellar 0d897c469d Add a utility to report some rough recent client activity metrics. 
(imported from commit 27b4a70871939b2728fcbe0ce5824326ff4decc2)
 2013-09-25 14:32:53 -04:00
Zev Benjamin 871afde142 Prepare process_queue to be used with supervisor's numprocs 
(imported from commit 5a652b93f1e8b32b5ed89d622035161abaedfb11)
 2013-09-24 20:44:15 -04:00
Zev Benjamin e450fc80c9 Move existing queue processing workers to new framework 
(imported from commit b448eeefcddb1840344bd337046be30714a7fdbc)
 2013-09-24 20:44:15 -04:00
Zev Benjamin 6cf5ebc63d Add new system for defining and running queue-processing workers 
(imported from commit 7e77da57b8ad0b70837785c85f546601ba5b1957)
 2013-09-24 20:44:15 -04:00
Jessica McKellar d434bf4bb4 email mirror: use a message part's specific charset rather than the message charset. 
The overall message charset may be null or not match the part's
charset. Even though it's unclear from the documentation,
experimentally using the charset for a message part seems to give you
the charset even for non-multipart emails.

(imported from commit 0e1d23073f4c53041f9760e66a6635f8a94893d1)
 2013-09-24 16:19:53 -04:00
Jessica McKellar 7bb75e3acb Populate date_created for existing realms. 
(imported from commit 6a10150905a23589f098778ccbd8075025ff900f)
 2013-09-24 16:19:52 -04:00
Zev Benjamin 233290d578 Add a management command to purge a rabbitmq queue 
This is useful in debugging when you just want to discard all the
messages in a queue because they have the wrong structure.

(imported from commit 8559ac74f11841430b4d0c801d5506ebcb74c3eb)
 2013-09-23 11:25:58 -04:00
Jessica McKellar ee2cd51c4c generate_invite_links: work for open realms too. 
(imported from commit ec2143d25cb2f6f4181ece7a25c79666334a1634)
 2013-09-20 10:00:10 -04:00
Kevin Mehall 63cff6eb71 Make populate_db set the is_bot flag on bot accounts. 
(imported from commit 93a507faa2b19fde05376fd73ed3a65149cfce68)
 2013-09-18 16:16:17 -04:00
Jessica McKellar c2e704e4b6 Instead of erroring out, do our best to format HTML-only e-mails. 
(imported from commit a58e65864ebb3ef539cf15902b59636cd3912b69)
 2013-09-18 15:41:27 -04:00
Jessica McKellar 8ab3ae9da2 email mirror: handle non-ASCII message bodies. 
(imported from commit 1a8e21b7ff5441c21147bab85f5a460a7567c5e3)
 2013-09-18 15:41:27 -04:00
Jessica McKellar 28f80211ec email mirror: handle non-multipart e-mails when checking for attachments. 
(imported from commit 2132d604bb353ef6e8ca2c371479b2b2146d8603)
 2013-09-17 16:20:28 -04:00
Jessica McKellar 3db51fa17f Upload and display attachments sent through the email mirror. 
(imported from commit b3ac24b71a2e7c354d69ecae9174316102ab8f84)
 2013-09-17 10:00:01 -04:00
Jessica McKellar 2c0ba365ae email mirror: show more debugging information on errors. 
(imported from commit 57f5b8b57062501443367953239526f1ec78eb7f)
 2013-09-17 10:00:00 -04:00
Leo Franchi 2eb8c6aae3 Add documentation and a duration argument to analyze_user_activity.py 
(imported from commit 5668ec02a23dcfeb29760b55d8acee17cfdea9cf)
 2013-09-10 13:25:59 -04:00
Tim Abbott 9fcca3df4e [schema] Collect data on when users are active on site. 
These engagement data will be useful both for making pretty graphs of
how addicted our users are as well as for allowing us to check whether
a new deployment is actually using the product or not.

This measures "number of minutes during which each user had checked
the app within the previous 15 minutes".  It should correctly not
count server-initiated reloads.

It's possible that we should use something less aggressive than
mousemove; I'm a little torn on that because you really can check the
app for new messages without doing anything active.

This is somewhat tested but there are a few outstanding issues:

* Mobile apps don't report these data.  It should be as easy as having
  them send in update_active_status queries with new_user_input=true.

* The semantics of this should be better documented (e.g. the
  management script should print out the spec above)x.

(imported from commit ec8b2dc96b180e1951df00490707ae916887178e)
 2013-09-10 13:25:59 -04:00
Leo Franchi a33957db32 Handle mailchimp error 214 (duplicate email) in signup worker 
(imported from commit cb34c153fc96bca7c8faed01d019aa2433fcf568)
 2013-09-09 10:57:09 -04:00
acrefoot cf3e98d74a use staging for wdaher.com for the email script 
(imported from commit fbad47386f5c48785f766886218370612db31c26)
 2013-09-05 19:19:41 -04:00
acrefoot 592e1ead6b Actually use the realm specified by the email mirror when doing the markdown 
(imported from commit 00b909b1d0c70a3523ab53d2f91815ab14ba2a6f)
 2013-09-05 18:24:55 -04:00
Luke Faraone 085fdebc98 Use the X-Gm-Original-To header instead of iterating through To/Cc/Bcc 
This will allow mail with an implicit destination (mailing lists, complex
forwarding) to be received by our system.

See http://support.google.com/a/bin/answer.py?hl=en&answer=2368151 for
documentation of Google's behaviour that adds this header.

(imported from commit f8fd500e3c27e12af5941c63c91d5c796a2cd24a)
 2013-09-03 17:46:44 -04:00
Jessica McKellar acdfb6afa0 email mirror: make being one of many recipients okay. 
Previously the email gateway had to be the only address in a recipient
field or we'd mis-parse the recipient.

This commit also makes the mirror correctly handle addresses of the
form "Jessica McKellar <jesstess@zulip.com>".

(imported from commit 7435f2b59b8f47dc599cc869f64597a730af7d12)
 2013-08-29 18:13:21 -04:00
Jessica McKellar c4157ed2b6 Make email mirror testing easier by using a different mailbox locally. 
The mirror will use INBOX when deployed and Test locally. Send an
e-mail to the Test mailbox by including the word "localhost" in the
subject; a GMail filter will place it in Test on receipt.

(imported from commit bacf9a9554c8c5e1f3ec8497761edf2c15d3745d)
 2013-08-29 18:13:21 -04:00
Jessica McKellar 95c70967d2 rename-stream: support non-ASCII stream names. 
(imported from commit 5fe2c803c0645bb6367f39bfe58acc478a66a0eb)
 2013-08-28 10:23:39 -04:00
Jessica McKellar 8f3c5dc924 Add a management command to rename a stream. 
(imported from commit b3acadc09b289b48e2ac298e50a5427545b6a473)
 2013-08-28 10:23:39 -04:00
Kevin Mehall e2a13637f2 Management command for realm emoji 
(imported from commit 4de3ac69c34bc6e92d180c672fa8276338fecf53)
 2013-08-23 21:39:29 -04:00
Leo Franchi 8484cac5c9 Add a messages/flags POST api call to change a message's flags 
(imported from commit b51ebd94c99f57f1cda19039023013209556e343)
 2013-08-20 16:19:07 -04:00
Jessica McKellar f3a32d8382 Report email forwarding errors to our internal Zulip. 
For now, just do this, and we'll reach out to realms having trouble
manually. We may eventually need to automatically reply to the e-mail,
reach out to a realm admin, etc.

(imported from commit 5c5ac354066f9e9be3fb928e1f8801613c22c1ac)
 2013-08-16 16:09:18 -04:00
Jessica McKellar 557bdd2a6e email gateway: Make forwarding even "richly formatted" messages work. 
(imported from commit b15f24fa6898eb2eaf7d27bf862e8bead76f544d)
 2013-08-16 16:09:18 -04:00
Jessica McKellar cc21429b99 Add a management command to dump all messages on public streams for a realm. 
(imported from commit f4f8bfece408b466af4db93b2da15cf69b68e0a3)
 2013-08-13 18:16:30 -04:00
Jessica McKellar 15afdf65eb email forwarder: encode stream names so they are safe in email addresses. 
(imported from commit 6699d656e3cae58fad062a1403fb9923429fde89)
 2013-08-13 14:28:47 -04:00
Jessica McKellar c4a0797413 Decode MIME-encoded emails before forwarding them to Zulip. 
Otherwise you'll see garbage.

(imported from commit 07e68a5e209fdea4f54470396a67986dd9fa4016)
 2013-08-13 14:28:47 -04:00
Jessica McKellar a878f8a668 populate_db: make emailgateway@zulip.com a default user. 
(imported from commit 000d03f59afbb1da125d5d610e1fff6b54d5dbd0)
 2013-08-09 15:22:59 -04:00
Jessica McKellar a228bb6671 Implement a v1 email-to-zulip forwarder. 
(imported from commit 5f91fd69ee2075413d0d4aedf71b878bf98c640e)
 2013-08-09 15:22:58 -04:00
Jessica McKellar 639aac964e Add a management command to generate hashes for all streams that need them. 
(imported from commit b3a05e47dfe69b44f984185e360d79bf04f7885b)
 2013-08-09 14:59:27 -04:00
Tim Abbott 2ea1bb05a5 Rename ~/.humbugrc to ~/.zuliprc. 
(imported from commit a0d53dd20097a56971874dc1d84c6f95267e84f2)
 2013-08-08 10:22:31 -04:00
Tim Abbott 4c8fdec554 Rename humbug => zulip in comments. 
(imported from commit 26f305fa9351135a1b874f3c2106f03aed7683cb)
 2013-08-07 10:00:08 -04:00
Tim Abbott ae615dfb4d Rename humbug => zulip in some settings. 
(imported from commit 4d10e6213cf828f16b833ecba98b950972c32442)
 2013-08-07 10:00:08 -04:00
Tim Abbott d59ba39518 Rename humbug_finish to zulip_finish. 
(imported from commit a73ef9302d16a4068c7d050d4882d2eba699488d)
 2013-08-07 10:00:08 -04:00
Tim Abbott 661ca38b58 Change humbug => zulip in some local variables. 
(imported from commit 88caa4a87ea0fd269ab741645c124c5d07d69c0a)
 2013-08-07 10:00:08 -04:00
Tim Abbott 2db6dc6c88 Change humbug => zulip in populate_db variables. 
(imported from commit 14f8e072fd87478e8877925912fde113a75aba2c)
 2013-08-07 10:00:08 -04:00
Tim Abbott d8f78a5e2d Fix some remaining references to zephyr. 
(imported from commit 2072d3451cc08bc32b997d7fafaafd7b6ac568df)
 2013-08-07 10:00:08 -04:00
Tim Abbott 1f1af26048 Change Humbug => Zulip in text/comments. 
(imported from commit 2f9d73431ae40e1b9e9e11bc2f4f62f566ae758a)
 2013-08-07 10:00:07 -04:00
Tim Abbott e05979d4c4 Rename humbug_realm to zulip_realm. 
(imported from commit b36dcc7e5240159357a0f30036394a2af9b85844)
 2013-08-07 10:00:07 -04:00
Tim Abbott e111a2f9a5 [manual] Rename Django app from zephyr to zerver. 
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).

At the time it is deployed, we need to make a few changes directly in
the database:

(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';

(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
 2013-08-06 07:39:36 -04:00