Commit Graph

284 Commits

Author SHA1 Message Date
Jessica McKellar 0c3382fabb Always give hashlib.sha1 and friends bytes.
This fixes an experienced bug where you couldn't subscribe to a stream
with non-ASCII characters (failing with a UnicodeEncodeError), as well
as many other potential bugs.

(imported from commit f084a4b4b597b85935655097a7b5a163811c4d71)
2013-03-23 00:10:10 -04:00
Tim Abbott a8febf8fce Revert "to_dict: Update rendered_content in the database after rendered."
This reverts commit 4e5a370bc43defb5083cac3df45c649cf2366abf.

(imported from commit 745b4a98d0e3f67acaa207ba53e9250060dcfe06)
2013-03-21 10:30:23 -04:00
Jessica McKellar 37e58b613d Truncate the tutorial stream name to fit the DB max stream name length.
(imported from commit e267656337f97d3ea14e14e957a405cf4fd2d06c)
2013-03-20 10:15:08 -04:00
Tim Abbott 02554b28cb to_dict: Update rendered_content in the database after rendered.
This change could significantly load the database if pushed to prod at
the same time as:

    send_message: Save rendered message content to the database.

goes out.  So this should go out in a different prod push from that
commit.

(imported from commit 4e5a370bc43defb5083cac3df45c649cf2366abf)
2013-03-20 07:34:10 -04:00
Tim Abbott 17e9e56899 Fix get_stream caching the DoesNotExist value of None.
(imported from commit bdef7438758517ba9083ecc8bf110e3caa1ec041)
2013-03-19 11:17:31 -04:00
Tim Abbott 9977cdde54 send_message: Save rendered message content to the database.
(imported from commit e5bd224687d5420f556b341286f03d3829973a3c)
2013-03-19 08:09:26 -04:00
Tim Abbott bb80d1c58c Move check_send_message and helpers to actions.py.
(imported from commit d74c90e25bd63931955d2ad9b3890be53d674a48)
2013-03-18 16:15:11 -04:00
Tim Abbott a23f6be28d Cache the results of Recipient queries in our memcached cache.
(imported from commit a6ba25a9c62d2738e4738b076370e4b2bc0ffaba)
2013-03-18 16:15:10 -04:00
Tim Abbott 3c143db01a Add memcached caching for get_stream lookups.
(imported from commit a089c8a99eaa1fa28ce401859cac1bdf0eb92a69)
2013-03-18 16:15:10 -04:00
Tim Abbott c0fece1cbd get_stream: Fetch the realm object as well.
(imported from commit 065ae574cb933cfc48371efd4327e70d37184dd4)
2013-03-18 16:15:10 -04:00
Tim Abbott 1e6ed16c23 Flush the User cache when saving User objects too.
(imported from commit e595209fe1190e2f3e0d55ad3b9bae0997b91d1a)
2013-03-15 18:09:34 -04:00
Tim Abbott b6f42cb1e1 Update UserProfile objects in memcached when we save them.
(imported from commit 59426e19eeeba131f363a4bc5f74f59b6dbe1066)
2013-03-15 18:09:33 -04:00
Leo Franchi 93a3f14c43 Add backend support for handling new 'read' message flag
(imported from commit 6194e9332caa2d279cbc304f0d6a69f969aa9a72)
2013-03-13 14:14:45 -04:00
Leo Franchi 6a53d1c35d Add flags to dict in load_old_messages
(imported from commit 7f68ea97a424aa0b01983258dbf6e9a2a07a732a)
2013-03-13 14:14:45 -04:00
Leo Franchi a678ef2af8 Add flags field on UserMessage, created by previous south migration
(imported from commit e839a3871d27cd7082ecbc84946216143d373da0)
2013-03-13 14:14:45 -04:00
Tim Abbott 254bc9f361 Attempt to render messages before accepting them from the user.
This fixes trac #407.

(imported from commit e33647eec32266790f864d14ad377d51956d2a6f)
2013-03-08 17:24:33 -05:00
Tim Abbott 2f022ebb34 bugdown: Allow caller of convert to tell whether we succeeded.
(imported from commit 9e3cab73e0d9e9ad31ba770210e535584cca3bb2)
2013-03-08 17:24:33 -05:00
Jeff Arnold fcd033e33e [schema] Save enter_sends on the server in the database.
(imported from commit 4d82f6aaf5918f155a930253c9cc334dbcc0d97a)
2013-02-27 17:25:29 -05:00
Leo Franchi 982a3fda9a Only ignore website messages from user when calculating unread counts
(imported from commit 95bd6f8c2ce0ad1f5d59e311107f81e4b07b33dc)
2013-02-21 15:05:59 -05:00
Keegan McAllister 2a556fd002 Re-enable markup for CUSTOMER1
(imported from commit 3d74260796c85216d71d8261020cc149e0b6ce1c)
2013-02-12 17:09:47 -05:00
Leo Franchi 31f87481d0 [manual][schema] Add an API for user presence (idle) information
Adds a new db table for storing presences, and an API for setting
an individual user's idleness as well as fetching all idle status
for all users in a realm

(imported from commit 5aad3510d4c90c49470c130d6dfa80f0d36b0057)
2013-02-11 18:05:57 -05:00
Jessica McKellar 8d06143115 [manual][schema] Add a restricted_to_domain bit on realms.
Default True; if False, arbitrary e-mail addresses can use your realm.

(imported from commit 6eec7516a535429cbaa74f6348651feef73cacd6)
2013-02-08 13:16:35 -05:00
Luke Faraone cb5c9ed047 [schema] Add new boolean field in_home_view to Subscriptions
This is part of the Oppa Gmail Style proposed by Waseem.

(imported from commit 0663889bcb1f691842f69d243fd4ac0e4e3e0868)
2013-02-05 15:34:03 -05:00
Jessica McKellar a8a5d0b511 Strip whitespace before looking up a stream name with get_stream.
This came up when I tried to subscribe myself to an existing stream
name but with leading whitespace added; the get_subscribers request
failed with a 400.

(imported from commit 03fe370d1e80fd7e8c282f260e63b7e5ada05960)
2013-02-05 10:01:45 -05:00
Leo Franchi 7e567f527f Move gather_subscriptions to actions.py as it's not a view
(imported from commit cdf94ded51e441fe3515641458023be44c1ba40c)
2013-01-28 17:32:58 -05:00
Leo Franchi 7d05450411 [schema][manual] Add schema invite_only flag for streams
(imported from commit 6234b67b3c72912bf40e39007bbc884d20774806)
2013-01-24 17:34:21 -05:00
Tim Abbott 8cb8c0065c Move get_stream to models.py and accept an id for the realm argument.
(imported from commit 7ce09b7ed450cfa3ddf1de22bbb0b19abfc4a2a2)
2013-01-17 16:20:18 -05:00
Reid Barton 7b7f6390f9 Add a notion of "public" stream
A public stream is one for which any user can view all messages sent
to the stream, regardless of whether the user was subscribed when
those messages were sent.

For now, to avoid a database schema change and to facilitate testing,
public streams are all streams on the customer29.invalid or
humbughq.com realms.

(imported from commit 7a71fd788d585a6f5b3e494e771ec85b632bb36e)
2013-01-16 14:19:05 -05:00
Luke Faraone 22a4b76290 [manual] [schema] Add backend function to invite users to the app.
(imported from commit 17ae27b22bdca33e63b96fa8c819759e4f4315fa)
2013-01-14 10:25:25 -05:00
Tim Abbott e36fd4b817 tornado: Send rendered messages to Tornado via memcached.
(imported from commit c1e8a017fa61b4e3f5c44ad4e5f59e4faf012ca3)
2013-01-11 16:11:07 -05:00
Tim Abbott 1a82741650 Move action functions from models.py to zephyr/lib/actions.py.
(imported from commit 9d577dd53ce7d4c9faf6cc8a56129d684a50811b)
2013-01-11 16:11:03 -05:00
Tim Abbott 761b7cd5fa Rename zephyr.lib.time to zephyr.lib.timestamp.
Otherwise one gets:

AttributeError: 'module' object has no attribute 'time'

when trying to use the time module from inside zephyr.lib.

(imported from commit 645368672a3eff68320278dd480edeed56721fcc)
2013-01-11 15:58:19 -05:00
Tim Abbott bd05bdd777 Cache client objects in get_client.
This should remove a database query invoked from Tornado by our
decorators in the common case.

We use hashlib.sha1() on the key because memcached doesn't support
keys with whitespace in them.

(imported from commit 351ef4c76bc68e2d53f0e7ddca91dcf95e7bb2a6)
2013-01-10 16:59:36 -05:00
Tim Abbott 3c10a2e6a0 Move bulk_create_* methods to bulk_create.py.
(imported from commit 30d5f86209f9c8807516f0440276fafeb6590c1a)
2013-01-10 16:59:36 -05:00
Tim Abbott 21c2317bd2 Move callbacks library to tornadoviews.py.
(imported from commit a4e545d82133ae9dc34e283e3bf268225e35f400)
2013-01-10 16:59:36 -05:00
Tim Abbott e7b74060d2 Log password change events via the password reset feature.
(imported from commit bbec7074229e8779c81d439d4eef373b5dac9fa7)
2012-12-13 15:22:35 -05:00
Tim Abbott 7cebbd4a52 Add datetime_to_timestamp library function.
(imported from commit 28466ac5c8fc406ec70a61cae5975a784eb0f238)
2012-12-12 15:45:37 -05:00
Tim Abbott c21535d420 models: Create database indexes on auxiliary Message fields.
I suspect we'll want these for narrowing code at some point.

(imported from commit d2077aa4d7a6967bb4cb7d9404a4db4fcf824343)
2012-12-11 17:17:38 -05:00
Tim Abbott 4dae30e750 Re-enable markdown for all realms but CUSTOMER1Labs.
(imported from commit 51e770b50e8c678c245d099dc9177aa49ecb5084)
2012-12-11 12:04:46 -05:00
Zev Benjamin ebb9cfa65d Re-enable markdown rendering for API users and the github bot
(imported from commit 31db13ef8b7e5224a07f784247092e2eb9cd1654)
2012-12-11 12:02:05 -05:00
Tim Abbott 100df7e349 Add a maximum message length of 10000 characters.
(imported from commit a07c872d612521c9792be8dc704a9cfd6c752c75)
2012-12-11 11:33:40 -05:00
Luke Faraone b9b7d1b8f1 Linebreak zephyrs.
(imported from commit c77499b17ce195e36169bcb39a5f5bdfe6a3ce51)
2012-12-10 15:45:04 -05:00
Tim Abbott 6cff3dc735 models: Disable markdown (escaping instead) for now.
(imported from commit e92920919cd5d0ef11a478be5a0f8a108b62fff9)
2012-12-10 15:23:57 -05:00
Luke Faraone ced8062509 Use MAX_SUBJECT_LENGTH instead of hardcoded constant.
(imported from commit cc885f7ca196ec1de0279a6a6192044ff7c8420f)
2012-12-07 17:32:16 -05:00
Luke Faraone 7fbee21584 Factor out Message() creation for internal use into separate function.
(imported from commit 0352010da4255fc74cd67835e04dbdb1f8bbf4e9)
2012-12-07 17:32:15 -05:00
Tim Abbott 83af3d4090 do_activate_user: Fix join_date default.
(imported from commit ea9f933d6f7e36964ed19ef6d819ad5edc34c483)
2012-12-06 15:49:03 -05:00
Keegan McAllister b64c8e8979 Add notes about StreamColor schema
(imported from commit 2ca71d63a208be1555866235a2dc631f21a0d181)
2012-12-06 11:04:32 -05:00
Tim Abbott 41ec4d44a0 Don't let users change their fullname to be whitespace.
(imported from commit 015fc7cbc06acd5057952a3f09a9b6d3fe68d9fd)
2012-12-06 10:32:45 -05:00
Tim Abbott 970969fc44 Add new timestamp_to_datetime helper function.
(imported from commit 6791d009ae2e8371abe2c929e87c816a1981f5fe)
2012-12-05 15:34:19 -05:00
Tim Abbott f843152427 log_event: Clean up duplicated timestamp setting code.
(imported from commit dc3b3144ca23106bef2d953b00dc042115008703)
2012-12-05 15:34:19 -05:00
Tim Abbott 156f6ee576 Log an event on new realm creation.
(imported from commit 98872c7e5922fecd2bed9b544ff5fffd8147343c)
2012-12-05 15:34:18 -05:00
Tim Abbott 81d560dc1f populate_db: Preserve the user's date_joined on replay.
(imported from commit 91dcb4f41e0e58f165db6a7417e57548914ddab7)
2012-12-05 15:34:18 -05:00
Tim Abbott 8e3bcdaa19 Add user_created event type and appropriate replay code.
This is necessary for correctly replaying new user signups on non-MIT
domains.

(imported from commit e5a9d37b8772bd9794579ef16e7039196ac06f62)
2012-12-05 15:34:18 -05:00
Zev Benjamin 7e4776322d [manual] Unify huddles and personals into private messages on the receive path
feedback-bot and zephyr_mirror will need to be updated and restarted
when this is deployed to prod.

(imported from commit fe2b524424c174bcb1b717a851a5d3815fda3f69)
2012-12-04 18:01:51 -05:00
Zev Benjamin f4898ea490 do_send_message: use JSON for the user list instead of joining/splitting with commas
(imported from commit 85d94ca647d0e6707f4f96e2b7a7b330f76db9d9)
2012-12-04 12:44:44 -05:00
Zev Benjamin 2907c3ca01 Create the Message and associated UserMessages in the same transaction
(imported from commit 96daa925142c97e34e68fae8e4bef713731b4173)
2012-12-04 12:44:44 -05:00
Tim Abbott cf31374497 bulk_create_streams: Fix missing select_related().
bulk_create_streams was taking about 10 seconds to run with prod data;
this should be a basically immediate operation.  The cause was a
missing select_related on one of the loops through all streams.

(imported from commit 8b82f0c41facc3999bb699dbc350708ac69797e9)
2012-12-04 10:55:31 -05:00
Jessica McKellar 55e68fc89a Log subscription color changes so they can be replayed by populate_db.
(imported from commit 5c7f46ed0d727926e9819cb2d71eb31f241e2e34)
2012-12-03 12:46:27 -05:00
Jessica McKellar 71371823f8 [schema] Add a StreamColor table.
(imported from commit 2906c30ad326373bc6265bf30dd0a70e11d69c7d)
2012-12-03 12:46:26 -05:00
Keegan McAllister 36816c46e2 Add a function to remove all Messages that are not referred to by any UserMessage
(imported from commit cdad013aed0fd8d1bc6ec8c2dcf5905390a01339)
2012-11-30 17:24:37 -05:00
Zev Benjamin f568ff20c2 Remove format_delayed_updates_response
(imported from commit d1aa1a1f36fc5ec36f1658aec489db84ae469f7c)
2012-11-28 16:36:43 -05:00
Jessica McKellar 4832a6a1fe [schema] Add a management script to create default streams for a domain.
Run this script on an existing realm to create or change default
streams, which new users will get upon account creation.

(imported from commit 8938dcbd3520d97d25b4c6ca783d35c9aef52df0)
2012-11-28 14:07:16 -05:00
Jessica McKellar cca39ca5e8 Add default subscriptions upon non-MIT user creation.
We'll probably eventually get fancier with this, suggesting various
default sets depending on the company or making it easy for one person
to create default subs for several people at once.

Until we know what we want, keep it simple.

(imported from commit 14319dd50c67fe33ac6f15288dba4916ce0a89ac)
2012-11-26 21:02:29 -05:00
Jeff Arnold a6ec875268 [schema] Add enable_desktop_notifications to user profile
(imported from commit f5b543e8446263ce8855561441a6d4c1a8d41f4d)
2012-11-26 14:43:55 -05:00
Jessica McKellar bdf3473bac Fix bug where sending a PM would display as sent to the wrong person.
In get_display_recipient, the userprofile was selected incorrectly by
user_id instead of the userprofile_id. In production, this hasn't
resulted in a user-visible error because we use MySQL and user ids are
always equal to userprofile ids.

This does happen if you are using SQLite locally and run populate_db,
which adds a bunch of users in parallel in an insufficiently
transactional way.

(imported from commit c25a04b4919e3efdfc6996b03492f7714d9034e8)
2012-11-20 15:08:50 -05:00
Keegan McAllister 1ea92c0953 Manage file locking using the 'with' statement
This is clearer and more exception-safe.

(imported from commit b67641b05da9dbf8e5a887f398bac81ab5985cf3)
2012-11-16 13:43:44 -05:00
Keegan McAllister af36b437b6 log_event: Use 'with' for files
(imported from commit d49b46ecd8aa74679ade3820004d10cb42940ad3)
2012-11-16 13:43:44 -05:00
Keegan McAllister ccf14b89c2 log_event: Don't flush the log file right before we close it
There's no need.

(imported from commit 8c3b933b7fddfb858ee4dc201b63b6fe921dc8e9)
2012-11-16 13:43:44 -05:00
Keegan McAllister f132cd311b log_event: Close the lock file after we're done with it
(imported from commit 023e50e1aeeafee334f9b55e12ab91aa84a7872a)
2012-11-16 13:43:44 -05:00
Tim Abbott 7227cb32f8 zephyr_mirror: Compute zephyr mirror fullnames server-side.
(imported from commit 9e6f6f9fe49d60ed30753582480e592613feb3c8)
2012-11-14 15:03:46 -05:00
Tim Abbott 0971b3d8e6 Add proper transaction handling in create_user_if_needed.
(imported from commit cab9cb4886acf61f36a0b0bc80d87ed9037cc414)
2012-11-14 14:33:31 -05:00
Keegan McAllister 42a5ea9d2e Specify requests.post data as a dict
These were lists of pairs because we were going to repeat keys, but that didn't
work anyway.

(imported from commit 687b3f7b8a2821d057719c725f1f39db3992ae5c)
2012-11-13 17:33:41 -05:00
Keegan McAllister f82e8fc4d1 Move Markdown rendering out of the Tornado server
(imported from commit fc726939aa1061c40b292899dbbc9ade3b29ea01)
2012-11-13 17:33:41 -05:00
Keegan McAllister c0204ea5b3 get_display_recipient: Sort huddle recipients by email address
We need a deterministic order for the client test suite, and it seems like a
good idea generally.

(imported from commit cc8fc555611f2d2f1b21e63ce6860d446baa3410)
2012-11-13 15:09:15 -05:00
Keegan McAllister a954277647 get_{display,log}_recipient: Use a join query
(imported from commit 1278f46655760842e21a62ca5add891515ee1945)
2012-11-13 15:09:15 -05:00
Keegan McAllister 31496e9189 Generalize Tornado-related settings
(imported from commit 76a1338a87e1a6663aa7602a499e2d769814bf08)
2012-11-13 10:59:02 -05:00
Tim Abbott 9e9438153c [schema] Store analytics on usage of our product.
(imported from commit 4fbf8215225fc8f505b9c749ccf80c556af13e87)
2012-11-08 18:00:57 -05:00
Tim Abbott 9a14073dda models: Add unique_together constraint on UserMessage table.
(imported from commit 79fcc1c9db11cc89a300b9d78a3c2f8cccb0917c)
2012-11-08 15:10:34 -05:00
Keegan McAllister d52728e3ae Recipient: use a dict to name types
(imported from commit 35ab065e7def38cb684672e48f2ca10f5f6479ad)
2012-11-08 10:48:34 -05:00
Keegan McAllister b57270322a models: Always pass a tuple to string %
(imported from commit c2881ef8f60fd7be2f70138868214d42d41b492d)
2012-11-08 10:48:13 -05:00
Keegan McAllister 979d997cf8 bulk_create_huddles: Use a set for existing_huddles
(imported from commit af66c6e4e87109feafbafdc6e84cf02903ca93ca)
2012-11-08 10:48:13 -05:00
Keegan McAllister 246c0a37bc Get rid of bare 'raise' outside 'except'
Within 'except', 'raise' re-raises the current exception.  But outside, it produces

    TypeError: exceptions must be old-style classes or derived from BaseException, not NoneType

which is pretty confusing as a generic "something has gone wrong" exception.

(imported from commit 9fcd003a952b82df67726c26161dced079978a32)
2012-11-08 10:48:11 -05:00
Keegan McAllister 5933a902c7 Remove unnecessary line-continuation backslash
(imported from commit afabcea50432b0e00f17e7a4cc4cb6c4a0c19cf2)
2012-11-08 10:47:09 -05:00
Keegan McAllister a70d82b8fc Use comprehensions more consistently in bulk create
(imported from commit f6b67b9ce74b379004e5fde6c99368c077a77f7b)
2012-11-08 10:47:09 -05:00
Keegan McAllister 51a96484f9 Consolidate callback code
(imported from commit e79ea42df3506bedd2077dcc63a45d32fe7b73de)
2012-11-08 10:47:09 -05:00
Keegan McAllister 9b9373d2ba Make callbacks_table class data rather than global
(imported from commit 39699304255b475ccab8a4760b155f3307cfe5d9)
2012-11-08 10:47:09 -05:00
Keegan McAllister f89366f5dd Use a new-style class for Callbacks
For consistency with the rest of our Python code.

(imported from commit 02659914f50b488039fe8c65bd1de796e2a16e65)
2012-11-08 10:47:09 -05:00
Tim Abbott 218588ead0 models: Comment the IntegrityError code in get_client (#326).
(imported from commit 85c3580084dcd4c28f319a19d684e5a5af7069c7)
2012-11-07 18:22:32 -05:00
Tim Abbott 573cc4db33 models: Add unique_together restrictions for various tables.
(imported from commit 30957d465d5e7bc728e854b8b99f6ad99b98dba6)
2012-11-07 17:31:16 -05:00
Tim Abbott f342217700 Add timestamps to our various non-message logged events.
(imported from commit 65102419ac0032ccedcbdb342dbea9b91bbe3e13)
2012-11-06 14:12:19 -05:00
Tim Abbott 1adc54185f Fix unnecessary usage of realm ids creating new users.
(imported from commit 5870b583efffa4a5280c81125eefecc23978cf11)
2012-11-02 20:55:31 -04:00
Keegan McAllister 02aa92a604 Correct comment on filter_by_subscriptions
This isn't dead code; it's used by tests.

(imported from commit ce6cfc049d2cb8050aff1fef1b77a3778b7ceacb)
2012-11-02 14:57:38 -04:00
Tim Abbott 426ab9559d Fix creating new users via zephyr mirroring.
(imported from commit 05e01efabdce081fc574486916c7db343f2cdb1d)
2012-10-31 14:06:35 -04:00
Waseem Daher e7e05d8617 Fix up some overzealous "instance->subject" renames.
(imported from commit f4ddcfd3749d374cc6abee3b1e7a5aacc3b83e16)
2012-10-30 16:48:00 -04:00
Tim Abbott 44b332693e Fix the fact that new auto-created MIT users are active.
(imported from commit 577f1a0165a56fc3cc1ed6f0a54f6c5f31345cc7)
2012-10-29 17:30:06 -04:00
Luke Faraone 5dad59e864 Implement MIT signups.
Here we introduce a new manage.py command, activate_mit, which takes a
number of usernames and sends out emails to the users with instructions on
how to activate their accounts.

(imported from commit f14401b55f915698e83ff27b86434f53e64685f3)
2012-10-29 16:59:37 -04:00
Tim Abbott 6c39bf5cbd Cut out duplicated code between clear_database and populate_db.
(imported from commit 4bb44c20d2f0477d8a3db72788f82b0cf8572b2d)
2012-10-29 14:54:18 -04:00
Zev Benjamin ec9e0a4b9f Have get_updates include an "update_types" array in its response
This is similar to the previous "reason_empty" variable, but captures
why we've returned from the call even when there are updates and all
the reasons if there are multiple.  For now, it's useful for debugging.

(imported from commit fd8d9e859660e51b57178d066b184f831b71a0b6)
2012-10-25 18:12:11 -04:00
Tim Abbott 7c8bde5d90 Add log/restore for fullnames and passwords.
(imported from commit 048ca3c86b9f077fcbccd5df4a509191a545da4c)
2012-10-25 15:52:26 -04:00
Tim Abbott 8c4a834206 Add function for activating MIT users.
(imported from commit 0de45fc341249332909d5a0ecef4d49918658a1a)
2012-10-25 14:46:58 -04:00