Commit Graph

280 Commits

Author SHA1 Message Date
Steve Howell d740d7c082 Show user-uploaded avatars on the website.
Show user-uploaded avatars on the website for users who have
UserProfile.avatar_source == 'U'.  (Continue to show gravatars
for other users.)  This includes the home page, the visible-phone
div, and the settings page.

This fix does NOT address a few things:
* There is no GUI to actually upload user images yet on the website.
* The !gravatar syntax in bugdown will continue to show gravatar images
  only.
* We are not changing identicon behavior.

(imported from commit 9f5ac0bbe21ba56528048233aab2430e4dd431aa)
2013-06-12 15:34:51 -04:00
Steve Howell ac7c8d18bf [schema] Add avatar_source column to UserProfile.
(imported from commit b4a6788002851b2e8b6f7be61b11af57378c0ee6)
2013-06-12 15:34:40 -04:00
Tim Abbott 0ad1094e45 bugdown: Add custom per-realm filters to linkify certain strings.
I've tried to do this in a way that's scalable and easily configured,
so that we can add new such filters for customers on-demand without
needing to add anything other than a bit of configuration.

Once we're confident in the arguments to this system, I think we'll
want to move the regular expression lists into the database so that we
don't need to do a prod push to modify the regular expression lists.

The initial set of regular expressions are:
(1) Linkifying e.g. "trac #224" in the Humbug realm, so we're exercising this code.
(2) The various ticket number things CUSTOMER7 uses for the CUSTOMER7 realm.

(imported from commit 992b0937b9012c15a7c2f585eb0aacb221c52e01)
2013-06-05 17:48:21 -04:00
Leo Franchi 4b2a2d01e6 [schema] Add a rate_limits column on UserProfile
(imported from commit 3682212d91ce5a91cc2e7d5611da0265518cf18a)
2013-06-04 09:43:09 -04:00
Jessica McKellar 79d494282d Use a case-insensitive wrapper around retrieving a realm by its domain.
We were previously able to create realms for different casings of the
same domain.

(imported from commit c13a86c9eb9fb4934bebcda7a578e2a071dd1962)
2013-06-02 15:12:28 -04:00
Leo Franchi 56af48cd50 Split wildcard mentions from personal mentions, only email on personals
(imported from commit 18971d9e40dd2c93ef9c30edc963da33e3c65309)
2013-05-31 12:24:31 -04:00
Leo Franchi 81058f5f9b Reset our 24hr email blackout when a user logs in
(imported from commit 93bed7c4d049ba4d7b30d2e02fbbe0d2debca6f8)
2013-05-24 21:23:59 -04:00
Leo Franchi e5b1e16885 [schema] Make last_reminder field nullable
(imported from commit e7d14c222d64ac317fc96d0c6bb17220aa6ba097)
2013-05-24 21:23:58 -04:00
Leo Franchi 3c81664641 Calculate mentions in a message server-side, and save as UserMessage flag
(imported from commit a3a868a8c6f48b68010108e0ec18ca65b7f0d498)
2013-05-24 21:23:58 -04:00
Tim Abbott 8479b4f0cd [schema] Record and display whether a message was edited.
We also record the historical edits to the message in this JSON format:

 [{"prev_content": "new test message 14", "timestamp": 1369157249},
  {"prev_content": "new test message 13", "timestamp": 1369157118}]

but we don't actually do anything with the information as of yet.

(imported from commit 2d5ca449b87b33ad035ab0e076a22e150c8e7267)
2013-05-24 21:23:55 -04:00
Jessica McKellar b5e22bf6b6 [schema] Add the migration for adding an onboarding_steps field to UserProfile.
(imported from commit e0dea167d5d0f54a8471524b6ab6717e4c3226ef)
2013-05-14 10:27:39 -04:00
Zev Benjamin 73be68fbb9 [schema] Add support for bot users
(imported from commit 634a8211b41fd0040c95b51b96a88d3517fa8cf4)
2013-05-11 02:38:32 -04:00
Jessica McKellar c7555c6cf9 Add support for completely collapsing messages through the info popover.
(imported from commit 0a01c4cc8f99019233659c175d73826e16cc95ee)
2013-05-10 18:00:57 -04:00
Leo Franchi 29f857cca6 [schema] Add database fields for offline email notifications and timestamp
(imported from commit ef2ef9cb6c7b02d4a47c0061c58a40a37d91c353)
2013-05-09 10:35:48 -04:00
Tim Abbott 77aad147f1 Enable historical messages for all future realms.
(imported from commit b8f13a996ae00644f808297a87c6a92c89aaf4ab)
2013-05-07 11:47:57 -04:00
Luke Faraone 6bc2d21b87 Make audible notifications a tunable.
This decouples from Chrome notifications, which gives us cross-platform
support in at least modern browsers.

We log this action so its replayable in our message logs.

This implements the model change indicated by the previous schema commit.

(imported from commit b21213cdde54f43670bbb0bf1f607147fc732b38)
2013-05-03 15:08:49 -07:00
Tim Abbott bfaa698ea6 get_old_messages: Don't fetch messages from database unless we have to.
Previously, we were fetching Message.objects.select_related() from the
database, even if we actually ended up fetching the message dicts from
memcached and thus not actually using them.  Especially in the cached
case, this resulted in a lot of overhead where the Django ORM put
together Message objects with lots of data in them that were never
used.  This commit switches the model to only fetch the full message
objects from the database for those messages which are not found in
the memcached caches.

Here are the timings for get_old_messages before this patch was applied:

(cached)
127ms (db: 42ms/2q) /json/get_old_messages (starnine@mit.edu via website)
385ms (db: 105ms/1q) /json/get_old_messages (starnine@mit.edu via website)

(uncached)
315ms (mem: 6ms/41) (db: 90ms/22q) /json/get_old_messages (starnine@mit.edu via website)
507ms (db: 94ms/14q) /json/get_old_messages (starnine@mit.edu via website)

Here are the timings for get_old_messages after this patch was applied:

(cached)
 80ms (db: 9ms/2q) /json/get_old_messages (starnine@mit.edu via website)
133ms (db: 4ms/1q) /json/get_old_messages (starnine@mit.edu via website)

(uncached)
230ms (mem: 9ms/41) (db: 48ms/23q) /json/get_old_messages (starnine@mit.edu via website)
385ms (db: 55ms/15q) /json/get_old_messages (starnine@mit.edu via website)

(imported from commit c4748513392a906393314aa7cd41d98a69865411)
2013-05-02 15:32:23 -04:00
Tim Abbott 4f6c46c090 Add sender's domain to display recipient structures.
(imported from commit 6ed2d869b36ad03eaa17eded57e7e46cf638dfd7)
2013-05-02 15:31:10 -04:00
Tim Abbott 1986b65c6a Enable historical messages for customer33.invalid.
(imported from commit ed95813f20ba29b425be4d90d6a54beb22ec81ad)
2013-04-30 11:26:20 -04:00
Tim Abbott 8760c1c7be get_display_recipient: Use an in-memory cache as well as memcached.
(imported from commit 825cb77b29cd635add252e4b9497feeb7ed7e177)
2013-04-25 17:02:20 -04:00
Tim Abbott 7c001822f2 Use bulk requests for updating memcached in get_old_messages.
Otherwise we end up doing 1000 requests to memcached, which can be
quite expensive.

(imported from commit be247f63b5fb88c6f4a45326261b66ea67fe1028)
2013-04-25 14:43:37 -04:00
Tim Abbott 9b8f0fab0f Retrieve message objects from memcached in a bulk request.
On my laptop, this saves about 80 milliseconds per 1000 messages
requested via get_old_messages queries.  Since we only have one
memcached process and it does not run with special priority, this
might have significant impact on load during server restarts.

(imported from commit 06ad13f32f4a6d87a0664c96297ef9843f410ac5)
2013-04-24 10:44:56 -04:00
Waseem Daher eab43f28f4 Properly compute the name of our tutorial stream server-side.
It's subtle, but the slice was in the wrong place and wasn't
actually truncating the stream name at all, so the client and
server disagreed about where the tutorial messages should go.

(It might be the case that we should accept the tutorial stream
name from the client directly, rather than computing it in two
places.)

(imported from commit 8273223f182e8ad36eaea1cbf75e1426fcfdfbab)
2013-04-24 10:32:30 -04:00
Luke Faraone 71a91197fa Enable absolute imports.
See PEP 328[1] for details. This feature was introduced in Python 2.5 and
will become mandatory in Python 3.

[1]: http://www.python.org/dev/peps/pep-0328

(imported from commit 7444eeba8a08d5f91b94c7921848f2274979bd76)
2013-04-23 09:51:17 -07:00
Tim Abbott dc0736ef97 Redefine is_public to be 'humbughq.com realm and not invite_only'.
(imported from commit 52a5b461eff926b28c2bb914b26b697de5fbfc3d)
2013-04-12 15:43:42 -04:00
Leo Franchi 302cfcd48c Send client information for initial presence and process time differential
(imported from commit 99a51b7cc8b6c51c4e82757a984d07603b2980e3)
2013-04-12 09:11:40 -04:00
Leo Franchi 5d4b2305fe Send presence updates when a new user logs in for the first time, and when returning from inactive
This commit will incorrectly list past-online users as active, a shortcoming that is
addressed in the next commit

(imported from commit b018767df686f88c0ca939c067c573e4d7cea357)
2013-04-12 09:11:40 -04:00
Jessica McKellar 7175dc534a Send invitation e-mails asynchronously through RabbitMQ.
This avoids 10s of seconds of delay when you invite several people at
once through the web UI.

(imported from commit 75acdbdb04caf62bbb08affc7796330246d8a00e)
2013-04-10 16:57:49 -04:00
Zev Benjamin e754479e9c [schema] Add notifications to Subscription model
(imported from commit 4d6a7aa17f3fad4b6f8fb7a100b3b578446e3625)
2013-04-10 16:11:27 -04:00
Leo Franchi 8fe82085c4 [schema][manual] Automatically subscribe users to default streams only after tutorial
(imported from commit 6511851c0aee2628bef597bf1310d6f96b0fd1d4)
2013-04-04 17:11:39 -04:00
Tim Abbott a1aae4a39f [South] [schema] Add an index on Userprofile.email.
We accidentally lost this when we did the User/UserProfile merge (this
commit also deletes the old code to add the auth_user index in
do-destroy-rebuild-database).

This below is mostly just notes for future reference, but when
deploying this change to staging, we should consider running the
following instead of using the migration directly:

CREATE UNIQUE INDEX CONCURRENTLY zephyr_userprofile_email_uniq ON zephyr_userprofile(email);
ALTER TABLE zephyr_userprofile ADD CONSTRAINT zephyr_userprofile_email_uniq UNIQUE USING INDEX zephyr_userprofile_email_uniq;
CREATE INDEX CONCURRENTLY zephyr_userprofile_email ON zephyr_userprofile(email);

But I think it might be the case that it's fine to just run it
directly, since the ALTER TABLE part seems to hang if there's an open
transaction working on a UserProfile object anyway.

(imported from commit 1bf34ce242de51e97c91c8bab86b6b273e17fb43)
2013-04-04 15:45:42 -04:00
Tim Abbott 0ee684a4b5 [schema] [manual] Add colors to the subscription model.
This is preparatory for removing the StreamColor model, so we also set
things up so anything changing the StreamColor model changes the
Subscription model too.

The manual task is to run the copy_colors.py management command after
deployment to each of staging and prod.

(imported from commit 1be7523ca59f5266eb2c4dc2009e31209ed49635)
2013-04-04 14:17:01 -04:00
Tim Abbott 1cec86eb2d [manual] Remove now-unused User model.
I think all that one needs to do to deploy this commit is on developer
laptops, run `generate-fixtures --force`.

(imported from commit 34916341435fef0875b5a2c7f53c2f5606cd16cd)
2013-04-02 12:57:10 -04:00
Tim Abbott 1443edce00 Access the UserProfile's new email field rather than using User.
This is preparatory for stopping using the User model.

(imported from commit a1b0808c8cc2ddd19a25163f91c4f18620c9ce90)
2013-04-02 12:07:08 -04:00
Tim Abbott b82edb6fd6 [manual] Add User fields to the UserProfile model.
And keep the fields updated, by copying on UserProfile creation and
updating the UserProfile object whenever we're updating the User
object, and add management commands to (1) initially ensure that they
match and (2) check that they still match (aka that the updating code
is working).

The copy_user_to_userprofile migration needs to be run after this is
deployed to prod.

(imported from commit 0a598d2e10b1a7a2f5c67dd5140ea4bb8e1ec0b8)
2013-04-01 14:34:25 -04:00
Tim Abbott dc5839f66d tests: Fix User/UserProfile confusion in filter_by_subscriptions path.
Our testing code had a number of places where it was using User
objects where it should have been using UserProfile objects --
e.g. using a User id as the type_id in a Recipient table.  This commit
addresses this in the filter_by_subscriptions code paths.

(imported from commit e305bc8e2a8bdbfd04c93c59d56955e7971552af)
2013-03-29 16:18:38 -04:00
Tim Abbott 198480ef99 Use get_user_profile_by_email more consistently.
The previous situation was bad for two reasons:

(1) It had a lot of copies of the code, some of them missing pieces:
  UserProfile.objects.get(user__email__iexact=foo)

This was in particular going to be inconvenient since we are dropping
the __user part of that.

(2) It didn't take advantage of our memcached caching.

(imported from commit 2325795f288a7cf306cdae191f5d3080aac0651a)
2013-03-29 16:18:38 -04:00
Jessica McKellar f1645f5fc9 Add a `starred` bit to the UserMessage.flags BitField.
(imported from commit 3998edcdafbf6b452b05aa72db86bfaafdd83e01)
2013-03-28 16:59:58 -04:00
Tim Abbott 78f5c2e877 Use the update_fields option to Django ORM .save().
This can result in a significant performance benefit because we only
need to update the columns that changed..

(imported from commit 42bef1fcc58ad79bd864f89263fe82e90743ee5b)
2013-03-28 07:36:11 -04:00
Tim Abbott 7b968cd088 Revert "Revert "to_dict: Update rendered_content in the database after rendered.""
This reverts commit 745b4a98d0e3f67acaa207ba53e9250060dcfe06.

(imported from commit ea59de391b93b82b08c35d92620d7d8ae263f34d)
2013-03-28 07:36:10 -04:00
Tim Abbott e8aa77c9b4 Set timeouts for our memcached caches.
The policy this implements is:
* 1 week for most persistent data (Clients, etc.)
* 1 day for messages

(imported from commit d57bb2c6b9626ffa2155c6d0ef9b60827d1f2381)
2013-03-28 07:36:10 -04:00
Tim Abbott e2d010ef2d Fix most unnecessary database queries in huddle creation.
This saves 2 database queries per user in the huddle when sending the
first message to a particular huddle.

(imported from commit f71aa32df846fb4b82651a93ff9608087ffcaa5a)
2013-03-28 07:36:10 -04:00
Tim Abbott 8b2cb6472b Cache the lookup of Huddle objects by hash.
This is used in the send_message code path.

(imported from commit 4d9a4d079fe5b10b94c11016e9ec8c5601addc2c)
2013-03-27 12:59:24 -04:00
Tim Abbott b2e76f50b9 Populate cache of Client objects on server restart.
(imported from commit 6d0ef309dadcd6c24fc9e17b0859becb053adccf)
2013-03-27 12:59:23 -04:00
Tim Abbott a774598a10 Populate cache of Recipient objects on server restart.
(imported from commit 752851c7c3ee565201dd59fb0455c82a8885d42d)
2013-03-27 12:59:23 -04:00
Tim Abbott 4076704476 [django 1.5] models: Always return a string from __repr__() methods.
See http://bugs.python.org/issue5876 for an explanation for why this
is needed -- basically __repr__() needs to return a string, not a
unicode object in Python 2.

This causes problems on Django 1.5 because the more expressive
exception code in model.objects.get() will crash with a __repr__()
containing non-ascii unicode characters.

(imported from commit f44085e67d9d14629b821a29bbf65738f1794d6c)
2013-03-27 08:19:26 -04:00
Jessica McKellar 0c3382fabb Always give hashlib.sha1 and friends bytes.
This fixes an experienced bug where you couldn't subscribe to a stream
with non-ASCII characters (failing with a UnicodeEncodeError), as well
as many other potential bugs.

(imported from commit f084a4b4b597b85935655097a7b5a163811c4d71)
2013-03-23 00:10:10 -04:00
Tim Abbott a8febf8fce Revert "to_dict: Update rendered_content in the database after rendered."
This reverts commit 4e5a370bc43defb5083cac3df45c649cf2366abf.

(imported from commit 745b4a98d0e3f67acaa207ba53e9250060dcfe06)
2013-03-21 10:30:23 -04:00
Jessica McKellar 37e58b613d Truncate the tutorial stream name to fit the DB max stream name length.
(imported from commit e267656337f97d3ea14e14e957a405cf4fd2d06c)
2013-03-20 10:15:08 -04:00
Tim Abbott 02554b28cb to_dict: Update rendered_content in the database after rendered.
This change could significantly load the database if pushed to prod at
the same time as:

    send_message: Save rendered message content to the database.

goes out.  So this should go out in a different prod push from that
commit.

(imported from commit 4e5a370bc43defb5083cac3df45c649cf2366abf)
2013-03-20 07:34:10 -04:00