To apply this to your system, run:
python manage.py update_permissions
A rebuilt database will already have the correct permission defined.
(imported from commit 5b826b940257e14b3e9e44f9c32b20ed0183dedf)
This creates the required model fields to use the Django permissions
framework or various other third-party frameworks.
To apply this commit, run:
python manage.py migrate zephyr
(imported from commit a14fa7552c5389522d15edecedfd8a34418bb23d)
save it. This fixes the "Humbug note:" error on old messages that
do not have rendered_content in the DB.
(imported from commit f4664d712fc40e4af9eb12eae74f37d100499def)
This would have made reactivations hard, and doesn't really buy us much
additional security.
During deactivation, all a user's current sessions are deactivated and
they are marked as not active. This prevents them from logging in via
the web UI, and makes their API key unusable.
Randomizing their password is probably gratuitious, especially as we
start to allow authorized end-users to deactivate others.
(imported from commit c63d23816da0452a1df821f2fa6c1db2761733da)
Prior to this commit, populate_db would crash if you had ever deactivated
a user in your development instance's message log.
(imported from commit 227b2c0226a46ef5680443d3dbf62a13ce961e64)
This reduces roundtrips hopefully and will provide a friendlier error
message than what would otherwise be produced by Django.
(imported from commit 034aeef00043e3bf059583770f6c08c4f73ceeb5)
Previously we rested on the safety of randomised API keys to ensure that
deactivated users could not use the product. Here, we add an explicit check
to prevent that.
(imported from commit e32f355fb35b898d43503b96efb291f795aac881)
Deleted in 064c159, but it's still used. For some reason, nginx still
serves it on staging and prod despite it being deleted from git.
(imported from commit 11d6d29f974caca6c75d36f273674d3084ad9182)
Fixes the JS traceback "'null' is not an object" introduced in b67e52d.
Testing: Receive a message when narrowed to a different subject, with
the window focused.
(imported from commit 54b9e7924a2bf66ba5cc9799fc3687a084496465)
Fixes the JS traceback "Selected message id not in MessageList"
introduced by b67e52d
Testing:
* Narrow to a subject and send a PM
* Narrow to a PM recipient and send a stream message
* Narrow to a subject and send a stream message to a different subject
(imported from commit 1171c3f97813dc7db891042906762be8afb2a1b5)
This reverts commit f8fbf70c8502370a78159e24f3cf9589fb9d384f, since
we're waiting on some Firefox and no-hover fixes.
(imported from commit 6b13f5bb9d907303ab311afd7da584bc06538c91)
(The dead code made sense in the very early phases of sketching out
the feature, but it's no longer executed.)
(imported from commit 464145f227ddb25f0554bbbade0b0e3e0e399bc3)
This is slightly inconsistent, but keeps the unread count from decreasing
when narrowing and un-narrowing.
(imported from commit 185e8653c31a312c166e784b335ae7ae7e9b78e9)
Previously we added code which prevented us from serving custom fonts to
Humbug Desktop user agents due to concerns QtWebKit bugs with
@font-face.
On OS X, we use the system WebKit so we don't have this bug. In fact,
the aformentioned change caused us to have no custom fonts at all on Mac
systems!
Here we fix this by resuming the serving of such fonts to Macs.
(imported from commit b222e9dd721914d17aed8341244cfb5c71149a12)
I tried 30px at first, but I think a slightly bigger avatar helps
fill out the table a bit. It should be easier to tweek these in
CSS now, although Allen agrees with me that the tabular display
may be short lived when we add edit/delete features.
(imported from commit b4d69cddf63fa122374e20731a5755e7dec86304)
The JS tests would fail on the second run due to memcache having
dirty data. This change sets a new KEY_PREFIX whenever you launch
a server in test mode.
(imported from commit 4d41e6b79ab3bb7cb4c96b37050f0b1c9abc6b5e)
We did this as a hack to prevent traffic while you were in the
tutorial. In the revised tutorial, we want you to see your actual
streams while you go through it, and we'll instead store the events
client-side for processing after you finish.
(imported from commit 75af93d0661c9d14b2c85624aef3f71a78053980)
We did this to check if you sent a message that would progress the
tutorial. Since the tutorial is getting overhauled we don't need this
anymore.
(imported from commit 25ee55ab034fff42a220ddd7b222b3f7459af3a1)
* This makes bugdown.convert take a `message` parameter. Properties
for parsed mentions are added to the message object by the `Pattern`
for use in do_send_messages.
* Refactor repeated markdown rendering code into `Message` model methods.
(imported from commit 4f0ed5570104c0210f984b6de21e9048e2b53fa0)
We use get_user_profile_by_email() in all our tests now, as it
gives us code coverage on the function itself, and it should be
faster for tests that call it multiple times.
(imported from commit 51ebffb193980fd6f81b0ef5574d96cd92e87364)
This uses a new configuration that enables memcache, but we have
to be careful to bounce KEY_PREFIX on every new test, since data
gets rolled back in the databases between tests, but not in
memcached. We had to break up one test to work around UserProfile
objects actually being cached.
(imported from commit f201cf9cd9e0e4c61d3c384fa8d2bbd5134161e8)
The goal here is to make it easier to do ad hoc profiling on
our codebase, particularly by running tests.
(imported from commit 71da06feb3a369dec8dc4d8391f7f40e4c2d02ff)
After fixing the high numbers of database queries earlier in this
branch, I found that sending 500 RabbitMQ messages for a bulk change
in subscriptions was consuming more than half the time for these (and
then we'd end up with 500 events in a queue). To handle this, we
create a "user X subscribed to these N streams" event, rather than
sending one event for each individual subscription.
(imported from commit 44a34a9fab9b67e9f0da6fee53335d8c5030392b)
This improves the performance of unsubscribing to N streams by more
than a factor of 10 for large N.
(imported from commit a529e6d3ac4452f49c2294908d275280019bbd05)
Otherwise we could in theory make dozens or hundreds of
memcached/database queries to handle a narrow.
(imported from commit 232f38d8c005b9aef6f12f2f9a4d68a19134d038)
Previously we only used bulk queries when adding many users to a
single stream, resulting in very slow performance when subscribing
users to large numbers of streams (as happens when setting up a new
MIT realm user).
(imported from commit 849fa7b2a1a146c0a9adc1c727c20c9fbfb7b425)
This comment was only ever accurate for prototype versions of
bulk_add_subscriptions prior to it being committed to master.
(imported from commit 89b9dc49423c45553cb6c810d97eea4583ff0f69)
The message_stream_count() function uses Django's count() method,
which is more efficient than doing len() on array of full objects.
(imported from commit 9c20a89a2cd02d9d39341132330d03a7f6c8be25)
This change removes an "if True:" that was
introduced to make the prior commit a bit more readable.
It also combines two loops, since the second loop is no
longer conditional.
(imported from commit df58f1e5de72d5669f6468fbff54fb62cd22cedb)
The tests in GetUpdatesTest had some callback logic that has
been dead code for at least three months. We now fully exercise
the callback codepath and make sure that the callbacks do happen.
(imported from commit f5d8fbab28ecc34dc81d3d0c29058b66c10f378f)
These contain timerender spans with unique IDs. If this string is cached
and re-used in a narrowed view, there are elements with duplicate IDs,
and only one of them is updated at midnight.
(imported from commit 29469fb5f0d8a9b7fe7988849d2936c49d4a038d)
As of f69d01b and e4a9f80, timerender only deals with the date part.
Since the time does not change when the friendly dates are updated,
just append the time outside of the span.timerender.
(imported from commit 18ec93550782c6fa8447ebc608f95da79d28dc8d)
QTBUG-3467 prevents non-normal-face @font-face fonts from being used when
defined as such in CSS. To work around this, the desktop applications now
ship the Humbug font themselves, and this commit causes the server to no
longer send the problematic CSS rules to those clients.
We have some duplication insofar as we now have two minified CSS files, but
this is better than conditionally applying the CSS at page runtime.
(imported from commit 9a887f9fb8002d44171d366d1249ebbf21cc9c77)
Trac #1403.
This shows the 5 most recent subjects, as well as any others with unread messages. This
requires tracking all subjects and filtering at display time, rather than filtering when
building the subject list.
(imported from commit 8bda7d50e6785a6e70abea4b3af4d03a16d076d3)
(The file-input widgets that come with browsers are ugly and
nonstandard across browsers, so it is a common technique to
have your own button that controls the file upload, and it
delegates to a hidden copy of the browser file-input widget.
We also allow you to clear the file.)
(imported from commit b55ef655e75746330dc3cc396cb908670e5019cc)
The add-bots form used to have a landscape alignment, as it was
integrated into the same HTML table that showed your existing bots.
This became unwieldy once we allowed users to upload avatars.
(imported from commit 246a35be77ce1679d595271e6911dc339a6813ab)
Caveats:
- Since Chrome has trouble using W3C Notification when it's not
initiated by a user gesture, we try to use webkitNotification first.
- FF doesn't allow iconUrl to be of a different origin, so it won't display
our gravatars
(imported from commit c4f99ce6927a0d203d9f220d50b06737779bd7f8)
Previously, every function which had the @json_* decorators on it
would independently call get_client when views.py was imported; we
resolve this using a small bit of in-memory caching, which is safe
since Client objects are immutable.
This also more generally cleans up the code path to be more common
between the API and JSON views.
(imported from commit 674ea327fc4cb5fa982a75e388f0b604bae66567)
Previously some database queries that happened prior to the request
could be counted as part of the request's in-database processing time,
resulting in confusing output where get_events spent more time doing
database queries than the total time spent processing the request.
(imported from commit 509247462d276d6020a2970f62edb33a1d633087)
As does the hotkey "i". It's somewhat less appropriate because it's all actions,
but it's where our "info" menu used to be, and we can workshop a better one. "a"
feels weird to me, but maybe it's just me.
You can also hover on the .message_content to see a popover with extra message details
This is for Trac #1334.
(imported from commit f8fbf70c8502370a78159e24f3cf9589fb9d384f)
This mostly applies to the previous two commits. In principle,
this could be teased out and merged into them, but the hassle
here doesn't totally seem to be worth it.
(imported from commit ee2469ca3762c50c6db49b93eec02b32589eafe3)
* Switch to a 2-column layout by removing the old unused message tools
column and the pointer columns.
* Change column spans to 2 instead of 4.
* Change recipient bar to have a colorblock on the left.
* Add a special "last_message" class to messages that appear last
in a message block.
(imported from commit 55036587445c699d5c55d52b0236daf402a80cff)
Previously, we'd highlighted the top of the block. The theory here is
that it will make things less visually cluttered/noisy, which should
improve readability.
(imported from commit f94ec6b8e55aef9c2413718aea23b1124f5308e7)
This allows us to do some nice styling on the final message in a group
(in this commit series, this leads to some gently rounded corners.)
(imported from commit 86e7e8586dd4ee3cd8d4ce3be43fd2abab3f1d16)
Showing a user's bots in alphabetical order leads to a mildly
confusing experience when we append a new bot to the end of the
table, but then you hit refresh, and the new bot goes to a different
position. Since any given user is unlikely to have zillions of
bots, I don't think we need alphabetical order to help them find
old bots.
(imported from commit 4f19dbd7a016e7d867e88248190849dcd52c6d71)
(and remove the colon after the sender name to make them more
consistent with our desktop notifications.)
(imported from commit f2f9e1ecf7766c717fe7afb0559c4771081c255b)
This stop words file is just the default Postgres english stop file
with all the rest of the letters of the alphabet added. Adding the
extra letters ensures that, e.g., "bed" doesn't get transformed into
"bed | b".
(imported from commit 0be3ef9a43eb524ed4f081d5081a786cf602c487)
Compare two user objects by id to prevent false negatives
when the objects are fetched thru different paths.
(imported from commit a41f30d27e2b8021600d89f32d6526f48677fd95)
This also has the property of fixing this for all forms-that-contain-modals
since the selector is more generic.
(imported from commit 18fd26dd94fe4f6f3fe9a01884b867a9d37993d0)
Since in the future we might want requests to add subscriptions to
include things like colors, in_home_view, etc., we're changing the
data format for the add_subscriptions API call to pass each stream as
a dictionary, giving a convenient place to put any added options.
The manual step required here is updating the API version in AFS
available for use with the zephyr_mirror.py system.
(imported from commit 364960cca582a0658f0d334668822045c001b92c)
Previously, it always failed because we had hooked up the API endpoint
to a function that doesn't exist.
(imported from commit b5269f6d8e385facae4362742fe69a422f6315b7)
This way we can return properties of the streams other than just their
names in future versions of the API without breaking old clients.
The manual step required is to deploy the updated version of
sync-public-streams on zmirror.humbughq.com when we deploy this code
to prod.
(imported from commit 42b86d8daa5729f52c9961dd912c5776a25ab0b4)
The previous API was very redundant, which meant that any refactoring
would result in lots of unnecessary changes.
(imported from commit e04f6cbd87c8f65d4eebbe6972d26998faa28a56)
I believe this should require no special work on deploy, since some
grepping of logs suggests we are not currently using this API query.
(imported from commit 240086f900c6680cbc90bf6a2f334a9e1f172df6)
Trying to check whether a Django model object is inside a set of other
Django models is not correct in general, e.g.:
UserProfile.objects.only("id").get(id=17) in set([UserProfile.objects.get(id=17)])
returns False.
This bug appears twice in the function, once when computing which
users were mentioned and again when pushing the flags through to
Tornado.
(imported from commit b09ed550258f9df2611e1b0a60f87c48a51830f8)
Previously, we were only checking whether the message was sent by you
for PMs, not for streams that you marked as notifiable.
(imported from commit 668e102e885aa347cceff376c1873c1c62aff997)
(Before it had been disabled only on prod/staging, but we are
removing it everywhere, motivated by making tests run faster.
In particular, the call to embedly_client.is_supported() was
expensive, as it went over the Internet.)
(imported from commit ea12bf6e7ae84ce7e8023a0d314ecc4c07cbc0a8)
This file is a dict mapping filenames like "min/api.js" to
"min/api.HEX_HASH_HERE.js". This will be useful later for figuring out
which hashed version of a generated file belongs to a specific deploy,
for reuse in future deploys with the same source files.
(imported from commit d03c408b37a4e2d494abd4b1577a70b5ff5c2e34)
Whereas `SITE_ROOT` referred to the directory where settings.py is
located, *all* actual uses of `SITE_ROOT` were joining it with `..` to
get the root of the git checkout, a much more useful value.
`DEPLOY_ROOT` now represents the root of the git checkout.
(imported from commit 351437f9a5801e5c7c08a3a97619e863144e5cc8)
Previously we had an issue that every other update_active_status
request for a particular realm would result in doing the expensive
query to compute the list of active users in that realm. It turned
out this was because on every update_active_status request, we'd queue
an event that would have the effect of clearing the cache, even if
nobody's tatus changed. This fixes that issue, by only clearing the
cache for a realm if someone's status actually changed (or the 60s
timeout expires).
(imported from commit d5b829fe255a31c8cecb58458738f1e72a2cf6de)
This improves the performance of the get_old_messages queries when
loading the home page by about a factor of 2.5x.
(imported from commit 581840a6ed7b391c9d9fde67f368fce816e567e2)
This is in preparation for having a case in which we query the
database directly to get the message flags, without going through a
UserMessage object.
(imported from commit d5218974680b0c4b028a84f3aae1c8242ceb08ce)
This decreases the average size of the message dicts in memcached by
about half, without any significant change in the overall performance
of the query. Since these message dicts are a significant fraction of
what we put in memcached, this seems like a worthwhile optimization.
(imported from commit 3896328074aa4344b8ac7c7ba7685f0a167ec7ad)
memcached stores objects sent to it using pickling, which is very
slow. We work around this by sending memcached strings (i.e. JSON
dumps); pickling doesn't slow things down too much if all it is
getting is a string.
(imported from commit 0f0e534182eccb76c5731198e05a9324a1cef316)
This saves something like 15ms on our 1000 message get_old_messages
queries, and will save even more when we start sending JSON dumps into
our memcached system.
We need to install python-ujson on servers and dev instances before
pushing this to prod.
(imported from commit 373690b7c056d00d2299a7588a33f025104bfbca)
Previously, if you were narrowed when we reloaded you, you'd be
kicked out to the home view.
(imported from commit c00f1b92c9a4f559998463e141a402b69873fd56)
Some browsers don't support desktop notifications. Some people haven't allowed permissions
for humbug to have desktop notifications. This is a poor man's attempt at desktop notifications.
We are adding the bootstrap-notify package (http://nijikokun.github.io/bootstrap-notify/)
Resolves Trac #1336.
(imported from commit 6a54f7d1875e765dabd32d94ada8ebe4474a3d71)
We haven't been using this for months and not removing it before was
just an oversight.
(imported from commit d95c1911765a04a0c8713cc6c0dd346c123c97a3)
I noticed in our server logs that some users were sending
update_onboarding_steps requests to the server every time they sent a
message -- because they hadn't finished the onboarding process.
(imported from commit 8e1751a3344149f386022d96e7893e687d6631e8)
This reverts commit fec64815c879f9704847795cc163e2b517b4190d.
We've decided this experiment didn't work out.
(imported from commit 04f38e15c8ef19fbaa022a06bbcca3b0c67ec5ad)
This reverts commit 74fb298c711e61ae98c210d0ed11c875ce6ba591.
We've decided this experiment didn't work out.
(imported from commit 8acd3a030b692f9397155a20a9f89e63ed6a6cf7)
We had a few bugs where we were using a raw Django database query to
get a UserProfile object. This might seem OK, but going through
memcached is more efficient, and also guarantees that we get back the
.select_related() version of the object, so that if we later access
related fields like user_profile.realm.domain, we don't end up doing a
second database query as well.
Fixing these should in practice save a substantial number of database
queries on handling update_status_list requests, which happen very
often and access user_profile.realm.domain.
(imported from commit 0a2027da1b5bbc7a4f6c6927aca498530d7a4977)
Instead of calling test(result), we emulate the behavior of
calling SimpleTestCase.__call__, minus the results protocol.
(imported from commit e12e3d5caa8726c050a2a7cfa583ca4d571d6cbb)
Displaying the badge gets us a bunch of free mixpanel data points/month.
We only display the badge on the login page (which `/` redirects to)
and not on other portico pages.
(imported from commit 75871173a10ae888fdb7c92fe8e20586ce60c6bf)
The refactoring to use the cache_get() method incorrectly didn't
remove the addition of KEY_PREFIX inside cache_with_key. The result
was that the KEY_PREFIX was being added twice, once by cache_with_key
and again inside cache_get.
This had the impact of causing pointer saves to not take effect,
because our attempts to update the memcached cache when we save the
UserProfile object were using the correct cache key, but the actual
code reading values out of the caceh wasn't.
(imported from commit dcea000833f00622bdc0249488de3b186a7417b2)
The test_emoji test makes sure that we build the correct image
tag for all of our emojis. It's the same underlying algorithm
for all 800+ images, so we now test it in O(1) times instead
of O(N) time. Randomly sampling the emojis makes the tests
slightly non-determinisitic in a theoretical sense, but not in
a practical sense.
(imported from commit c78b50072953f2c081fca0272b9ecf762a4802db)
(We don't need the fixtures code once we override Django's
run_suite() method not to clobber our test database, since
all our tests use the same data anyway, and since Django
runs all the tests in a transaction and rolls back their
changes.)
(imported from commit d9daf09be3967de2de30934173b882d3d6b9ea27)
In tools/test-backend, we build the test database before calling
"manage.py test, so there is no reason for Django to wipe it
clean.
(imported from commit 7b09e9cc1a9259900ec1cc4f9ebbcacab1efb265)
(We now have a full replacement for TestResult, but we don't
support the full API of TestResult. If folks were to add new
tests that caused TestCase to call our class with an unsupported
method, this diff will provide hopefully a more clear error
message.)
(imported from commit 92033b12a1a901bd649ecb03e08779aa76bfef7a)
We use a simplified version of run_suite in our subclass of
DjangoTestSuiteRunner so that we can have more control over failing
fast and indicating progress. This change also speeds up the tests
by taking roughly 200 lines unittest/Django code out of the picture.
(We basically reinvent a simpler TextTestRunner and TextTestResult.)
(imported from commit 5216b6b582b071bd7c9e923c9886c3bc61faf581)
(This commit will probably be squashed, but it makes the subsequent
diffs more clear with respect to how we remove functionality from
run_tests.)
(imported from commit 9d0edcd5282a0f90ebd38e0ac229ef874dc00504)
Otherwise code paths that use those keys, like get_old_messages, will
incorrectly use the prefix-included keys.
This bug in our KEY_PREFIX system results in our memcached caching for
get_old_messages always missing.
(imported from commit 506c13e06d6f266596ead0b381c324c256e576c3)
If you click on "Streams" from the gear menu, we want to focus the
textbox to create a new stream. But if the Streams page is brought up
programmatically (e.g. to jump to a specific stream's settings), this
commit makes it not focus the textbox.
(imported from commit 900bdafb701180eac1d284120a91ea2a84d7177b)
The sidebar link now uses a one-time event handler for a custom event,
subs_page_loaded.zephyr. If the streams page is already open, we can't
rely on the event so we expand the stream in question immediately.
(imported from commit 3c22e1791d238a3be4a73edcfb5456e392cee608)
Get data directly from the main user list, rather than maintaining a
separate list just for autocomplete.
Fixes trac #1362 -- Does not depend on historical messages, so
will do the correct autocomplete after a single reload.
(imported from commit 6b35a709dba3384530082e8cfacf0151f9e0eb26)
Based on https://github.com/twitter/bootstrap/issues/6579. Doesn't
look like upstream is going to fix that or its 6 duplicates any
time soon.
(imported from commit b3e887d929cc42124aeb1f13abd7503f38f92a1f)
Wrapping render_to_response never actually worked correctly. On the
login page, mixpanel_token would be missing, but we wouldn't get an
error because it is surrounded by double quotes, which meant that it
was still valid Javascript.
(imported from commit 820ee42fab8f679983e5a3a4309a2feaf690f20f)
Steve Howell
Luke Faraone
Kevin Mehall
Jessica McKellar
Allen Rabinovich
acrefoot
Leo Franchi
Jeff Arnold
Tim Abbott
Waseem Daher
Scott Feeney
Zev Benjamin