Commit Graph

848 Commits

Author SHA1 Message Date
Jessica McKellar ebe3acf408 Generalize elide_subject in case the max topic length changes.
(imported from commit 6ffdd42d39580149a83869d6aff88bad8940fe02)
2013-11-22 11:28:19 -05:00
Tim Abbott c0e951f843 Add user_activity_test worker that does nothing.
This should help us debug the source of our memory leak problems.

(imported from commit 1bdc7ee2f72bdebb1cdc94601247834a434614d6)
2013-11-22 11:24:48 -05:00
Tim Abbott 9ba820703a Call reset_queries in our queue workers.
This fixes a small memory leak in our queue workers, where we don't
reset the accumulated times contained in our query logging data.

Longer-term, we may want to make something mergable for mainline where
we only store on the connection object the totals; that would be a
fixed amount of emmory per connection and thus not have this problem.

(imported from commit 914fa13acfb576f73c5f35e0f64c2f4d8a56b111)
2013-11-22 11:24:48 -05:00
Jessica McKellar ff800cdec3 Set unique client strings for our webhooks.
(imported from commit 48d8a2ec46399104d4f5de30722e65043572c69e)
2013-11-21 22:15:48 -05:00
Steve Howell 8f636bccf6 Triple-quote string in test_dangerous_block() for readability.
(imported from commit f8fdbff9cc992310907957ca0632bb646c726b9f)
2013-11-21 18:10:33 -05:00
Jessica McKellar f8435cde82 Truncate long Freshdesk subjects.
We're getting errors on prod for subjects that are too long.

(imported from commit 742d92745be543d4b33d0888c6653bbee3f86c25)
2013-11-21 17:45:02 -05:00
Steve Howell ace12bcab6 Support arbitrarily nested fenced quote/code blocks.
Now we can nest fenced code/quote blocks inside of quote
blocks down to arbitrary depths.  Code blocks are always leafs.
Fenced blocks start with at least three tildes or backticks,
and the clump of punctuation then becomes the terminator for
the block.  If the user ends their message without terminators,
all blocks are automatically closed.

When inside a quote block, you can start another fenced block
with any header that doesn't match the end-string of the outer
block.  (If you don't want to specify a language, then you
can change the number of backticks/tildes to avoid amiguity.)

Most of the heavy lifting happens in FencedBlockPreprocessor.run().
The parser works by pushing handlers on to a stack and popping
them off when the ends of blocks are encountered. Parents communicate
with their children by passing in a simple Python list of strings
for the child to append to.  Handlers also maintain their own
lists for their own content, and when their done() method is called,
they render their data as needed.

The handlers are objects returned by functions, and the handler
functions close on variables push, pop, and processor.  The closure
style here makes the handlers pretty tightly coupled to the outer
run() method.  If we wanted to move to a class-based style, the
tradeoff would be that the class instances would have to marshall
push/pop/processor etc., but we could test the components more
easily in isolation.

Dealing with blank lines is very fiddly inside of bugdown.

The new functionality here is captured in the test
BugdownTest.test_complexly_nested_quote().

(imported from commit 53886c8de74bdf2bbd3cef8be9de25f05bddb93c)
2013-11-21 17:13:17 -05:00
Jessica McKellar ae2202caad Update bankruptcy unread count to not include muted topics or read messages.
This should address user reports of huge bankruptcy counts even when
they are relatively caught up. The root issue is that we sometimes
don't mark messages as read for some reason.

(imported from commit 8799305a8665f9ee239575e6e95f603f89c1d427)
2013-11-21 13:01:01 -05:00
Tim Abbott 272e1c2d92 Revert "Don't send heartbeat events to clients that don't request them."
This reverts commit 1147814b22fb9737a807057ddbdbe0e9554086e0.

This seems to with some probability screw up our Zephyr mirroring
script.

(imported from commit 4f82452f1b0ca98e6b895db020e071d2daa325e4)
2013-11-21 12:53:26 -05:00
Tim Abbott ca8225cf47 [manual] Add endpoint to cleanup a finished events queue.
This requires a puppet apply on each of staging and prod0 to update
the nginx configuration to support the new URL when it is deployed.

(imported from commit a35a71a563fd1daca0d3ea4ec6874c5719a8564f)
2013-11-20 18:34:15 -05:00
acrefoot 5973f20b69 fix use of EXTERNAL_URL to EXTERNAL_HOST, and some typos
(imported from commit 3e6e0f337671b289202bc86c6224854b6ee623c5)
2013-11-20 17:39:07 -05:00
Zev Benjamin d2d695dcb6 Add an in-memory Client object cache
Client objects are immutable and there are very few of them, so caching them in
memory is cheap and saves a trip to memcached.

(imported from commit 300b9b402f4e509f86a7fd86b5f898dc3f43738f)
2013-11-20 16:55:10 -05:00
Steve Howell 792e0c5976 Add FencedBlockPreprocessorTest.
(imported from commit a7f73f04408288192b33f688bd5baadd362f9458)
2013-11-20 16:06:54 -05:00
Steve Howell 0d920dae87 Extract FencedBlockPreprocessor.placeholder().
(imported from commit cf6dab74ae97a57498e1e1807c7c4f272f3f448b)
2013-11-20 16:06:53 -05:00
Steve Howell 2955cac47e Extract vars in FencedBlockPreprocessor.process_fence().
(imported from commit 092a8e5f7479f1d918219cae058b6a867743c7f9)
2013-11-20 14:30:56 -05:00
Steve Howell b3344c2f49 Move langclass var into FencedBlockPreprocessor.format_code().
(imported from commit e468a8b6a12494facb0dfa97eff90e02cd14f169)
2013-11-20 14:30:55 -05:00
Steve Howell 46838ec6d0 Simplify FencedBlockPreprocessor.format_fence().
(imported from commit 741f4f0640756d545019b6fe6850af3198b351ba)
2013-11-20 14:30:55 -05:00
Steve Howell c0f312ec29 DRY up m.group('code') in FencedBlockPreprocessor.
(imported from commit b23caae1ab63a9adb80025b628bc1a7c6a988ce0)
2013-11-20 14:30:55 -05:00
Steve Howell 6bc25d3d7b DRY up m.group('lang') in FencedBlockPreprocessor.
(imported from commit 9f9ab5604b7df711a3a369247def1277880b0b5f)
2013-11-20 14:30:55 -05:00
Steve Howell 7bb30a96ac Extract FencedBlockPreprocessor.format_fence().
(imported from commit 8a35de7df9ad956496263c82668b118910edd6b3)
2013-11-20 14:30:55 -05:00
Steve Howell 64410fae5e Extract FencedBlockPreprocessor.format_quote().
(imported from commit 59db5415d624402b37ade7fe5547b16f58ae565f)
2013-11-20 14:30:55 -05:00
Steve Howell 9d4b6d0684 Extract FencedBlockPreprocessor.format_code().
(imported from commit 3c98b9a7391a71e9c55a7b5c97abd0ee74178b44)
2013-11-20 14:30:54 -05:00
Zev Benjamin 25c63f08de Use ujson instead of pickle for serializing event queues
(imported from commit 2d90574ae97cc973c9686c2bd557cc199d4b0850)
2013-11-20 14:25:36 -05:00
Zev Benjamin 69e5e2c3d2 Remove old migration code
(imported from commit 5366411706ef296dc14d0d0dd9e2566cb8abb8b7)
2013-11-20 14:25:36 -05:00
Zev Benjamin 06ee83b531 Construct EventQueues externally from ClientDescriptors
(imported from commit 7ef882074ed1c3663939da199d54e4959df8f127)
2013-11-20 14:25:36 -05:00
Tim Abbott 3caee45cf4 Fix missing support email address in signup notifications.
(imported from commit f1a4fbe5fbc68292080f87960f038fe530ba87e2)
2013-11-20 11:01:11 -05:00
Zev Benjamin db29e60249 Move the last event queue pickle file to /var/tmp instead of deleting it
(imported from commit b8edaa686e2dadfdb2111ee1da93456591e44a50)
2013-11-19 15:05:19 -05:00
Jessica McKellar 151fbd832c do_update_message: use is_admin to check who can edit a topic.
(imported from commit e96245506c7eecc8aed3fbdb6dfb7c44aca3234e)
2013-11-19 14:31:02 -05:00
Jessica McKellar d1619b3f3c Use a more accurate "is admin?" check.
UserProfile.show_admin was intended to be a check for users that have
administrative rights in other realms, which we've harmlessly but
erroneously been using to check if they are an admin in their realm.

Use the more straightforward check instead, with a more intuitive
name.

(imported from commit d81050c7dbbb19e59c5e31750be303a4630e1456)
2013-11-19 14:31:02 -05:00
Jessica McKellar 2c841042ae Allow admins to rename any topic.
(imported from commit 5b557e8244e5196c9aa0ee29d4efb00986434528)
2013-11-19 14:31:02 -05:00
Zev Benjamin 15a837d12e [manual] socket: Send explicit ACKs
There will be browser errors on staging when this is deployed due to the socket
protocol changing.

(imported from commit f1eda5b5c2ec9c60c23b3ca96277a61debadf5bb)
2013-11-19 11:06:14 -05:00
Zev Benjamin 7870b4d158 socket: Fix a traceback on server restart
I believe there may also be others.  I'm still not sure why clients would be
sending open requests without session or csrf values in their cookies, though.

(imported from commit 7e9660c1c4d5c2abf55ff21b433ba0117180eb82)
2013-11-19 11:06:13 -05:00
Tim Abbott c66113104c Clean up logging of send times to not waste a log line.
(imported from commit 824488a323a4361ef058f27da6e0768dffc43b6d)
2013-11-18 18:05:19 -05:00
Tim Abbott 26f7b783a2 logging: Fix request processing time to not count initialization.
Previously, we counted not just the time required to process a
particular request, but also the time required to import+find the view
function via urls.py.  The latter is usually fast, but when a new
Django thread starts up, it can take significant time, resulting in us
flagging slow requests on each server restart and also when a new
Django thread starts up on prod to handle requests.

This change means that we no longer include that startup time as part
of request processing time -- but we still log it in the case that it
was more than 5ms, so that we can identify why a particular request
was slower than expected if high startup times become a problem.  We
annotate the time in log lines as "+start" rather than just "start" to
make clear that it's not counted in the total.

(imported from commit c677682e23b26005060390d85d386234f4f463ad)
2013-11-18 18:05:19 -05:00
Tim Abbott d44c6636c6 Add setting to enable profiling of all requests.
This is useful for the occasional case where we cannot figure out what
is causing a particular problem, but it can be easily reproduced on
staging.

(imported from commit 8b51184a8b686814f2c6ff103ba355538463ceb0)
2013-11-18 18:05:19 -05:00
Tim Abbott 74c304ae40 Don't send heartbeat events to clients that don't request them.
(imported from commit 1147814b22fb9737a807057ddbdbe0e9554086e0)
2013-11-18 13:57:01 -05:00
Tim Abbott 14f6625569 Don't send server restart events to clients that don't request them.
(imported from commit b3381e904704c18918fcce97d2493241e49004be)
2013-11-18 13:57:01 -05:00
Kevin Mehall b78fa0857f Allow overriding the default avatar image
(imported from commit 66d413682a822e0019f28033f19908bdd9fa0156)
2013-11-18 11:48:53 -05:00
Kevin Mehall a63f4284f0 Allow enterprise deployments to disable the use of Gravatar.
(imported from commit 49c14cc7629f2ffe6cedaab5d42ad5bf7f6569a4)
2013-11-18 11:48:53 -05:00
Jessica McKellar 9cba43d49c GitHub: Always report the topic as 'GitHub' for customer14.invalid.
(imported from commit d9d35f4dcabf6246b1c9bc623a01bc68e1142b35)
2013-11-18 11:44:25 -05:00
Zev Benjamin 5c6a85f99a socket: Change path for 'not yet authenticated' error messages
(imported from commit 826d3a0ea9084644810a00a14f2f19936e21118c)
2013-11-18 11:42:57 -05:00
Zev Benjamin 319852c60e socket: Fix keyword argument name for error logging
(imported from commit e61eace1aa48fe17d5d2338f7990e32d3404d390)
2013-11-18 11:42:57 -05:00
Tim Abbott f3fa78d30c Rename do_[rd]activate to do_[rd]activate_user.
For similarity with do_activate_user.

(imported from commit 2874af84dbe1ce666de10e0d8ffe2daa8bad9731)
2013-11-16 11:35:26 -05:00
Tim Abbott a78ec1d0d3 get_members_backend: Report a bit more data on users.
(imported from commit af1c2a30abed4aa2796f0babe311f993965a008b)
2013-11-16 11:35:26 -05:00
Steve Howell 1144e0b3f9 Update active_user_dicts_in_realm cache when touching is_active flag.
(imported from commit c9407e2c18dcfcc7a4b1ec79c471f825dff04b1e)
2013-11-16 11:35:25 -05:00
Steve Howell 0b16dfc999 Add /json/users/<email>/reactivate endpoint.
(imported from commit 7afa29f7718430ea2df7c489ab2af4d93d4ce497)
2013-11-16 11:35:25 -05:00
Steve Howell bdfd3ed9c2 Add do_reactivate() action.
(imported from commit fc73d0fdd3a6125b3b31a75c3b108ceb266e528b)
2013-11-16 11:35:25 -05:00
acrefoot bafb6f8dc3 fixup day1/day2 email templates
(imported from commit 30846fc298d04ac763025e82a60731d158840462)
2013-11-15 21:51:37 -05:00
acrefoot 1a904185f2 fix path for do_confirm redirect
(imported from commit d61d8f155fc06fbadc68155f32f510e212adf6c0)
2013-11-15 21:31:37 -05:00
acrefoot 9616ea7afd remove enqueue_followup_emails management command
(imported from commit 20ce0b698cf795a6e1c7ff5ba7b546901e1fd8a4)
2013-11-15 21:31:37 -05:00