mirror of https://github.com/zulip/zulip.git
07c4291749
The previous query suffered from bad corner cases when the user had received a large number of direct messages but sent very few, comparatively. This mean that the first half of the UNION would retrieve a very large number of UserMessage rows, requiring fetching a large number of Message rows, merely to throw them away upon determining that the recipient was the current user. Instead of merging two queries of "last 1k received" + "last 1k sent", we instead make better use of the UserMessage rows to find "last 1k sent or received." This may change the list of recipients, as large disparities in sent/received messages may result in pushing the most-recently-sent users off of the list. These are likely uncommon edge cases, however -- and the disparity is the whole reason for the performance problem. This also provides more correct answers. In the case where a user's 1001'th message sent was to person A today, but my most recent message received was from them yesterday, the previous plan would show the message I received yesterday message-id as the max, and not the more recent message I sent today. While we could theoretically raise the `RECENT_CONVERSATIONS_LIMIT` to more frequently match the same recipient list as previously, this increases the cost of the most common cases unreasonably. With a 1000-message limit, the common cases are slightly faster, and the tail latencies are very much improved; raising `RECENT_CONVERSATIONS_LIMIT` would increase the result similarity to the old algorithm, at the cost of the p50 and p75. | | Old | New | | ------ | ------- | ------- | | Mean | 0.05287 | 0.02520 | | p50 | 0.00695 | 0.00556 | | p75 | 0.05592 | 0.03351 | | p90 | 0.14645 | 0.08026 | | p95 | 0.20181 | 0.10906 | | p99 | 0.30691 | 0.16014 | | p99.9 | 0.57894 | 0.19521 | | max | 22.0610 | 0.22184 | On the whole, however, the much more bounded worst case are worth the small changes to the resultset. |
||
|---|---|---|
| .. | ||
| actions | ||
| data_import | ||
| integration_fixtures/nagios | ||
| lib | ||
| management | ||
| migrations | ||
| models | ||
| openapi | ||
| tests | ||
| tornado | ||
| transaction_tests | ||
| views | ||
| webhooks | ||
| worker | ||
| __init__.py | ||
| apps.py | ||
| context_processors.py | ||
| decorator.py | ||
| filters.py | ||
| forms.py | ||
| logging_handlers.py | ||
| middleware.py | ||
| signals.py | ||