The Redis-based rate limiting approach takes a lot of time talking to
Redis with 3-4 network requests to Redis on each request. It had a
negative impact on the performance of `get_events()` since this is our
single highest-traffic endpoint.
This commit introduces an in-process rate limiting alternate for
`/json/events` endpoint. The implementation uses Leaky Bucket
algorithm and Python dictionaries instead of Redis. This drops the
rate limiting time for `get_events()` from about 3000us to less than
100us (on my system).
Fixes#13913.
Co-Author-by: Mateusz Mandera <mateusz.mandera@protonmail.com>
Co-Author-by: Anders Kaseorg <anders@zulipchat.com>
We extract a general purpose widget to create dropdown lists with
search. This widget is used for default code block language, but can
be easily extended to cover notifications_stream and similar options.
The current usage is:
```js
const widget = DropdownListWidget({
setting_name: 'realm_alpha_beta',
data: [{name: 'hello', value: 'world'}, {...}, ...],
subsection: 'msg-editing',
default_text: 'Nothing is selected',
});
```
and
```handlebars
{{> dropdown_list_widget
setting_name="realm_alpha_beta"
list_placeholder=(t 'Filter the data')
reset_button_text=(t '[Unset]')
label=admin_settings_label.realm_alpha_beta }}
```
This can further be refined by shifting more variables from handlebars
to javascript in the future.
By taking these functions out of exports.build_page, we can
reuse them for handling other widgets. We also declare
default_code_language_widget after the helper functions to
avoid the linter complaining.
I pushed this risk commit to the end of
a PR that had a bunch of harmless prep
commits at the front, and I didn't make
it clear enough that the last commit (this
one) hadn't been tested thoroughly.
For the list_render widget, we can simplify
the intialization pretty easily (avoid
extra sorts, for example), but the cache aspects
are still tricky on subsequent calls.
Changes .data() Jquery methods to .attr() to prevent unnecessary data
type conversions of the emoji name.
Tested the fix manually and verified the test-js-with-node test suite.
Fixes: #14377
For some widgets we now avoid duplicate redraw
events from this old pattern:
widget = list_render.create(..., {
}).init();
widget.sort(...);
The above code was wasteful and possibly
flicker-y due to the fact that `init` and
`sort` both render.
Now we do this:
widget = list_render.create(..., {
init_sort: [...],
});
For other widgets we just clean up the need
to call `init()` right after `create()`.
We also allow widgets to pass in `sort_fields`
during initialization (since you may want to
have `init_sort` use a custom sort before the
first render.)
The get() logic here was broken, because
when you enter the settings panel for invites
on the 2nd or 3rd time, the text filter
would not work.
This commit doesn't intend to fix the problem; it
just simplifies the code for a later commit
that fixes this holistically.
The way that we update `list_render` objects
is janky with respect to events, so we can end
up double-sorting lists (which puts them back
to normal) and strange things like that.
This is all cosmetic.
Instead of:
const.widget = {
foo: function () = {
},
bar: function () {
},
};
We have:
const widget = {};
widget.foo = function () = {
};
widget.bar = function () {
};
Before this fix, we'd get a traceback if you looked
at invites in the settings (and if one of them was
a multi-user invite link). This commit fixes
that problem by adding a custom sort.
We also rename the "Email" column to "Invitee",
since it's often the case the invitee isn't an
actually an email, but it's instead a multi-use
link.
Note that the invites UI only works the first time you
enter settings. Many of the controls break the second
time you enter it. You can't sort by column header
or use the text filter.
I'll fix that in a subsequent commit.
Giving these functions a name and moving them to
the top-level scope has a couple tactical advantages:
- names show in tracebacks
- code is less indented
- setup code is less cluttered
- will be easier to add unit tests
- will make some upcoming diffs nicer
These are technically more `compare_foo` than `sort_foo`,
but we already had a naming convention that was sort of
in place.
This defends against cross-origin session fixation attacks. Renaming
the cookies means this one-time upgrade will have the unfortunate side
effect of logging everyone out, but they’ll get more secure sessions
in return.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Instead of sneakily injecting HttpOnly into the cookie via the path
setting, use the setting that was designed for this purpose.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Since commit 1d72629dc4, we have been
maintaining a patched copy of Django’s
SessionMiddleware.process_response in order to unconditionally ignore
our own optional cookie_domain setting that we don’t set.
Instead, let’s not do that.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Right now, the message is "Invalid characters in emoji name" when
the emoji_name is empty. Changing check_valid_emoji_name() in
zerver/lib/emoji.py which validates the name to accomodate the case
of missing name. The new message is "Emoji name is missing".
The error is PGroonga specific since `pgroonga_query_extract_keywords` does
not handle empty string inputs correctly. This commit prevents search
narrows from having empty operands.
Closes#14405
Instructions were added by doing the setup on Ubuntu 18.04 WSL 2.
While the setup should be similar for other distributions supported by
our `./tools/provision` script inside WSL, it has not been tested.
Polished by tabbott.
We had a bug where if your peer mentioned you in
message, but then edited the message not to mention
you, the latter wouldn't reset your unread counts
for "Mentions". And the same problem would happen
vice versa.
The fix basically extracts `update_message_for_mention`
and makes sure it handles all combinations of
unread/mentioned flags, instead of assuming
any invariants about which directions of change
are possible.
And then we call that new function from
`message_events.js` whenever we get message
edit events.
Fixes#14544
We use a somewhat more realistic message, mostly
to prep for testing some mention/unread stuff in
a subsequent commit.
We also set message booleans.
Unfortunately, `recent_senders` is kind of awkward
for checking a single message, since its only
public API is for sorting. I don't bother with it.
But I do check the `topic_data` interaction.
We figure out the dev host using the same logic as
dev_settings.py, so that we don't use wrong things
like 127.0.0.1 for droplet users.
And we display the link in cyan.
Mateusz Mandera
Hashir Sarwar
Rohitt Vashishtha
Puneeth Chaganti
Tim Abbott
wowol
Siddharth Varshney
Steve Howell
Ankita Bansal
Anders Kaseorg
Abhishek-Balaji
arpit551
Aman Agrawal