Commit Graph

36022 Commits

Author SHA1 Message Date
Steve Howell aa5ffcbd2e admin user: Sort bot owners by name.
The original commit here was sorting bot owners by
id, which is of course meaningless to users:

    444ce74a8e

It was also returning 1/-1 in cases where the bot
owner on both sides of a comparison were missing,
which is a big no-no for sorting algorithms.
2020-04-15 15:13:26 -07:00
Steve Howell 0b71b092b3 admin users: Fix email sort.
The email sort now works correctly for admins in
realms with hidden emails.  (We want to sort on
delivery email.)
2020-04-15 15:13:26 -07:00
Steve Howell cf741e9a11 list_render: Add replace_list_data().
The data() function was used in only one place,
and it can be replaced now with two simple
lines of code.
2020-04-15 15:13:26 -07:00
Steve Howell d406df75b3 list_render: Remove support for jQuery items.
All of our widgets now use templates to turn
items into HTML strings, so we can simplify
the code.
2020-04-15 15:13:26 -07:00
Steve Howell c9d0c6852e admin users: Use plain HTML and static dates.
We want to avoid creating jQuery objects that just
get turned right back into strings by the list
widget, so we now have our template just include
`last_active_date` instead of kludging it in
after the fact, and we return the template
string in `modifier` rather than wrapping it.

To deal with plain HTML we switch to using
`render_now`.

Calling `render_now` leads to a more simple
codepath than `render_date`, beyond just dealing
with text.

The `render_date` function has special-case logic
that only applies to our time dividers in our
message view, which is why we were passing the
strange `undefined` parameter to it before this
fix.

The `render_date` function was also putting
the dates into `update_list` for once-a-day
updates, which is overkill for an admin screen.
We don't use this logic for drafts or attachments
either.  I'm not sure how well tested that logic
is, and it's prone to slow leaks.

This commit sets us up to simplify the list
widget not to have bit-rot-prone code related
to jQuery objects.
2020-04-15 15:13:26 -07:00
sahil839 0d2d7d31e2 admin users: Simplify logic for last active date.
We now:

    - Skip the broken "Never" case.  (The way
      we were distinguishing "Unknown" from
      "Never" was based on brittle checks that
      were just wrong due to bitrot--see Steve
      Shank on czo as an example.  If we want
      to make this distinction rigorous in the
      future, we should have a clear mechanism.
      If somebody's never actually been active,
      we probably want to treat that more like
      a dead-on-arrival login, anyway, and make
      it easy to clean them up.)

    - Use the `presence.last_active_date` instead
      of reaching into private data structures.

    - Avoid the unnecessary intermediate constants
      of LAST_ACTIVE_NEVER and LAST_ACTIVE_UNKOWN.

    - Avoid setting `last_active` in `populate_users`.

This commit was modified by @showell:

    - I cleaned up the commit message.

    - I simplified the diff a bit to avoid
      some renaming and lexical moves.
2020-04-15 15:13:26 -07:00
Steve Howell fad4ad1c7f default streams: Streamline list in settings.
There is no need to wrap these rows in jQuery,
when the list widget is fine with concatenating
strings.
2020-04-15 15:13:26 -07:00
Steve Howell 19cf6d0e04 list_render: Remove data-list-render markup.
We already know which list widget a `<th>`
tag is associated with when we set up the
event handler, so it's silly to read data
from the DOM to find that widget again
when the handler runs.

This commit eliminates a whole class of possible
errors and busy work.
2020-04-15 15:13:26 -07:00
Steve Howell 37eeb90695 list_render: Clean up create/update.
For some widgets we now avoid duplicate redraw
events from this old pattern:

    widget = list_render.create(..., {
    }).init();
    widget.sort(...);

The above code was wasteful and possibly
flicker-y due to the fact that `init` and
`sort` both render.

Now we do this:

    widget = list_render.create(..., {
        init_sort: [...],
    });

For other widgets we just clean up the need
to call `init()` right after `create()`.

We also allow widgets to pass in `sort_fields`
during initialization (since you may want to
have `init_sort` use a custom sort before the
first render.)

Finally, we make the second and third calls
eliminate the prior updates from the previous
widget.  This can prevent strange bugs with
double-reversing columns (although that's
been prevented in a better way with a recent
commit), as well as avoiding double work
with sorting.
2020-04-15 15:13:26 -07:00
Steve Howell 0ff62fc6a2 list_render: Use namespaces for events. 2020-04-15 15:13:26 -07:00
Steve Howell eb1344c41c list_render: Fix filtering/sorting.
This code has always been kind of convoluted
and buggy, starting with the first
sorting-related commit, which put filtering
before sorting for some reason:

    3706e2c6ba

This should fix bugs like the fact that
changing filter text would not respect
reversed sorts.

Now the scheme is simple:

    - external UI actions set `meta` values like
      filter_value, reverse_mode, and
      sorting_function, as needed, through
      simple setters

    - use `hard_redraw` to do a redraw and
      trigger external actions

    - all filtering/sorting/reverse logic on
      the *data* happens in a single, simple
      function called `filter_and_sort`
2020-04-15 15:13:26 -07:00
Steve Howell ec854f4b12 list_render: Remove opts.filter.event logic.
We don't use this anywhere.  You can do
`git grep -A 40 list_render.create` to verify
this (with a little bit of noise in the grep).

A better strategy for generalizing
this code is to extract the useful logic
into a function that callers can use in their
own custom event handlers, which I'll do
in an upcoming commit.
2020-04-15 15:13:26 -07:00
Steve Howell 3aef11dc0e list_render: Extract get_list_scrolling_container().
We put this in `scroll_util` to make it more likely
we will eventually unify this with other scrolling
logic.  (A big piece to move is ui.get_scroll_element,
but that's for another PR.)

And then the other tactical advantage is that we get
100% line coverage on it.

I changed the warning to an error, since I don't
think we ever expect scrolling at the `body` level,
and I don't bother with the preview node.
2020-04-15 15:13:26 -07:00
Steve Howell 839a817d0e linkifier settings: Simplify default sort.
In 3d7b9a1349 we
removed the `reverse` operation from `sort`, so
this code had bit-rotted, and it's simpler to
just sort by pattern.
2020-04-15 15:13:26 -07:00
shubhamgupta2956 cb71b8c1cf api_docs: Move /rest-error-handling examples.
This commit moves /rest-error-handling examples to components section so
that they can be re-used in individual endpoints where it's example can
be highlighted more easiy.
2020-04-15 12:17:50 -07:00
shubhamgupta2956 f2c9a9a832 api_docs: Add "event_types" common component.
To facilitate re-use of the same parameters in other paths. this commit
store the content of the parameter "event_types" in components.
2020-04-15 12:17:50 -07:00
Mateusz Mandera fbc8325d0e test-backend: Remove rate_limiter from not_yet_fully_covered.
rate_limiter.py now has sufficient test coverage to remove from the
list of exclusions.

Tweaked by tabbott to handle @abstractmethod in a better way.
2020-04-15 11:20:37 -07:00
Mateusz Mandera 5f9da3053d rate_limiter: Handle edge case where rules list may be empty. 2020-04-15 11:20:37 -07:00
Hashir Sarwar b577366a05 rate_limiter: Add an in-process implementation for Tornado.
The Redis-based rate limiting approach takes a lot of time talking to
Redis with 3-4 network requests to Redis on each request.  It had a
negative impact on the performance of `get_events()` since this is our
single highest-traffic endpoint.

This commit introduces an in-process rate limiting alternate for
`/json/events` endpoint. The implementation uses Leaky Bucket
algorithm and Python dictionaries instead of Redis. This drops the
rate limiting time for `get_events()` from about 3000us to less than
100us (on my system).

Fixes #13913.

Co-Author-by: Mateusz Mandera <mateusz.mandera@protonmail.com>
Co-Author-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-15 11:20:37 -07:00
Mateusz Mandera 95fa8b2a26 rate_limiter: Fix too early return if no rules are passed in.
In the redis implementation, if rules was an empty list,
this would return too early - before checking if the key isn't
manually blocked.
2020-04-15 11:20:37 -07:00
Rohitt Vashishtha cb85763c78 settings: Extract DropdownListWidget from default_code_language_widget.
We extract a general purpose widget to create dropdown lists with
search. This widget is used for default code block language, but can
be easily extended to cover notifications_stream and similar options.
The current usage is:

```js
const widget = DropdownListWidget({
    setting_name: 'realm_alpha_beta',
    data: [{name: 'hello', value: 'world'}, {...}, ...],
    subsection: 'msg-editing',
    default_text: 'Nothing is selected',
});
```

and

```handlebars
{{> dropdown_list_widget
  setting_name="realm_alpha_beta"
  list_placeholder=(t 'Filter the data')
  reset_button_text=(t '[Unset]')
  label=admin_settings_label.realm_alpha_beta }}
```

This can further be refined by shifting more variables from handlebars
to javascript in the future.
2020-04-14 12:50:10 -07:00
Rohitt Vashishtha 546c6f99a7 settings: Extract some helper functions from build_page.
By taking these functions out of exports.build_page, we can
reuse them for handling other widgets. We also declare
default_code_language_widget after the helper functions to
avoid the linter complaining.
2020-04-14 12:50:10 -07:00
Puneeth Chaganti b04813d351 webhook/semaphore: Add only summary line to the notification message. 2020-04-14 12:42:36 -07:00
Tim Abbott 0dd0227c8d send_email: Move custom email code to the bottom.
It's of interest to a relatively small subset of developers, in
comparison to Zulip's generic code for sending outgoing emails.
2020-04-14 10:57:20 -07:00
Tim Abbott 2732c477bc send_custom_email: Improve docstrings a bit. 2020-04-14 10:55:04 -07:00
wowol fcf4c9639e emails: Add tests for sending custom emails.
This requires configuring the linter to allow using "subject" in these
files.
2020-04-14 10:50:43 -07:00
wowol 74b757c43c emails: Add support for email headers in send custom email function.
This makes it a bit more convenient to encode most of the email
configuration inside a single template file.
2020-04-14 10:50:29 -07:00
Tim Abbott 8f57ec80f6 help: Add KaTeX links explaining how our LaTeX formatting works.
Also improve the headings for it.

Fixes #14568.
2020-04-13 15:55:39 -07:00
Siddharth Varshney 9cc64156e4 compose: Fix color of preview icon.
Fixes the color of preview iocn to match other message-control-button icons.
2020-04-13 14:58:10 -07:00
Siddharth Varshney ac690a99b8 message-header: Bump up the opcaity of icons.
Because of visibility issues both in day and night mode this commit
will increase the opacity of recipient_bar_controls icons from 0.1 to 0.2.
2020-04-13 14:58:10 -07:00
Steve Howell 4e11e7ee5b Revert "list_render: Clean up initialization."
I pushed this risk commit to the end of
a PR that had a bunch of harmless prep
commits at the front, and I didn't make
it clear enough that the last commit (this
one) hadn't been tested thoroughly.

For the list_render widget, we can simplify
the intialization pretty easily (avoid
extra sorts, for example), but the cache aspects
are still tricky on subsequent calls.
2020-04-13 06:22:28 -04:00
Wowol 0bf5ad3265 emails: Move send custom email function to library. 2020-04-12 16:11:44 -07:00
Ankita Bansal 2b179143a8 static/js: Fix custom numeric emojis not working in reactions.
Changes .data() Jquery methods to .attr() to prevent unnecessary data
type conversions of the emoji name.

Tested the fix manually and verified the test-js-with-node test suite.

Fixes: #14377
2020-04-12 15:02:56 -07:00
Steve Howell 0681e4ba36 list_render: Clean up initialization.
For some widgets we now avoid duplicate redraw
events from this old pattern:

    widget = list_render.create(..., {
    }).init();
    widget.sort(...);

The above code was wasteful and possibly
flicker-y due to the fact that `init` and
`sort` both render.

Now we do this:

    widget = list_render.create(..., {
        init_sort: [...],
    });

For other widgets we just clean up the need
to call `init()` right after `create()`.

We also allow widgets to pass in `sort_fields`
during initialization (since you may want to
have `init_sort` use a custom sort before the
first render.)
2020-04-12 14:59:32 -07:00
Steve Howell ef749dba31 invite settings: Avoid list_render.get() call.
The get() logic here was broken, because
when you enter the settings panel for invites
on the 2nd or 3rd time, the text filter
would not work.

This commit doesn't intend to fix the problem; it
just simplifies the code for a later commit
that fixes this holistically.

The way that we update `list_render` objects
is janky with respect to events, so we can end
up double-sorting lists (which puts them back
to normal) and strange things like that.
2020-04-12 14:59:32 -07:00
Steve Howell 2b07512d22 list_render test: Split out scrolling/filtering.
We split one test into two simpler ones, and we
no longer bother with the load_count override,
which was only used in tests.
2020-04-12 14:59:32 -07:00
Steve Howell ced5511cdd list_render: Rename __set_events().
I rename it to set_up_event_handlers.

This commit does not attempt to fix any buggy
behavior with how we set up event handlers; it's
purely cosmetic.
2020-04-12 14:59:32 -07:00
Steve Howell 42c2e9c429 list_render: Dedent 200+ lines of code.
This is all cosmetic.

Instead of:

    const.widget = {
        foo: function () = {
        },

        bar: function () {
        },
    };

We have:

    const widget = {};

    widget.foo = function () = {
    };

    widget.bar = function () {
    };
2020-04-12 14:59:32 -07:00
Steve Howell 29b22da1ff list_render: Rename prototype to widget. 2020-04-12 14:59:32 -07:00
Steve Howell 22c8a7ccef invite settings: Fix sorting with multi-user links.
Before this fix, we'd get a traceback if you looked
at invites in the settings (and if one of them was
a multi-user invite link).  This commit fixes
that problem by adding a custom sort.

We also rename the "Email" column to "Invitee",
since it's often the case the invitee isn't an
actually an email, but it's instead a multi-use
link.

Note that the invites UI only works the first time you
enter settings.  Many of the controls break the second
time you enter it.  You can't sort by column header
or use the text filter.

I'll fix that in a subsequent commit.
2020-04-12 14:59:32 -07:00
Steve Howell a06d455228 settings: Extract sort helpers for various lists.
Giving these functions a name and moving them to
the top-level scope has a couple tactical advantages:

    - names show in tracebacks
    - code is less indented
    - setup code is less cluttered
    - will be easier to add unit tests
    - will make some upcoming diffs nicer

These are technically more `compare_foo` than `sort_foo`,
but we already had a naming convention that was sort of
in place.
2020-04-12 14:59:32 -07:00
Tim Abbott 4f23f13c55 fetch-contributor-data: Fix running in production.
The previous logic attempted to check for a venv, rather than putting
itself into the virtualenv automatically.
2020-04-12 12:14:07 -07:00
Anders Kaseorg b1e7d8b51d settings: Harden session and CSRF cookies with __Host- prefix.
This defends against cross-origin session fixation attacks.  Renaming
the cookies means this one-time upgrade will have the unfortunate side
effect of logging everyone out, but they’ll get more secure sessions
in return.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-12 11:55:55 -07:00
Anders Kaseorg f47a9408cd settings: Use existing Django setting to mark CSRF cookie HttpOnly.
Instead of sneakily injecting HttpOnly into the cookie via the path
setting, use the setting that was designed for this purpose.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-12 11:55:55 -07:00
Anders Kaseorg dce6b4a40f middleware: Remove unused cookie_domain setting.
Since commit 1d72629dc4, we have been
maintaining a patched copy of Django’s
SessionMiddleware.process_response in order to unconditionally ignore
our own optional cookie_domain setting that we don’t set.

Instead, let’s not do that.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-12 11:55:55 -07:00
Abhishek-Balaji 68257e28ce emoji_name: Raise correct exception if emoji_name is missing.
Right now, the message is "Invalid characters in emoji name" when
the emoji_name is empty. Changing check_valid_emoji_name() in
zerver/lib/emoji.py which validates the name to accomodate the case
of missing name. The new message is "Emoji name is missing".
2020-04-12 11:52:45 -07:00
arpit551 d60efa1478 thumbor: Fix __file__ typo.
Replaced '__file__' typo with __file__ which used to add
wrong path to sys.path.
2020-04-12 11:23:03 -07:00
Puneeth Chaganti 504216c04e webhooks: Remove deprecated Stash webhooks' fixture. 2020-04-12 11:04:30 -07:00
Puneeth Chaganti 67822a0544 messages: Fix 500 on queries with empty string as search operand.
The error is PGroonga specific since `pgroonga_query_extract_keywords` does
not handle empty string inputs correctly. This commit prevents search
narrows from having empty operands.

Closes #14405
2020-04-12 11:04:09 -07:00
Aman Agrawal c792abebc0 docs: Fix broken link to WSL 2 doc. 2020-04-12 11:03:27 -07:00