This class gives us more control over attaching
suggestions to our eventual result. The main
thing we do now is remove duplicates as they're
encountered.
This will make sense in the follow up commit,
where we can short circuit actions as soon as
we get enough results.
This has a few benefits:
- we remove some duplicate code
- we can see finalize_results in profiles
It turns out finalize_results is expensive
for some searches. If the search itself doesn't
do a ton of work but returns a lot of results,
we see it in finalize_results. It brings to
attention that we should be truncating items
earlier instead of doing lots of unnecessary
work.
This isn't a huge speedup, but it's an easy
code change.
We remove the two-liner highlight_with_escaping,
which was only called in one place, and when
we inline it into the caller, we can pull the
first line, which builds the regex, out of the
loop.
The code we removed in highlight_with_escaping
is exactly the same code as in
highlight_with_escaping_and_regex.
I actually copy/pasted this code five years
ago and am now removing the duplication. :)
When we're highlighting all the people that show
up in a search from the search bar, we need
to fairly expensively build a regex from the
query:
query = query.toLowerCase();
query = query.replace(/[\-\[\]{}()*+?.,\\\^$|#\s]/g, '\\$&');
const regex = new RegExp('(^' + query + ')', 'ig');
Even though the final regex is presumably cached, we
still needed to do that `query.replace` for every person.
Even for relatively small numbers of persons, this would
show up in profiles as expensive.
Now we just build the query once by using a pattern
where you call a function outside the loop to build
an inner function that's used in the loop that closes
on the `query` above. The diff probably shows this
better than I explained it here.
This fixes a cross-site scripting vulnerability in the upcoming Inline
URL Previews feature found by Graham Bleaney and Ibrahim Mohamed using
Pysa.
This commit doesn't get a CVE because the bug was present in a code
path introduced in the 2.1.x development branch, so it doesn't impact
any Zulip release.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
The streams:all adveritsement notice in search should only appear
after we've already received the response from the server, to avoid a
mix of problems ranging from misplaced loading indicator to scrolling
issues to the notice just being distracting while you're waiting for
the server to return results.
We need to add a pre_scroll_cont parameter to the message_fetch API,
since adding this notice would otherwise potentially throw off the
scroll positioning logic for which message to select.
Fixes#13441.
In 452e226ea2 and
648a60baf6, we changed how `search:`
narrows work to:
(1) Never mark messages as read inside searches (search:)
(2) Take you to the bottom, not the first unread, if a `near:` or
similar wasn't specified.
This is far better behavior for these use cases, because in these
narrows, you can't actually see all the context around the target
messages, so marking them as read is counterproductive. This is
especially important in `has:mention` where you goal is likely
specifically to keep track of which threads mentioning you haven't
been read. But in many other narrows, the current behavior is
effectively (1) setting the read bit on random messages and (2) if the
search term matches many messages in a muted stream with 1000s of
unreads, making it hard or impossible to find recent search matches.
The new behavior is that any narrow that is structurally a search of
history (including everything that that isn't a stream, topic,
pm-with, "all messages" or "private messages") gets that new behavior
of being unable to mark messages as read and narrows taking you to the
latest matching messages.
A few corner cases of interest:
* `is:private` is keeping the old behavior, because users on
chat.zulip.org found it confusing for `is:private` to not mark
messages as read when one could see them all. Possibly a more
complex answer is required here.
* `near:` narrows are getting the new behavior, even if it's a stream:
+ topic: narrow. This is debatable, but is probably better than
what was happening before.
Modified significantly by tabbott for cleanliness of implementation,
this commit message, and unit tests.
Fixes#9893. Follow-up to #12556.
In 1fe4f795af, we added the
wildcard_mentions_notify setting, which controls whether wildcard
mentions should be treated as mentions for the purposes of
notifications. The original implementation focused on the more
important area of email/push notifications, and neglected to address
desktop notifications for wildcard mentions.
This change makes the wildcard_mentions_notify flag behave correctly
for desktop/sound notifications, including unit tests.
Fixes#13073.
We register ZulipRemoteUserBackend as an external_authentication_method
to make it show up in the corresponding field in the /server_settings
endpoint.
This also allows rendering its login button together with
Google/Github/etc. leading to us being able to get rid of some of the
code that was handling it as a special case - the js code for plumbing
the "next" value and the special {% if only_sso %} block in login.html.
An additional consequence of the login.html change is that now the
backend will have it button rendered even if it isn't the only backend
enabled on the server.
Adds required API and front-end changes to modify and read the
wildcard_mentions_notify field in the Subscription model.
It includes front-end code to add the setting to the user's "manage
streams" page. This setting will be greyed out when a stream is muted.
The PR also includes back-end code to add the setting the initial state of
a subscription.
New automated tests were added for the API, events system and front-end.
In manual testing, we checked that modifying the setting in the front end
persisted the change in the Subscription model. We noticed the notifications
were not behaving exactly as expected in manual testing; see
https://github.com/zulip/zulip/issues/13073#issuecomment-560263081 .
Tweaked by tabbott to fix real-time synchronization issues.
Fixes: #13429.
If a message begins with /me, we do not have any cases where the
rendered content would not begin with `<p>/me`. Thus, we can safely
remove the redundant checks both on the backend and frontend.
The "Stream settings" UI was always intended to be initialized in the
"Subscribed" tab when opened not through navigation that explicitly
aims to via "All streams". We had implemented that through how the UI
is rendered as well as the internal state tracking variable
`subscribed_only`, which was initialized to `true`.
The bug was that we didn't reset that to `true` when re-opening
"Stream settings" via a code path that calls `setup_page` (e.g. via
the menus on the left sidebar).
Ths fixes a bug where the stream-list in the stream settings would
list all streams but would show the 'Subscribed' label after
navigating to "All streams", closing "Manage streams", and then
reopening it.
Fixes#13297.
In e42c3f7418, we made the assumption
that compose_pm_pill.get_recipient() would return no users for stream
messages. It turns out, due to the confusing name of
compose_state.recipient (which we just renamed to
compose_state.private_message_recipient), this assumption was wrong.
As a result, when composing a stream message using the reply hotkeys,
we'd end up sending typing notiifcations to the person who sent the
message we're replying to as though a PM was being composed.
We fix this by avoiding passing an (expected to be unused) value for
private_message_recipient to compose_state.start.
The compose_state.recipient field was only actually the recipient for
the message if it was a private_message_recipient (in the sense of
other code); we store the stream in compose_state.stream instead.
As a result, the name was quite confusing, resulting in the
possibility of problematic correctness bugs where code assumes this
field has a valid value for stream messages. Fix this by changing it
to compose_state.private_message_recipient for clarity.
Fixes commit id 648a60baf6. When
allow_use_first_unread_when_narrowing() is false last message of
narrow is shown in view.
Comments rewritten by tabbott to explain in detail what's happening.
This simple change switches us to take advantage of the
server-maintained data for the pm_conversations system we implemented
originally for mobile use.
This should make it a lot more convenient to find historical private
message conversations, since one can effectively scroll infinitely
into the history.
We'll need to do some profiling of the backend after this is deployed
in production; it's possible we'll need to add some database indexes,
denormalization, or other optimizations to avoid making loading the
Zulip app significantly slower.
Fixes#12502.
message_id, rather than timestamps, is our standard way to sort by
time. And this refactor is important because we're about to start
using data from the server to populate this data structure.
This avoids a stream having potentially near-infinite height when
opened in a stream with a large number of unread topics; the benefit
is that you can easily access the next stream.
We show an unread count next to "more topics" to make it hard to miss
that there might be more, older topics with unread messages.
With CSS work by Anders Kaseorg.
Fixes#13087.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This commit was automatically generated by `tools/lint --only=eslint
--fix`, except for the `.eslintrc.json` change itself.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Updates the message editing process to do a local 'echo'.
On slow connections, now there is visual confirmation of the edit,
similar to when sending messages. The contains_backend_only_syntax
logic and check are the same as there.
We showing "(SAVING)" until the edit is completed, and on successful
edit, the word "(EDITED)" appears. There's likely useful future work
to do on making the animation experience nicer.
Substantially rewritten by tabbott to better handle corner cases and
communicate more clearly about what's happening.
Fixes: #3530.
This change makes it possible for users to control the notification
settings for wildcard mentions as a separate control from PMs and
direct @-mentions.
This commit was automatically generated by `tools/lint --only=eslint
--fix`, after an `.eslintrc.json` change.
A half dozen files were removed from the changes by tabbott pending
further work to ensure we avoid breaking valuable PRs with merge
conflicts.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Hovering over user names (and user circles for PM List) now displays
Name, Status Message and Last online time in a js tooltip.
Hovering over group names displays the names of all group members.
Unavailable users are shown as "Last active: Today".
Hovering on a user circle in the Buddy List results in a js tooltip
with Active/Idle/Offline/Unavailable for
green/orange/white/white-with-line.
Resolves#11607.
When strings are tagged for translation using `tr this`, the strings
were passed into the frontend i18n as-is (including new line and tab
characters that are not functional in the text, existing just to
format the HTML files reasonably).
This did not match the algorithm used in `manage.py makemessages` for
extracting strings for translation, which (correctly) removed that
whitespace to provide a good experience for translators. The fix is
for the `tr this` implementation to use that same whitespace-stripping
algorithm.
Tested manually by checking if those strings that were not translated
earlier were translated, and also fixed an automated test that had the
wrong result, which should help prevent regressions.
Fixes#13389.
These should work consistently with how the individual user setting
works; see the last commit.
With changes from tabbott to fix real-time sync.
Fixes#12553.
This fixes two regressions in 1946692f9a.
The first bug was actually introduced much earlier, namely that we
were not sending a `bot_owner_id` field at all for bot users without
an owner. The correct behavior would have been send `None` for the
owner field.
The second bug was simply that we needed to update the webapp to look
for the `bot_owner_id` field, rather than an old email-address format
`bot_owner` field.
Thanks to Vinit Singh for reporting this bug.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`. It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
The js_typings directory is not set up correctly for us to add new
type declarations for untyped external modules. The correct
configuration would be something like
{
"compilerOptions": {
"baseUrl": ".",
"paths": {
"*": ["js_typings/*"],
},
"typeRoots": ["js_typings"],
},
"exclude": [
"js_typings",
],
}
but that configuration is incompatible with using the same directory
for _internal_ modules like the ones declared here.
Also, correct some mistakes the generation of this list.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Set `--esModuleInterop` and `--isolatedModules` for consistency with
Babel. `tsc --init` adds `--esModuleInterop` by default.
Set `--moduleResolution node` so we can find type definitions in
modules that provide them.
Set `--forceConsistentCasingInFileNames`, which seems like a good
idea, and which `tsc --init` will add by default in TypeScript 3.7.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
`--jsx preserve` and `--removeComments false` are already the default.
`--strict` already implies `--noImplicitAny`, `--noImplicitThis`,
`--alwaysStrict`.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Commit d17b577d0c (#13321) incorrectly
transformed this line, even though I thought my script had a specific
guard against this.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Webpack code splitting will make the inclusion order of CSS files less
obvious, and we need to guarantee that these rules follow the rules
they override.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
ESLint won’t convert these automatically because it can’t rule out a
behavior difference arising from an access to a self-referential var
before it’s initialized:
> var x = (f => f())(() => x);
undefined
> let y = (f => f())(() => y);
Thrown:
ReferenceError: Cannot access 'y' before initialization
at repl:1:26
at repl:1:15
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Because of the separate declarations, ESLint would convert them to
`let` and then trigger the `prefer-const` error.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
With webpack, variables declared in each file are already file-local
(Global variables need to be explicitly exported), so these IIFEs are
no longer needed.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
This feels a bit more semantically appropriate: it more clearly says
"here's some information: there is no (relevant) recipient", rather
than "no information available". (Both `null` and `undefined` in JS
can have either meaning, but `undefined` especially commonly means
the latter.)
Concretely, it ensures a bit more explicitness where the value
originates: a bare `return;` becomes `return null;`, reflecting the
fact that it is returning a quite informative value.
Also make the implementation more explicit about what's expected here,
replacing truthiness tests with `!== null`. (A bit more idiomatic
would be `!= null`, which is equivalent when the value is well-typed
and a bit more robust to ill-typing bugs. But lint complains about
that version.)
It'd already been the case for some while that calling `stop` had the
same effect as calling `update` (previously `handle_text_input`) with
a falsy recipient. With the API changes in the previous few commits,
this becomes quite natural to make explicit in the API.
This was named after when it gets called from the UI, rather than
after what it can be expected to do.
Naming it after what it's meant to do -- and giving a summary line to
expand on that -- provides a more helpful semantic idea for reasoning
about the function. Doubly so for using the function in a different
client with its own UI, like the mobile app.
The main motivation for this change is to simplify this interface
and make it easier to reason about.
The case where it affects the behavior is when
is_valid_conversation() returns false, while current_recipient
and get_recipient() agree on some truthy value.
This means the message-content textarea is empty -- in fact the
user just cleared it, because we got here from an input event on
it -- but the compose box is still open to some PM thread that we
have a typing notification still outstanding for.
The old behavior is that in this situation we would ignore the
fact that the content was empty, and go ahead and prolong the
typing notification, by updating our timer and possibly sending a
"still typing" notice.
This contrasts with the behavior (both old and new) in the case
where the content is empty and we *don't* already have an
outstanding typing notification, or we have one to some other
thread. In that case, we cancel any existing notification and
don't start a new one, exactly as if `stop` were called
(e.g. because the user closed the compose box.)
The new behavior is that we always treat clearing the input as
"stopped typing": not only in those cases where we already did,
but also in the case where we still have the same recipients.
(Which seems like probably the common case.)
That seems like the preferable behavior; indeed it's hard to see
the point of the "compose_empty" logic if restricted to the other
cases. It also makes the interface simpler.
Those two properties don't seem like a coincidence, either: the
complicated interface made it difficult to unpack exactly what
logic we actually had, which made it easy for surprising wrinkles
to hang out indefinitely.
Returning true from this function means we go on to send, or extend
the lifetime of, a typing notification; returning false means we don't.
It's hard to see why having a partially-entered name in the recipient
box should mean we're *more* inclined to send a typing notification to
the set of recipients that are already entered; if anything, it seems
like it should make us *less* inclined to do so. So we're better off
without this conditional.
The conditional was introduced in commit 72295e94b, as part of a
conversion from user emails to user IDs; there, it seems to replace a
condition that went in the opposite direction, returning *false* if
there were any invalid emails in the recipient box. So perhaps it's
just inverted.
Moreover, the (re-)inverted version would also be wrong: if the user
is typing a PM addressed to some users, and they hit send, the message
will go to those users whether or not they have any unconverted text
in the recipients box. So the typing notifications should too.
The real purpose these two callbacks serve is exactly what an ordinary
parameter is perfect for:
* Each has just one call site, at the top of the function.
* They're not done for side effects; the point is what they return.
* The function doesn't pass them any arguments of its own, or
otherwise express any internal knowledge that doesn't just as
properly belong to its caller.
So, push the calls to these callbacks up into the function's caller,
and pass in the data they return instead.
This greatly simplifies the interface of `handle_text_input` and of
`typing_status` in general.
Users generally don't expect wildcard mentions in muted streams and
topics to be treated as a mention, either for the purposes of desktop
notifications or the unread mention counts.
This fixes the unread mention counts part of the issue.
Fixes part of #13073.
When email address visibility is set to everyone, there is no change in
behavior, but when it is set to "admins-only", we don't show any email
in user profile modal (just like popovers) for everyone but admins.
When email address visibility is set to everyone, there is no change in
behavior, but when it is set to "admins-only", we don't show any email
in popovers for everyone but admins.
This adds the general machinery required, and sets it up for the file
`typing_status.js` as a first use case.
Co-authored-by: Anders Kaseorg <anders@zulipchat.com>
These indeed used to be strings, but were converted to arrays in
b8250fc61, and these names didn't get updated to match.
A classic example of why type-checking is a great job to get
machines to do. :-)
When a user performs a search that might contain historical public
streams messages that the user has access to (but doesn't because
we're searching the user's own personal history), we add a notice
above the first search result to let the user know that not all
messages may have been searched.
Fixes#12036.
When typing_status adds 10000 to this value, it would previously
obtain wacky strings like
"Fri Oct 04 2019 16:45:59 GMT-0700 (Pacific Daylight Time)10000"
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
The historical behavior of having `Enter` exit was optimized for the
"View source" use case; but `Esc` now handles that reasoanbly, and we
really should make it convenient to type in the user-editable text
box here.
Fixes part 1 of #11834.
This ensures that typing '```java' and pressing enter would result in
getting dropped into a java codeblock instead of javascript codeblock.
We implement this by pushing the exact match of a query to be pushed to
the top of the returned matches in `sort_languages`.
With some comments added by tabbott in the tests explaining the
current reasoning.
Fixes#13109.
Apparently, the changes in fe2adeeee1 to
fix a Firefox focus bug accidentally had the side effect of removing
the topic text box from the area being considered, resulting in the
escape key no longer working to end the message edit from within that
text box.
This is a simple and small commit which will alphabetically order the
entries of the fixtures dropdown menu in the "integrations developer
panel" devtool.
Apparently, the Zulip notifications (and resulting emails) were
correct, but the download links inside the Zulip UI were incorrectly
not including S3 prefix on the URL, making them not work.
While we're at this, we rewrite the somewhat convoluted previous
system for formatting the data export output.
Bootstrap v2.2.0^2~40^2~6 changes this default to false, so this is a
prerequisite to upgrading Bootstrap, and it’s also safer.
This closes an HTML injection path via user full names in the emoji
reaction tooltip. It doesn’t appear to be exploitable for cross-site
scripting because we disallow `>` in full names, and the code happens
to be written such that the next `>` is in a different parser
invocation.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
In a gigantic realm where we send several MB of `page_params`, it’s
slightly better to have the rest of the `<body>` available to the
browser earlier, so it can show the “Loading…” spinner and start
fetching subresources.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Precompiling regexes gives a performance increase of around 10-15%
based on tests. See https://jsperf.com/typeahead-regex. This stacks
up when we have a lot of users in an organisation.
This changes the availability icon for bot users to user_circle_green;
previously it was accidentally defaulting to user_circle_empty, making
it appear that bots were never available.
Fixes#13149.
As it turns out, our rerender_the_whole_thing function (used whenever
we were adding messages and discovered that the resulting message list
would be out-of-order) was just broken and scrolled the browser to a
random location.
This caused two user-facing bugs:
* On very fast networks, if two users sent messages at very close to
the same time, we could end up with out-of-order message deliveries,
triggering this code path, which was intended to silently correct
the situation, but failed.
* In some narrows to streams with muted topics in the history but some
recent traffic, the user's browser-cached history might have some
gaps that mean the server fetch we do after narrowing discovers the
history is out-of-order, again triggering the
rerender_the_whole_thing code path.
The fix is to just remove that function, adding a new option to the
well-tested rerender_preserving_scrolltop (which has explicit logic to
preserve the scroll position) instead.
Fixes#12067. Likely also fixes#12498.
This sidesteps tricky escaping issues, and will make it easier to
build a strict Content-Security-Policy.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This sidesteps tricky escaping issues, and will make it easier to
build a strict Content-Security-Policy.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Steve Howell
Anders Kaseorg
Tim Abbott
Mohit Gupta
Mateusz Mandera
Nat1405
Gaurav Thapar
Rohitt Vashishtha
joaomcarvalho
Jack Tiggleman
Vinit Singh
Dinesh
Sophie
Matheus Melo
Thomas Ip
Greg Price
Vishnu KS
Pragati Agrawal
YashRE42
bartek
Hemanth V. Alluri
clarammdantas