Using startsWith is faster than indexOf, especially for long strings
and short prefixes. It's also a lot more readable. The only reason
we weren't using it was when a lot of the code was originally written,
it wasn't available.
This change makes these two functions more alike:
- get_search_result
- get_search_result_legacy
To test the UI modify zerver/views/home.py by
replacing `settings.SEARCH_PILLS_ENABLED` with
`True`. I only did a quick sanity check, since
any bugs with the new system are more likely due
to bitrot than any changes I have made here.
The history is this:
Tim cloned the code (before the smaller
helpers were extracted):
db4f6e278f
In 8b153f6452
Shubham removed get_operator_subset_suggestions but
accidentally left a `concat` statement in that got
misapplied to the previous suggestions:
- suggestions = get_operator_subset_suggestions(operators);
result = result.concat(suggestions);
The error there was carried over in some recent changes,
but this commit fixes that strangeness.
In 73e4f3b3fa
Shubham made this change, which makes sense only for
pills, and this code remains intact.
- if (operators.length > 0) {
- last = operators.slice(-1)[0];
+ if (query_operators.length > 0) {
+ last = query_operators.slice(-1)[0];
+ } else {
+ // If query_operators = [] then last will remain
+ // {operator: '', operand: '', negated: false}; from above.
+ // `last` has not yet been added to operators/query_operators.
+ // The code below adds last to operators/query_operators
+ operators.push(last);
+ query_operators.push(last);
}
Mohit made a couple changes to both old and new.
Anders made a couple non-substantive changes related to
the ES6 migration.
Steve (me) made several structural changes to the code. For
some of them I only changed the legacy code, not the pills
code. I didn't fix Shubham's mistake until this change.
Now the two functions should look similar except in the places
where they are intentionally different. I also added a comment
explaining the get_operator_subset_suggestions difference.
Fixes#13609
This should avoid some memory allocations.
We also use build_person_matcher to avoid
repeating the same logic over and over
again to process the query into termlets.
We also remove people.get_all_persons() and
people.person_matches_query().
Once we have max_items results, stop trying
to get more items.
This should really help large realms when
you do a search on streams that turns up
more than N streams (where N is about 12).
We won't even bother to find people.
This class gives us more control over attaching
suggestions to our eventual result. The main
thing we do now is remove duplicates as they're
encountered.
This will make sense in the follow up commit,
where we can short circuit actions as soon as
we get enough results.
This has a few benefits:
- we remove some duplicate code
- we can see finalize_results in profiles
It turns out finalize_results is expensive
for some searches. If the search itself doesn't
do a ton of work but returns a lot of results,
we see it in finalize_results. It brings to
attention that we should be truncating items
earlier instead of doing lots of unnecessary
work.
This isn't a huge speedup, but it's an easy
code change.
We remove the two-liner highlight_with_escaping,
which was only called in one place, and when
we inline it into the caller, we can pull the
first line, which builds the regex, out of the
loop.
When we're highlighting all the people that show
up in a search from the search bar, we need
to fairly expensively build a regex from the
query:
query = query.toLowerCase();
query = query.replace(/[\-\[\]{}()*+?.,\\\^$|#\s]/g, '\\$&');
const regex = new RegExp('(^' + query + ')', 'ig');
Even though the final regex is presumably cached, we
still needed to do that `query.replace` for every person.
Even for relatively small numbers of persons, this would
show up in profiles as expensive.
Now we just build the query once by using a pattern
where you call a function outside the loop to build
an inner function that's used in the loop that closes
on the `query` above. The diff probably shows this
better than I explained it here.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`. It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Because of the separate declarations, ESLint would convert them to
`let` and then trigger the `prefer-const` error.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
With webpack, variables declared in each file are already file-local
(Global variables need to be explicitly exported), so these IIFEs are
no longer needed.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Add ability to search entire message history of all public streams at
once. It includes all subscibed, non subscribed public streams messages
and even historical public stream messages sent before user had joined
an organization or stream.
Fixes#8859.
Fix the .get_suggestions and .get_suggestions_legacy
to correctly handle search terms in group PM and treat
it as search term by not concatenating it at end of pm-with
email list operand.
After adding search pills, suggestions were based only on the
current input and no validation against the existing pills was done.
operator_subset_suggestions have been removed. Default suggestions
for base_operators have also been removed.
Handle multiple operators:
if `is:starred stream:Ver` was typed without selecting the typeahead
or pressing enter in between i.e search pill for is:starred has not yet
been added, then the description of `is:starred` will act as a prefix
in every suggestion.
Also makes changes re-enabling person suggestions for names with spaces.
This large function will need to be modified significantly as part of
the pills effort, and copying it lets us preserve behavior in
production until we're ready to cut things over.
This commit prepares the frontend code to be consumed by webpack.
It is a hack: In theory, modules should be declaring and importing the
modules they depend on and the globals they expose directly.
However, that requires significant per-module work, which we don't
really want to block moving our toolchain to webpack on.
So we expose the modules by setting window.varName = varName; as
needed in the js files.
`format_as_suggestion` formats a list of operators into a
suggestion using the Filter.describe and Filter.unparse methods.
This change aims to increase readability.
Fixes#6515.
New suggestions for `sender:King ha` will respect spaces and the new
suggestion will be `Sent by King Hamlet <email>` instead of `Sent by King,
search for ha`. But if first term of sender operand is a valid user email,
tokens will be seperated by spaces. e.g `sender:hamlet@zulip.com abc`
will show `Sent by King Hamlet <email>, search for abc`.
Fixes#9492.
Default suggestion e.g `abc messages` as a suggestion for `is:abc`
is not shown in a new suggestion. But if the is operator is already
present before any other operator, the default message text will be
used. e.g `is:abc sender:abc@zulipchat.com` will have all the suggestions
with the prefix `abc messages, sent by abc@zulipchat.com`.
`get_containing_suggestions` was used to get the operand suggestions
for the `has` operator. `get_special_filter_suggestions` is now used
to get both the operand and operator suggestions for `has`.
Partially fixes#9461.
Negated suggestion for both operand and operators are handle in
get_special_filter_suggestions. A bug is get_operator_suggestions
causing the removal of `-` symbol from the operand was also fixed.
When suggesting operators to chose, category wise suggestions are
shown instead of a single default suggestion. e.g suggestions for
all the categories of has operator will be show instead of `Messages
with one or more` suggestion which did not make sense.
Fixes#9384.
Default suggestion e.g `messages with one or more abc` as a suggestion
for `has:abc` is not shown in a new suggestion. But if the has operator
is already present before any other operator, the default message text
will be used. e.g `has:abc sender:abc@zulipchat.com` will have all the
suggestions with the prefix `messages with one or more abc, sent by
abc@zulipchat.com`.
Zulip's search typeahead had a security bug, where when autocompleting
a specially crafted stream name, and then hitting space, code within
the stream name would be executed.
Zulip was doing HTML escaping correctly in the main code path using
Filter.describe to describe a narrow, but the escaping function was
not called in a few parallel code paths. We fix this in a way that
should protect all of these code paths, by making Filter.describe
return properly escaped HTML, rather than depending on its callers to
do so.
Thanks to w2w for reporting this issue.
We made this change because users often unnecessarily click "Home"
first in their use of Zulip, because it seems appealing. While "All
messages" isn't quite precise (it doesn't include muted streams), it
does describe relatively simply the interleaved view that this
represents.
This commit leaves everything as "home" in the code, and only changes
user-visible strings and docs. Changing the code will be a big project;
there are hundreds of relevant occurrences in variable names, etc.
Further, we'll probably want to convert those various variable names
in different ways.
Tweaked by tabbott to extend the commit message and update a few comments.
This allows user to view all group private conversation messages
with a specific user. That is, it views all the the group private
messages from groups which include the given user.
Add search suggestion for group-pm-with. Add operator name
and description in "Search operators" tab.
Add change in tab name to "Group Messages" when using this operator.
Add frontend_tests for group-pm-with search operator.
Fixes: #3882.
This new module tracks the recent topic names for any given
stream.
The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.
We introduced a minor one-line function called stream_has_topics.
We now have all of our callers into recent_topics code just
receive a list of topic names from get_recent_topic_names().
This is more encapsulated than handing off tiny little
structures to the three callers, two of whom immediately
mapped the objects to names, and one of whom needlessly
used the now defunct name canon_subject field.
The consolidation here removes some "subject" references, and
now all lookup are by stream id, not stream name.
The diff here is a bit daunting, but it's mostly simplification
of tests and calling code. Two of the callers now need to look
up stream ids, but they are otherwise streamlined.
The main change here is to stream_data.js, and we replace the
`canon_subject` and `subject` fields with `name`.
I deleted a test case that involved a highlighted stream, but
the query was empty. This produces kind of a weird result with
typeahead_helper.highlight_with_escaping, but this function already
has coverage in node_tests/typeahead_helper.js, so the check here
was essentially redundant anyway. Specifically, the highlighter
wraps every character individually with <strong>, and looks really
messy in html.
Fixes#5612. What this specifically does is that if you are
typing a group PM, this logic iterates through the possible
search suggestions for the next autocomplete. If that suggestion
contains a group PM that already exists, then prioritize it with
the most recent one on top.
Instead of having a custom (duplicate) matching function in
search suggestion, it was refactored to use the function in
people.js. This also gets the diacritic-ignoring feature
of the function in people.js.
Fixes#5315.
This involves updating filter.js, mostly. The
tests were updated appropriately for this change,
which also involved changing a caspar test for
narrowing.
This was never a feature in the old search_suggestions
version, so a new helper function for it was added.
Relevant tests were also added, maintaining 100% coverage.
The get_person_suggestions and get_group_suggestions functions
were updated to the new system. Support for negation is also
added in the new system.
Relevant tests were also updated. Also, note that the function
get_private_suggestions was removed, as it was rendered
obsolete by these updates.
Special filter was updated to work even when it is not the first
token in a search query. Furthermore, the default query was
moved around to work with the changes to come for the new
suggestion system.
A test also had to be modified to work with the new system.
Despite the length of this commit, it is a very straightforward
moving of code from narrow.js -> narrow_state.js, and then
everything else is just s/narrow.foo()/narrow_state.foo()/
(with a few tiny cleanups to remove some code duplication
in certain callers).
The only new functions are simple setter/getters that
encapsulate the current_filter variable:
narrow_state.reset_current_filter()
narrow_state.set_current_filter()
narrow_state.get_current_filter()
We removed narrow.predicate() as part of this, since it was dead
code.
Also, we removed the shim for narrow_state.set_compose_defaults(),
and since that was the last shim, we removed shim.js from the app.
Currently, searching for group private messages requires typing each
person's email individually. This change improves the typeahead
suggestions for group `pm-with` searches by suggesting additional people
whenever a comma is entered.
Fixes: #3575
Typing "stream:foo -topic:b" leads to "stream:foo -topic:bar" properly
as a suggestion now.
(imported from commit bb0acf52744f7b13977a3db5d3c130d1402b09b7)
The methods describe() and operator_to_prefix() are
now in filter.js. They seem like they eventually
may be useful outside the scope of search suggestions,
and they seem to go hand in hand with other Filter
responsibilities.
(imported from commit 55f14c3ee848febb2e7c8c6a27afd690569b4e08)
This is yet another change related to phasing out the
[operator, operand] tuple data structure for representing
terms in a narrow.
(imported from commit 508e58fc4eebae8a24a8ae59919ba5d94fc66850)
There was a bug where you would type "is:private je" into the search
suggestion and see undefined:jesstess@zulip.com. Now we use
the "pm-with" operator. The search suggestions for people are kind
of complicated now, because there is some overlap between
get_private_suggestions and get_person_suggestions.
(imported from commit 7d330f34f4a433995420de6eb90cb41229b70272)
Search suggestions were limiting you to the last 10 topics
in a stream, not the last 10 *matching* topics in a stream.
This was just a bug, where the slice() call was supposed to
come after the filter() call.
(imported from commit 7ad8f8409315d8ca389aa5bb085275a60f6e08c0)
I'm trying to move well-isolated methods out of narrow.js, so that
narrow.js is more strongly focused on UI/ajax interactions and
big, heavy lifting stuff. The logical home for parse/unparse
seemed to be Filter, and they brought along two private methods
with them. The big code moves involved trivial follow ups
like s/exports/Filter/.
(imported from commit ace0fe5aa1c7abce0334d079ba9eb8d9a57bd10f)
1) The class Filter now lives in its own module.
2) The function canonicalized_operators() is now a class method on Filter.
3) The function message_in_home moved to filter.js and became private.
4) Various calling code had to change, of course.
5) Splitting out Filter helped simplify a few tests.
(imported from commit e41d792b46d3d6a30d3bd03db0419f129d0a2a7b)
There are also one or two places we don't need to use it for security
purposes, but we do so for consistencey.
(imported from commit aa111f5a22a0e8597ec3cf8504adae66d5fb6768)
If you entered "stream:Denmark " in the search box, we would show
you two suggestions for "stream Denmark", despite our duplicate
detection, because we didn't canonicalize the suggestion that is
literally based off the user typed query, and so the other way
of generating the "stream Denmark" suggestion created a duplicate.
Now all the suggestions we generate are canonicalized, so the
generalized duplicate detection can work.
(imported from commit 52bf08ccf9bb2e2260ca8c20690169aead3732ab)
We now only suggest "Home" if you have an empty search box (suggestion #1),
or if you are typing "Home". Before we'd show it as a way to get back
from various narrows. This is trac #1596.
(imported from commit 9228bf45e9aaf9912d83efdf70338abb4b7e054f)
Moved 400+ lines out of search.js into search_suggestion.js. This
leaves search.js primarily responsible for lots of little DOM
interactions, whereas search_suggestion is more about data.
(imported from commit 53d08b29367c0172e483064f213538d45098279e)
Anders Kaseorg
Steve Howell
Tim Abbott
Mohit Gupta
Pragati Agrawal
Shubham Padia
Armaan Ahluwalia
Shubham Dhama
Rishi Gupta
Cory Lynch
Abhijeet Kaur
Steve Howell
Elliott Jin
Sampriti Panda
kevv87
Tim Abbott
Vishnu Ks
Steve Howell
Leo Franchi
Zev Benjamin
Steve Howell
Tim Abbott