This reverts part of b0d632577f.
The problem was that multiple queries were combined as a single
search pill. And since we create the pills then narrow / search,
we added a comma seperator between them for the typeahead lookups
as required by the logic in `input_pill.js`.
This however introduced a new bug where the search suggestions
were incorrect as the typeahead lookup table wasn't updated, so
every time an item from the type ahead was selected it updated
the input string with an invalid operator.
Thus to resolve the first problem, we follow a simpler approach
by extracting all operators from the search string using our
`Filter.parse` logic and next add the pills, one by one.
This is the exact same bug as observed in
02ab48a61e.
The bug is in the way we invoke `Filter.parse`.
`Filter.parse` returns a list of operators which
can contain only one 'search' term at max.
All strings with the 'search' operator present
in the query are combined to form this 'search'
term.
However on concatenating two filters we may get
two terms containing the 'search' operator. This
will lead to the search suggestions getting
generated based on only the last 'search' operator
term instead of all the terms having the 'search'
operator.
This is evident from the test change as suggestions
should be based on "s stream:of" but instead they
were based on just the latest query.
If typeahead is used, this adds comma separated search queries
so that multiple search pills don't get combined as one and the
search behaviour remains same as search_pills_enabled = False case.
If typeahead is not used, this prevent the typing of a single comma
after the pill gets created.
This makes it so that search_suggestion.js
does not depend on activity.js.
That dependency hasn't really been "elegant"
for quite some time, but it will become particularly
unnecessary when we go to remove the "Group PMs"
section from the right sidebar.
This commit introduces a temporary wart
where we have these two functions with the
same name in a sort of unnecessarily
complicated code stack:
activity.process_loaded_messages
huddle_data.process_loaded_messages
But we will eliminate the former function
very soon, and our message-related codepaths
will just call the `huddle_data` version
directly.
TESTING NOTES:
Now that `huddle_data` is a tiny leaf
module, it's super easy to just use the
real implementation of what was formerly
called `activity.get_huddles()` (and is
now in `huddle_data`).
When I first wrote this commit, introducing
the real implementation of `get_huddles` exposed
some bugs that I fixed in the immediately
prior commits to this.
When the tests were originally written,
I believe `activity.js` had some annoying
`jQuery` dependencies that made it hard
to unit test against. We've slimmed it over
time to be mostly just a "controller" module.
But even in its current state it would have
been a bit of a bloated dependency.
The other friction for using the actual
version of `get_huddles` was setting up
the message data, but that's pretty minor.
If you have a group PM where some users have
three-digit user_ids and some with four-digit
user_ids (or similar), a huddle could effectively
be ignored when determining the order of
search search suggestions.
Basically, we need a way to canonically sort
user_ids in "huddle" strings, and it's somewhat
arbitrary whether you sort lexically or sort
numerically, but you do need to be consistent
about it.
And JS is not exactly helpful here:
> [99, 101].sort()
[ 101, 99 ]
This is a pretty obscure bug with pretty low
user-facing consequences, and it was never
reported to us as far as I know, but the fix
here is pretty straightforward.
We have had similar bugs of slightly more consequence
in the past. The reason this bug has shown
up multiple times in our codebase is that every
component that deals with huddles has slightly
different forces that determine how it wants
to serialize the huddle. It's just one of those
annoying things. Plus, bugs with group PMs
do tend to escape detection, since most people
spend most of their time either on streams
or in 1:1 PMs.
This creates a little bit of noise in some
tests where we don't care about users, but
it's worth avoiding confusion about which
users exist at which time. Also the noisy
aspects here may actually catch regressions.
Finally, if the noise gets annoying, we can
do things like rename "Ted" not to collide
with the "Test" stream.
Using "bob" as the current user was a bad
choice, as our convention is to use "me" or
"myself" or "alice" for the current user.
It also particularly complicated the tests
around Group PMs.
Now we have both "bob" and "myself", which
makes the intentions of the tests a little
more clear.
`stream_topic_history` is a more appropriate name as this
module will contain information about last message of a
stream in upcoming commits. Function and variable names
are changed accordingly like:
* topic_history() -> per_stream_history()
* get_recent_names() -> get_recent_topic_names()
* name -> topic_name
This extracts a new module with three
functions, which we will test with 100%
line coverage:
- show_email
- email_for_user_settings
- get_time_preferences
The first two break several dependencies
in the codebase on `settings_org.js`. The
`get_time_preferences` breaks an annoying
dependency on `page_params` within people.
The module is pretty cohesive, in terms that
all three functions are just light wrappers
around `page_params` and/or `settings_config`.
Now all the modules that want to call show_email()
only have to require `settings_data`, instead of
having a dependency on the much heavier
`settings_org.js` module.
I also make some of the unit tests here be more
full-stack, where instead of stubbing show_email,
I basically just toggle `page_params.is_admin`.
Explicitly stubbing i18n in 48 different files
is mostly busy work at this point, and it doesn't
provide much signal, since often it's invoked
only to satisfy transitive dependencies.
We now treat util like a leaf module and
use "require" to import it everywhere it's used.
An earlier version of this commit moved
util into our "shared" library, but we
decided to wait on that. Once we're ready
to do that, we should only need to do a
simple search/replace on various
require/zrequire statements plus a small
tweak to one of the custom linter checks.
It turns out we don't really need util.js
for our most immediate code-sharing goal,
which is to reuse our markdown code on
mobile. There's a little bit of cleanup
still remaining to break the dependency,
but it's minor.
The util module still calls the global
blueslip module in one place, but that
code is about to be removed in the next
few commits.
I am pretty confident that once we start
sharing things like the typeahead code
more aggressively, we'll start having
dependencies on util. The module is barely
more than 300 lines long, so we'll probably
just move the whole thing into shared
rather than break it apart. Also, we
can continue to nibble away at the
cruftier parts of the module.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.
It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.
This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.
With many tweaks by tabbott to documentation under /help, etc.
Fixes#13616.
This change makes these two functions more alike:
- get_search_result
- get_search_result_legacy
To test the UI modify zerver/views/home.py by
replacing `settings.SEARCH_PILLS_ENABLED` with
`True`. I only did a quick sanity check, since
any bugs with the new system are more likely due
to bitrot than any changes I have made here.
The history is this:
Tim cloned the code (before the smaller
helpers were extracted):
db4f6e278f
In 8b153f6452
Shubham removed get_operator_subset_suggestions but
accidentally left a `concat` statement in that got
misapplied to the previous suggestions:
- suggestions = get_operator_subset_suggestions(operators);
result = result.concat(suggestions);
The error there was carried over in some recent changes,
but this commit fixes that strangeness.
In 73e4f3b3fa
Shubham made this change, which makes sense only for
pills, and this code remains intact.
- if (operators.length > 0) {
- last = operators.slice(-1)[0];
+ if (query_operators.length > 0) {
+ last = query_operators.slice(-1)[0];
+ } else {
+ // If query_operators = [] then last will remain
+ // {operator: '', operand: '', negated: false}; from above.
+ // `last` has not yet been added to operators/query_operators.
+ // The code below adds last to operators/query_operators
+ operators.push(last);
+ query_operators.push(last);
}
Mohit made a couple changes to both old and new.
Anders made a couple non-substantive changes related to
the ES6 migration.
Steve (me) made several structural changes to the code. For
some of them I only changed the legacy code, not the pills
code. I didn't fix Shubham's mistake until this change.
Now the two functions should look similar except in the places
where they are intentionally different. I also added a comment
explaining the get_operator_subset_suggestions difference.
Fixes#13609
We only ever show 3 or 4 people in search suggestions
(possibly w/a couple variations, like pm-with/sender/etc.),
so we can try to search a smaller subset of people
before going through the entire realm.
We use message_store.user_ids() for this, since you
typically want to search messages for people that
have sent messages recently, and we already sort
based on PM conversations.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`. It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Add ability to search entire message history of all public streams at
once. It includes all subscibed, non subscribed public streams messages
and even historical public stream messages sent before user had joined
an organization or stream.
Fixes#8859.
Fix the .get_suggestions and .get_suggestions_legacy
to correctly handle search terms in group PM and treat
it as search term by not concatenating it at end of pm-with
email list operand.
After adding search pills, suggestions were based only on the
current input and no validation against the existing pills was done.
operator_subset_suggestions have been removed. Default suggestions
for base_operators have also been removed.
Handle multiple operators:
if `is:starred stream:Ver` was typed without selecting the typeahead
or pressing enter in between i.e search pill for is:starred has not yet
been added, then the description of `is:starred` will act as a prefix
in every suggestion.
Also makes changes re-enabling person suggestions for names with spaces.
This forks off search_legacy.js and search_suggestion_legacy.js so
that we can continue running automated tests against the legacy search
code while we develop the input pills feature.
Fixes#6515.
New suggestions for `sender:King ha` will respect spaces and the new
suggestion will be `Sent by King Hamlet <email>` instead of `Sent by King,
search for ha`. But if first term of sender operand is a valid user email,
tokens will be seperated by spaces. e.g `sender:hamlet@zulip.com abc`
will show `Sent by King Hamlet <email>, search for abc`.
Fixes#9492.
Default suggestion e.g `abc messages` as a suggestion for `is:abc`
is not shown in a new suggestion. But if the is operator is already
present before any other operator, the default message text will be
used. e.g `is:abc sender:abc@zulipchat.com` will have all the suggestions
with the prefix `abc messages, sent by abc@zulipchat.com`.
`get_containing_suggestions` was used to get the operand suggestions
for the `has` operator. `get_special_filter_suggestions` is now used
to get both the operand and operator suggestions for `has`.
Partially fixes#9461.
Negated suggestion for both operand and operators are handle in
get_special_filter_suggestions. A bug is get_operator_suggestions
causing the removal of `-` symbol from the operand was also fixed.
When suggesting operators to chose, category wise suggestions are
shown instead of a single default suggestion. e.g suggestions for
all the categories of has operator will be show instead of `Messages
with one or more` suggestion which did not make sense.
Fixes#9384.
Default suggestion e.g `messages with one or more abc` as a suggestion
for `has:abc` is not shown in a new suggestion. But if the has operator
is already present before any other operator, the default message text
will be used. e.g `has:abc sender:abc@zulipchat.com` will have all the
suggestions with the prefix `messages with one or more abc, sent by
abc@zulipchat.com`.
This run_test helper sets up a convention that allows
us to give really short tracebacks for errors, and
eventually we can have more control over running
individual tests. (The latter goal has some
complications, since we often intentionally leak
setup in tests.)
Zulip's search typeahead had a security bug, where when autocompleting
a specially crafted stream name, and then hitting space, code within
the stream name would be executed.
Zulip was doing HTML escaping correctly in the main code path using
Filter.describe to describe a narrow, but the escaping function was
not called in a few parallel code paths. We fix this in a way that
should protect all of these code paths, by making Filter.describe
return properly escaped HTML, rather than depending on its callers to
do so.
Thanks to w2w for reporting this issue.
Also adds a custom rule to eslint. Since the recommended way of extending
eslint is to create plugins as standalone npm packages, the separate rule
is published as 'eslint-plugins-empty-returns'.
Fixes#8669.
This allows user to view all group private conversation messages
with a specific user. That is, it views all the the group private
messages from groups which include the given user.
Add search suggestion for group-pm-with. Add operator name
and description in "Search operators" tab.
Add change in tab name to "Group Messages" when using this operator.
Add frontend_tests for group-pm-with search operator.
Fixes: #3882.
This commit introduces a per-stream topic_history class
inside of topic_data.js to better encapsulate how we store topic
history.
To the callers, nothing changes here. (Some of our non-black-box
node tests change their way of setting up data, though, since the
internal data structures are different.)
The new class has the following improvements:
* We use message_id instead of timestamp as our sorting key.
(We could have done this in a prep commit, but it wouldn't
have made the diff much cleaner here.)
* We use a dictionary instead of a sorted list to store the
data, so that writes are O(1) instead of O(NlogN). Reads
now do sorts, so they're O(NlogN) instead of O(N), but reads
are fairly infrequent. (The main goal here isn't actually
performance, but instead it just simplifies the
implementation.)
* We isolate `topic_history` from the format of the messages.
This prepares us for upcoming changes where updates to the
data structure may come from topic history queries as well
as messages.
* We split out the message-add path from the message-remove
path. This prepares us to eventually get rid of the "count"
mechanism that is kind of fragile and which has to be
bypassed for historical topics.
This new module tracks the recent topic names for any given
stream.
The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.
We introduced a minor one-line function called stream_has_topics.
We now have all of our callers into recent_topics code just
receive a list of topic names from get_recent_topic_names().
This is more encapsulated than handing off tiny little
structures to the three callers, two of whom immediately
mapped the objects to names, and one of whom needlessly
used the now defunct name canon_subject field.
The consolidation here removes some "subject" references, and
now all lookup are by stream id, not stream name.
The diff here is a bit daunting, but it's mostly simplification
of tests and calling code. Two of the callers now need to look
up stream ids, but they are otherwise streamlined.
The main change here is to stream_data.js, and we replace the
`canon_subject` and `subject` fields with `name`.
Ryan Rehman
clarammdantas
Steve Howell
Hashir Sarwar
Steve Howell
Anders Kaseorg
Mohit Gupta
Pragati Agrawal
Shubham Padia
Tim Abbott
Rohitt Vashishtha
Cory Lynch
Abhijeet Kaur
Steve Howell