Previously, set_muted_topics was calling update_unread_counts once for each
topic in the input; this results in poor performance when there is a large
number of muted topics.
Fixes: #3605
We have added people.pm_with_url(message), which computes a
PM url from a private message using user ids rather than emails.
We call this in add_message_metadata(), since the slugs will
be valid even if emails change, so we don't need to compute
them on the fly during message rendering.
Currently, searching for group private messages requires typing each
person's email individually. This change improves the typeahead
suggestions for group `pm-with` searches by suggesting additional people
whenever a comma is entered.
Fixes: #3575
The function people.update_email() is not yet connected
to anything, but it sets the stage for upcoming changes.
When emails get updated, fundamentally we just update
the appropriate person object and add a new key to
people_dict. We sort of get a shim for free--old email
lookups will continue to work--but we add blueslip warnings
for stale lookups.
This helps make the Zulip development environment somewhat more robust
to new contributors, since it will give them a nice warning if they
try running any of our development tools outside the Zulip virtualenv.
Fixes#3468.
When filtering streams, we were incorrectly treating the regexp input
provided by the user as a regular expression, meaning that terms like
`c++` would trigger errors because they are invalid regular expression
syntax. We fix this by replacing RegExp with a simple IndexOf check.
Node test added by tabbott.
Fixes#3559.
This changes the markdown preview test to use the
waitForSelectorTextChange method rather than waitWhileVisible
because it was not being generated fast enough in some cases
to show correctly with waitWhileVisible.
We now trigger an event in user_events.js, and we dynamically
build the list of names in pm_list.js by calling out to
people.get_recipients().
We have a few variations of functions that build lists of names
for huddles, which should be cleaned up eventually. They are
called at different times in the code path, so the different
functions, while doing mostly the same thing, start with different
data sources.
Note that this only works for people who are currently logged in.
Folks that log in after you may pick up the old full name from
the message. (I'll address this in a separate commit.)
Replaces the hardcoded list of emoji_names and unicode_emoji_names in
static/js/emoji.js with a list generated from emoji_map.json, both to get
the list out of version control and so we can start modifying it for our
autocomplete. This does not change the contents of emoji_names. It sorts and
removes duplicates from unicode_emoji_names (causes no change in behavior,
since unicode_emoji_names is only used as if it were a set).
We now allow spaces and other special characters to be part
of the token (following "#", "@", or ":") that the typeahead
code will further evaluate as a typeahead candidate.
This is important for folks with short/common first names
on larger realms.
I moved the UI element for "Copy from Stream" to be above
the list of users, including the filter box and check/uncheck
links, which no longer get applied to the list of streams.
The reason I no longer apply the filter to streams is...
* It's kind of confusing to have filters apply to both
streams and users. There should be separate filters for
them, and I will try to resuscitate that feature later.
* The code to filter the streams was doing a sketchy
regex operation against user-inputted data. (`match()`)
* We want to use the same stream filtering code as the
right sidebar uses.
* It improves performance for the common case that you
are filtering users.
The reason I no longer apply the check-all/uncheck-all actions
to streams is that it would be crazy to select all your streams
to copy users from, and it would be expensive/slow for large
realms, and it would likely be done by accident if somebody was
trying to manage individual users.
Finally, the check-all/uncheck-all actions have been scoped
to the users filtered by the text box, so I moved the links
under the text box to make that hopefully more clear to users.
The function people.filter_by_search_terms() used
to return a JS object with emails as keys to represent
a set of users. Now we return a Zulip Dict() object
with user_ids as keys.
One of my commits from yesterday erroneously set the
"mentioned" flag on messages that weren't mentioning
the current user, so you would get the pink/salmon
background when you sent at-mentions to other people.
Now we check the user_id before setting the flag.
If we get a realm_user update for a user that is **not**
changing their full name, we no longer call
admin.update_user_full_name().
This was probably a fairly minor bug.
Earlier commits removed all uses of page_params.email outside
of people.js, and it turns out we have page_params.user_id, so
we don't even need page_params.email for seeding the data.
The local echo code now marks up mention buttons with user ids
instead of email. Our code in message_list_view.js deals with
either the old style or the new style of markup now to determine
which mention buttons need to be highlighted.
As part of this commit we extract mention_button_refers_to_me().
After this change, if a user sends a message with at-mentions, the
local echo code will add the `mentioned` flag to 'message.flags`
as part of the callback to build the HTML, rather then doing it
hackily during a post-processing step.
The function echo.apply_markdown() actually applies markdown to
a message now, instead of simply computing markdown. Passing
in the outer `message` object will allow us to avoid some hacky
post-processing of messages after rendering, because we can
have our parser callbacks update message on the spot in a more
atomic fashion.
This commit doesn't change any behavior yet, but it starts us
down the road of deprecating page_params.email and allowing
people.js to control all access to the current user's email,
which will be important for email changes.
- Remove `underscore.js` from `static/third` and fetch it from `npm`.
- Upgrade `underscore.js` to 1.8.3.
- Bump up the `PROVISION_VERSION` to 4.2.
Part of #1709
- Remove `codepointat` from `static/third` and fetch it from `npm`.
- Upgrade `codepointat` to 0.2.0.
- Bump up the `PROVISION_VERSION` to 4.1.
Part of #1709.
In people.emails_strings_to_user_ids_string, we just warn
for bad emails going forward.
Users can enter bad emails into the search location bar,
for example, and that causes us to compute a browser hash,
which in turn uses this function.
(It's possible that we should adjust the search code not
to compute hashes for narrowing when the narrow doesn't
make sense, but that could be a non-trivial fix.)
The slugs for PM-with narrows now have user ids in them, so they
are more resilient to email changes, and they have less escaping
characters and are generally prettier.
Examples:
narrow/pm-with/3-cordelia
narrow/pm-with/3,5-group
The part of the URL that is actionable is the comma-delimited
list of one or more userids.
When we decode the slugs, we only use the part before the dash; the
stuff after the dash is just for humans. If we don't see a number
before the dash, we fall back to the old decoding (which should only
matter during a transition period where folks may have old links).
For group PMS, we always say "group" after the dash. For single PMs,
we use the person's email userid, since it's usually fairly concise
and not noisy for a URL. We may tinker with this later.
Basically, the heart of this change is these two new methods:
people.emails_to_slug
people.slug_to_emails
And then we unify the encode codepath as follows:
narrow.pm_with_uri ->
hashchange.operators_to_hash ->
hashchange.encode_operand ->
people.emails_to_slug
The decode path didn't really require much modication in this commit,
other than to have hashchange.decode_operand call people.slug_to_emails
for the pm-with case.
This new module abstracts the setting up of a test
server for tests to run, pulling existing code from
casper and paving the way for API tests in the future.
The fix works by having build_stream_sidebar_row()
automatically update its own unread count when we
build a sidebar row. Currently we rebuild sidebar
rows when we pin/unpin rows.
As an aside, we currently don't really need to rebuild
the sidebar row when we pin, since we're only moving
the DOM, not altering it. But this may change in the
future, so I decided to leave that code path in place.
We may decide to do things in the future like showing
pinned streams with bolder fonts or special icons or
whatever.
Fixes#2902
When filtering users in the new stream form, check all
and uncheck all will only effect those users who are filtered,
visible in the dom.
Includes a Casper test for the new condition.
Merged two tests since they both use the same test data. The file name
of `presence_list_performance.js` also causes confusion as it is no longer
use for performance testing.
This adds some configuration options to settings.py, namely
PASSWORD_MIN_LENGTH and PASSWORD_MIN_QUALITY, which control
when the frontend validator invalidates the password.
Closes#2628
Stream descriptions are now displayed along with the name. The
autocomplete results include streams with matches in the stream
descriptions. Added styling for description in compose.css.
Fixes#2398.
When somebody changes their name, we will now update
the buddy list right away. The old code was trying
to do this through a code path that was designed for
true presence updates, but it was also passing in an
empty array, instead of undefined, which caused it to
fail to invoke the intended part of the codepath to
redraw the buddy list.
Now we just call the new activity.redraw() function,
which does the right thing for the buddy list.
The group PM list was live-updating before this change,
and it continues to live-update as part of the new
activity.redraw() function.
This commit replaces the placeholder "clipboard" button with a reaction button.
This is done on any message that can't be edited. Also, on messages sent by
the user the actions popover (toggled by the down chevron icon) contains
an option to add a reaction.
When clicked, a popover with a search bar and a list of emojis is displayed.
If the right sidebar is collapsed (the viewport is small), the popover is placed
to the left of the button.
Focus is set to the search bar. Typing in the search bar filters emojis.
Emojis with which the user has reacted to this message are highlighted.
Clicking them sends an API request to remove that reaction.
Clicking on non-highlighted emojis sends an API request to add a reaction.
When the popover loses focus it is closed.
The frontend listens for reaction events. When an add-reaction event is
received, the emoji is displayed at the bottom of the message with a
count initialized to 1. If there was an existing reaction to the message with
the same emoji, the count is incremented.
Old messages fetched from the server contain reactions.
They are displayed (along with title and count) at the bottom
of each message.
When clicking the emoji reaction at the bottom of the message, if the
user has already reacted with that emoji to this message, the reaction
is removed and the count is decremented. Otherwise, a reaction is added
and the count is incremented.
Hovering over the emoji reaction at the bottom of the message displays
a list of users who have reacted with this emoji along with the
emoji name.
Hovering over the emoji reactions at the bottom of the message displays
a button to add a reaction.
Fixes#541.
An exception in the webapp was trown when an empty mention was sent.
Examples of problematic messages are "@" or "@****".
In order to fix this, the regex that identifies mentions has been
modified, so it now requires the mention to have a "content" (by
replacing the ? quantifier by +).
A test case has been added to `frontend_tests/node_tests/echo.js` to
check that this works properly in the future.
* Doesn't pop up the warning until you actually try to send the message
* Eliminates the red warning.
* Changes confirm text to "Yes, send".
* Adds a stream size threshhold of 15 people; smaller streams don't
prompt about this.
Fixes#2257.
In the new stream creation modal, added checkboxes for each stream
and a toggle to see or hide the checkboxes. Altered filtering to
filter streams and users. Added corresponding casper tests.
When a stream is checked/unchecked, it does not affect the state
of any user checkbox. This may be visually unclear as users can be
added even if their checkboxes are empty.
Fixes#2448
This commit changes people.remove() to be people.deactivate(),
and it fixes a bug where deactivating users was causing tracebacks
in the PM list if somebody had PM'ed the deactivated user
recently.
We need this for node tests, so that you don't have to explicitly
remove every user between tests. (Also, people.remove() is about
to have different semantics.)
There is a change in Django 1.10 due to which whenever the password
of the user is changed the session hash changes. This change affects
us because we cache user profile objects and these cached objects need
to be refreshed. However, the signal sent by Django in which objects are
refreshed fails to refresh the cache for Tornado because it uses a
different cache prefix.
Note: Backend tests are not affected because they don't rely on Tornado.
Updated `get_editable()` so that organization admins only see their
own bots in their personal settings page; this removes a lot of
unnecessary clutter.
Fixes#2657.
This change introduces an unread_topic_counter object
that manages unread counts for streams and topics. Consolidating
all the logic into a single class will set us up to add
logic for dealing with topic counts that includes provisional
counts of unread messages from the server. It also makes
the current code a little easier to reason about.
Most of this change was simply extracting functions, but
I also removed a few unnecessary and inconsistent calls to
`stream_data.canonicalized_name` that preceded our use of
Dict with a fold_case argument.
This is a major change to the /#subscriptions page, converting it to
by a side-by-side list of streams and their settings in an overlay.
There are no new features added/removed, but it's a huge changeset,
because it replaces the old navigation logic and moves the stream
creation modal to appear in the right side of this overlay.
Added new option to download .zuliprc file directly from settings
page. This should help reduce friction when setting up new
bots/integrations. This new feature is available in the bot cards and
the 'show your API key' section. One caveat is that the filename is
automatically set to 'zuliprc' instead of '.zuliprc', since as most
browsers do not allow filenames to start with a dot.
Fixes#2327.
This commit adds rules for GCI, turns on rules that do not error
on our codebase, and changes frontend_tests/.eslintrc.json's no-sync
rule to off (as per AirBnB's style guide).
Rules for GCI:
no-restricted-syntax, no-nested-ternary, spaced-comment,
space-infix-ops, newline-per-chained-call, padded-blocks,
no-whitespace-before-property, space-in-parens
Rules that do not error:
no-useless-constructor, no-dupe-class-members, no-duplicate-imports,
no-iterator, no-undef, dot-notation, no-case-declarations, no-unneeded-ternary,
eol-last,
Finally, eqeqeq is changed from 2 to ['error', 'allow-null'], going
from jslint defaults to airbnb's recommendation (there were no errors)
* In most cases, eslint --fix with the right comma-dangle settings was
able to update the code correctly.
* The exceptions were cases where the parser incorrectly treated the
arguments to functions as lists/objects and added commas; these are
detectable with linters, and we fixed manually. Since this is test
code, we can be reasonably confident that just fixing the failures
suffices to correct any bugs introduced by making changes
automatically.
* In most cases, eslint --fix with the right comma-dangle settings was
able to update the code correctly.
* The exceptions were cases where the parser incorrectly treated the
arguments to functions like `assert_equal` as arguments; we fixed
these manually. Since this is test code, we can be reasonably
confident that just fixing the failures suffices to correct any bugs
introduced by making changes automatically.
Change `docs/testing.rst` to `docs/testing-with-casper.md` in
'Tips for debugging' because there is no `testing.rst` file, and
remote debugging description is placed in `testing-with-casper.md`.
Modify backend test of create_streams_if_needed so that the newly
created streams have descriptions.
Modify casperjs test of filling out stream_creation_form so that
the newly created stream has a description.
Fixes: #2428.
We now sort lists of users ids deterministically, and we also
sort list of emails deterministically and without regard to case.
This probably fixes the bug #2343, although I never got a great
repro on that.
This commit adds a basic eslintrc that emulates jslint defaults.
Rules that conflict with our existing code have been switched to
warnings instead of errors. Globals have been added to the eslintrc. The
bundled js file (generated by webpack) and blueslip.js are ignored with
.eslintignore.
To display warnings, run npm run lint-loud. This runs eslint without the
--quiet option on static/js and frontend_tests.
npm run --silent lint is run by tools/lint-all (in addition to jslint).
The --silent option is used to suppress the default output from npm run.
Fixes#535.
When we filtered buddy lists, a recent change introduced some
bugs related to case-insensitive emails. We now circumvent the
bug by indexing presence_info with user_ids.
We now use comma-delimited lists of user_ids for the following
data structures in unread.js:
- unread_privates[<user_ids_string>]
- get_counts.pm_count[<user_ids_string>]
Previously, this would incorrectly include a user with name and email
"" in the recipients list shown in the local echo code path.
We fix this and add a test for the issue.
We used to have hacky code where various functions would call
build_stream_sidebar_row() to get a jQuery object, and then they
would attach the jQuery object to the "sub" object from stream_data.js.
Now build_stream_sidebar_row() localizes the hack of attaching
a UI object to the "sub" object to just one function (and we can
clean this up in a follow-up commit).
Also, the UI object is now a JS object that can close on some useful
state information like the stream name and encapsulate how we
toggle the inactive_stream class.
Finally, we don't have build_stream_sidebar_row() needlessly append
list items to $('#stream_filters') when we know that our callers are
going to re-build the list anyway.
It used to be the case that you would get new messages for a
huddle, but the huddle wouldn't show up on your buddy list until
the every-50-seconds mass update of the buddy list.
Now we make sure to work with non-stale jQuery objects, and,
more importantly, we resize ourselves if we add new huddles.
(The resize issue arises due to some complicated heuristics
where we don't want group PMs to take up too much of the buddy
list for users who don't have many in their history.)
* Fixes handling of multiple stream links and invalid stream names.
* Fixes text regex so it handle hash sign the right way.
* Adds tests for these stream link cases.
The widget that gets built in topic_list.build_widget()
now knows how to add itself to its parent element and expose
an interface to retrieve the parent.
This moves one method over from stream_list.js. There's still a
lot of boilerplate here, unfortunately, as topic lists have a lot
of dependencies on other parts of the system--narrowing state,
muting state, jQuery, handlebars, etc.
We now use stream_id as our key to rename streams, which
should prevent a few race conditions long term. (We are
still possibly contending with other events that use
stream_name as a key, so this is not perfect.)
Now we just have two methods of importance:
compile_template
render_template
The compile_template() function will automatically
(and recursively) compile any partials it depends
on and mark those as compiled.
If I try to send a message to an unknown user (which is possible
for some types of realms), then I simply ignore them during the
send codepath, so that I don't later need to patch up their attributes.
We no longer store pm_recipient_count on person objects, but we
instead use a Dict to store them. Then the new API is this:
people.get_recipient_count()
people.incr_recipient_count()
Adds a database migration, adds a new string_id argument to the management
realm creation command, and adds a short name field to the web realm
creation form when REALMS_HAVE_SUBDOMAINS is False.
When the render function is run now, it uses the partial_finder
function to search recursively through files for partials and add them
so that test writers don’t have to.
This means that we no longer have to do any manual work to maintain
the templates.js check that all handlebars templates are rendered by
the node tests.
Previously, var/casper/server.log was overwritten before every run of
run-casper. This commit implements the simplest form of log rotation,
by overwriting server.log only if it has more than 100kb.
Previously we showed an "Edit" item in the actions popover menu when a user
could edit the content or topic of a message, and nothing otherwise. We now
show "Edit", "Edit Topic", or "View Source" in the popover menu for every
message, depending on the editability of the message, and present an
appropriate version of message_edit_form when the menu item is clicked.
Finishes #1604 and #1761.
This replaces add_stream_to_sidebar(), which was kind of a misleading
name, and it also adds a couple lines of code that were always
called right after calling add_stream_to_sidebar().
This function will make it easier to unit test upcoming
changes related to stream counts.
This was mostly moving code, but one change is that we
don't call create_subs() in subs.js any more (which would
have been kind of circular dependency), since the only thing
that it did besides calling a more appropriate function
in stream_data.js was to generate a trigger that was
subsequently ignored and possibly a UI trap, as we don't
want to be messing with the stream sidebar when we go into
the stream settings page.
We now simply call exports.create_sub_from_server_data() for
newly encountered unsubscribed streams (which don't belong in
the sidebar anyway.)
This function used to live in subs.js. It's mostly a code move,
but I simplified the logic to determine whether it's subscribed
not to do a lookup into the same data structure that the sub
already came from.
I also added some tests.
Previously, URLs were being incorrectly treated as unknown search
operators (since they had exactly one ":" in them, just like foo:bar
for an invalid choice of foo).
Fixes#1743.
Filter behaves similarly to filter in left sidebar, see PR #684. Added
stream input field to the stream creation modal along with other settings,
for clarity.
Fixes#455, #563.
Previously, we sent users to an "invite your friends" page after they
created an organization. This commit removes that step in the flow and sends
users directly to the home page. We also remove the now-unused
initial_invite_page.html template, initial_invite.js (which pre-filled the
invite emails with characters from literature), and the /invite URL route.
This fixes a problem where any absolute redirects in the routes
visited by the Casper tests will cause failures due to switching the
users to a "different" server where the cookies they'd received are no
longer valid.
Now, we at least consistently use the same hostname in the Casper
tests as EXTERNAL_HOST.
- Expand a box full of emojis into the
compose window for users to graphically select emojis.
- Append an emoji to the end of the message when a user
clicks the emoji in the emoji box.
- Trap the escape key to always close the emoji box
before closing anything else if the box is open.
- Fixes: #147.
(Most of this work was done by acrefoot in an earlier branch.
I took over the branch to fix casper tests that were broken during
the upgrade (which were fixed in a different commit). I also
made most of the changes to run-casper.)
This also upgrades phantomjs to 2.1.7.
The huge structural change here is that we no longer vendor casperjs
or download phantomjs with our own script. Instead, we just use
casperjs and phantomjs from npm, via package.json.
Another thing that we do now is run casperjs tests individually, so
that we don't get strange test flakes from test interactions. (Tests
can still influence each other in terms of changing data, since we
don't yet have code to clear the test database in between tests.)
A lot of this diff is just removing files and obsolete configurations.
The main new piece is in package.json, which causes npm to install the
new version.
Also, run-casper now runs files individually, as mentioned above.
We had vendored casperjs in the past. I didn't bring over any of our
changes. Some of the changes were performance-related (primarily
5fd58cf249), so the upgraded version may
be slower in some instances. (I didn't do much measurement of that,
since most of our slowness when running tests is about the setup
environment, not casper itself.) Any bug fixes that we may have
implemented in the past were either magically fixed by changes to
casper itself or by improvements we have made in the tests themselves
over the years.
Tim tested the Casper suite on his machine and running the full Casper
test suite is faster than it was before this change (1m30 vs. 1m50),
so we're at least not regressing overall performance.
These changes prepare us for the casperjs upgrade:
Extract init_viewport().
Have then_log_out() do more explicit waiting.
Add turn_off_press_enter_to_send().
This separates the display settings module from the
settings_table.handlebars template.
Additionally, it fixes the node tests to search in the
static/templates/settings directory and initialize any templates in there
while running tests on the settings templates.
Adds a new field org_type to Realm. Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level. Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.
Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.
Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.
The database defaults have not been changed, which allows our testing code
to work unchanged.
[includes some HTML/CSS work by Brock Whittaker to make it look nice]
Previously, the generate-fixtures shell script by called into Django
multiple times in order to check whether the database was in a
reasonable state. Since there's a lot of overhead to starting up
Django, this resulted in `test-backend` and `test-js-with-casper`
being quite slow to run a single small test (2.8s or so) even on my
very fast laptop.
We fix this is by moving the checks into a new Python library, so that
we can avoid paying the Django startup overhead 3 times unnecessarily.
The result saves about 1.2s (~40%) from the time required to run a
single backend test.
Fixes#1221.
I have a hunch that the messages sent from different files
are interfering with each other as well. This commit will make
it clear if indeed this is the case.
This adds support for running a Zulip production server with each
realm on its own unique subdomain, e.g. https://realm_name.example.com.
This patch includes a ton of important features:
* Configuring the Zulip sesion middleware to issue cookier correctly
for the subdomains case.
* Throwing an error if the user tries to visit an invalid subdomain.
* Runs a portion of the Casper tests with REALMS_HAVE_SUBDOMAINS
enabled to test the subdomain signup process.
* Updating our integrations documentation to refer to the current subdomain.
* Enforces that users can only login to the subdomain of their realm
(but does not restrict the API; that will be tightened in a future commit).
Note that toggling settings.REALMS_HAVE_SUBDOMAINS on a live server is
not supported without manual intervention (the main problem will be
adding "subdomain" values for all the existing realms).
[substantially modified by tabbott as part of merging]
This restructures the styling for the Zulip settings and
administration pages to minimize use of Bootstrap and use a consistent
styling library for similar elements.
While it is basically a wash in terms of the page's visuals, it will
make our life a lot easier for future work on improving the settings
pages section of the site.
In HTML, the line break immediately following a start tag is ignored
(see: https://www.w3.org/TR/html4/appendix/notes.html#h-B.3.1). An
extra span tag has been introduced in the upstream Pygments
HtmlFormatter in order to preserve the first new line. The Bugdown
Tests as well as our fenced_code.js frontend markdown processor have
been updated to reflect this new behavior.
Chaining together `wait*` functions can create race conditions in the
frontend tests. To avoid race condition, we need to insert `then`
function between two `wait*` functions.
Tim Abbott
Harshit Bansal
Elliott Jin
Steve Howell
sinwar
khantaalaman
brockwhittaker
Rishi Gupta
Rafid Aslam
JefftheBest1
polypmer
Sampriti Panda
Tommy Ip
Bojidar Marinov
Brock Whittaker
Akhil
Arpith Siromoney
Yago González
Sanskar Modi
Umair Khan
Brock Whittaker
Robert Hönig
lonerz
AZtheAsian
kevv87
Calvin Lee
Vladislav Manchev
paxapy
Igor Tokarev
umkay
sonali0901
hackerkid
Tomasz Kolek
reyha
trueskawka
Kartik Maji
Rishi Gupta
Ilona Brand
Sahil Dua