This commit adds mypy annotations for both the main
bots and the bots testing runner. It involves a change
to the BotHandlerApi send_message and update_message
funtions, which is compatible with every bot.
Tweaked by tabbott to use more expressive annotations.
We enable data_suffix option when creating Coverage instances which
causes the output files to include the hostname, pid, and random id.
Before each run erase is called which clears all existing coverage data
files. And then at the end of the test run use the combine method which
merges the reports.
We collect coverage in the main process which collects data from
imports and also when running in single process mode. In the workers we
collect coverage in run_subsuite. This creates more stats files than
strictly required but I don't see a better place to save the stats when
stopping workers.
Note that this has the side effect of enabling parallel testing in
Travis CI.
Moves creating the emoji folder from the provisioning script to
the build_emoji script.
Fixes the fact that the emoji cache directory wasn't being created
when not using the provision.py script.
Due to differences between the codepoints of flag emojis in
`emoji_map.json` and iamcal's dataset, we need to patch the
css classes for the flag emojis temporarily until the migration
to iamcal's dataset is complete inorder to render them properly.
There is a difference between the images of flag emojis in our
old emoji farm and iamcal's spritesheets and since we have not
yet switched to using spritesheets for displaying emojis in
messages, there is a difference between the flag emojis as
rendered in messages and in emoji pickers.
This fixes a significant static asset pipeline bug, which mean that
when we added `moment.js` to the Zulip npm dependencies, it wasn't
properly included in common.js; caching prevented common.js from ever
being rebuilt.
Modify the spritesheet generation code to account for the differences
between emoji_map.json and iamcal's dataset. Due to the differences
between the two mappings, some emojis like 1️⃣, 2️⃣ etc were not
getting rendered properly in the two emoji pickers where we are using
the iamcal's spritesheets for rendering them. This was so because there
was no CSS class corresponding to the codepoints of these emojis(as
mapped using emoji_map.json) in our spritesheet CSS(generated using
iamcal's dataset).
Fixes: #4775.
In this commit we remove user_sidebar_actions.handlebars from
IGNORE_FILES as well remove the check for files to be in a
IGNORE list thus reaching 100% 4 space indent checking for
handlebar templates.
Fixes: #1661.
This doesn't completely fix settings responsiveness, but it's a big
step along the way. Outstanding issues include:
1. When switching tabs from settings to organization, it will launch
the first item which is more annoying in this view since it brings you
into that tab. Haven’t decided on an elegant solution to this yet.
2. Sidebar scrolling doesn’t work. I have to restructure how the top
section and bottom sections of content are displayed to fix this.
Likely by enforcing min-height of 100% - bottom height on the top piece.
3. Most of it is actually reasonably responsive but some isn’t, and
should be fixed on a case-by-case.
This commit changes the backend testing framework to run
in parallel mode which is same as --processes=4. If --coverage
is supplied, we enforce serial mode, --processes=1, because
coverage is not compatible with parallel mode at the moment.
The only thing being translated in any email was the title of the Zulip
header image in the html file, probably because not doing so breaks a linter
rule.
he name "Zulip" doesn't make sense to translate anyway.
This removes scaling from the emojis by changing the background size to
a lower value and then allowing for the widths and heights of the
emojis to be proportionally smaller.
The transform: scale property would cause many more repaints in Chrome
and other browsers than should have been necessary which would render
messages above and below the feed light grey boxes that would
momentarily flash as blank before filling with content.
Modified by tabbott to use a percentage in the background-size.
Fixes#4660.
Modified composebox_typeahead.js to recognize the triple backtick
and tilde for code blocks, and added appropriate typeahead functions
in that file and in typeahead_helper.js.
Additionally, a new file pygments_data.js contains a dictionary of
the supported languages, mapping to relative popularity
rankings. These rankings determine the order of sort of the
languages in the typeahead.
This JavaScript file is actually in static/generated/pygments_data.js, as it
is generated by a Python script, tools/build_pymgents_data.py. This is
so that if Pygments adds support for new languages, the JavaScript file
will be updated appropriately. This python script uses a set of popularity
rankings defined in lang.json.
Corresponding unit tests were also added.
Fixes#4111.
We've found a couple major issues that we need to fix:
* TRAVIS_COMMIT_RANGE being computed incorrectly in some cases (?!)
* The imperative linter could use some work.
This makes it possible for the Zulip mobile apps to use the normal web
authentication/Oauth flows, so that they can support GitHub, Google,
and other authentication methods we support on the backend, without
needing to write significant custom mobile-app-side code for each
authentication backend.
This PR only provides support for Google auth; a bit more refactoring
would be needed to support this for the GitHub/Social backends.
Modified by tabbott to use the mobile_auth_otp library to protect the
API key.
Despite the length of this commit, it is a very straightforward
moving of code from narrow.js -> narrow_state.js, and then
everything else is just s/narrow.foo()/narrow_state.foo()/
(with a few tiny cleanups to remove some code duplication
in certain callers).
The only new functions are simple setter/getters that
encapsulate the current_filter variable:
narrow_state.reset_current_filter()
narrow_state.set_current_filter()
narrow_state.get_current_filter()
We removed narrow.predicate() as part of this, since it was dead
code.
Also, we removed the shim for narrow_state.set_compose_defaults(),
and since that was the last shim, we removed shim.js from the app.
This code makes the right pane work in "Manage Streams" when
you are editing a stream subscription. It handles basic
functionality (submitting forms, etc.), live updates, and
showing the pane as needed.
Most of the code here was simply moved from subs.js, but some
functions were pulled out of larger functions:
live update:
add_me_to_member_list
update_stream_name
update_stream_description
collapse/show:
collapse
show_sub
We also now export subs.show_subs_pane.
We eventually want stream_edit not to call into subs.js, and
this should be fairly easy--we just need to move some shared
methods to a new module.
'$COMMIT' was originally printed in '$COMMIT_FILE_PATH' before all
respective files were downloaded, meaning that this step
wouldn't be repeated if one download failed. This commit prints
'$COMMIT' only after all downloads were successful.
This commit extracts the method compose_actions.on_narrow()
to handle changing the compose box (as appropriate) after
any narrowing action.
This change should be mostly non-user-facing, but it's not
exactly a trivial extraction.
For the case where the user already had content in their
compose box, we continue to leave the compose box alone,
but we now update compose fading 150+ lines later in
narrow.activate().
Likewise, for cases where we cancel composing, this will
also happen later in the function.
Finally, for PM narrows, where we auto-open the compose box, we
no longer call compose.cancel() before calling compose.start(),
because either a) the compose box would have not been open
in the first place or b) the start() function can handle
clearing the old fields.
Add code to download iamcal's sprite sheets and generate CSS files required
for displaying these sprite sheets using emoji's unicode codepoints rather
than their names.
Use `emoji.json` to create a emoji catalog and add it to
`emoji_code.js` file. This catalog contains the unicode
codepoints of all the emojis grouped according to their
category. Emojis are sorted according to the `sort_order`
defined in the iamcal's dataset.
This fixes the fact that our test suites would have trouble connecting
to the other parts of the Zulip service when run with a proxy
configuration (e.g. trying to send requests to localhost through the
proxy!).
Thanks for Vishnu Ks for his work on this.
Fixes#971.
Previously, if you have a branch with a new migration,
and you rebase onto master past someone else's migration,
you have to manually renumber your migration.
Add a script that automatically renumbers the migrations.
Fix#4257.
Some Handlebars strings contained whitespaces characters at their ends.
With this, such characters are removed, as well as multiple spaces
(like the ones produced by code indentation).
This also includes a couple of fixes that removes spaces that were
intentionally placed before/after the string to translate.
This moves respond_to_mention() and reply_with_mention() to
compose_actions.js. These methods are basically thin layers
on top of compose_actions.start().
This module extracts these two functions that get called by
several other modules:
start()
cancel()
It is a little bit arbitrary which functions got pulled over
with them, but it's generally functions that would have only
been called via start/cancel.
There are two goals for splitting out this code. The first
goal is simply to make `compose.js` have fewer responsibilities.
The second goal is to help break up circular dependencies.
The extraction of this module does more to clarify
dependencies than actually break them. The methods start()
and cancel() had actually been shimmed in an earlier commit,
and now they no longer have a shim.
Besides start/cancel, most of the functions here are only
exported to facilitate test stubbing. An exception is
decorate_stream_bar(), which is currently called from
ui_init.js. We probably should move the "blur" handler out
of there, but cleaning up ui_init.js is a project for another
day.
It may seem slightly odd that this commit doesn't pull over
finish() into this module, but finish() would bring in the
whole send-message codepath. You can think of it like this:
* compose_actions basically just populates the compose box
* compose.finish() makes the compose box do its real job,
which is to send a message
In backend tests, only call generate-fixtures when --generate-fixtures
is explicitly passed or is_template_database_current() returns False.
We don't need to flush cache for backend tests because we bounce the key
prefix used to create cache keys before running every test
This adds the option '--rerun' to the `test-backend` infrastructure.
It runs the tests that failed during the last 'test-backend' run. It
works by stailing failed test info at var/last_test_failure.json
Cleaned up by Umair Khan and Tim Abbott.
This (1) changes test_server to use the common `check_venv` method and
(2) improves check_venv to provide a clearer error message in the case
that you're inside Vagrant but not in a venv.
Tweaked by tabbott to borrow logic from run_dev.py.
This is mostly a straight port from bash to Python, but we
rename the coverage option to `--coverage` and we add checks
for being in a venv and being correctly provisioned.
Fixes#4009.
* Change the classes and ids of different widgets and modals
and make suitable changes in `admin.js`.
* Remove any other occurrences of `alias` or `realm_alias`
from admin.js.
This moves the implementations of error/report/message from
ui.js to ui_report.js. They had been shimmed before, so calling
modules still use the same names to call the functions, but we
no longer need the shims.
While it's sometimes nice to put a few selectors on the same line,
it is generally better to have a consistent way of formatting our
selectors, and most of our code up until now lists them vertically.
This change fixes the linter to enforce one selector per line, and
it cleans up the places in the CSS where we had multiple selectors
on the same line.
The advantages of one-per-line are as followers:
* cleaner diffs
* easier to see when multiple areas of the app may have the
same format
* less likely to go over 80 cols
* makes it more clear where we have deep nesting in the
individual selectors
* makes it easier for our linting tools to enforce
whitespace violations
This also fixed an old bug where we had ".landing_page h2, h4", which
sets "h4" styles outside of the landing page.
Adds a new webhook integration for Slack to receive messages
from one's Slack team's public channels.
Contains negative tests for broken, missing or invalid data.
Allows two different option for integration:
1. Receive notification on a single stream with different topics
for each of Slack's public channels.
2. Receive notification on different streams for each of Slack's
public channels.
Steps to choose between the two options is described in the documentation.
Fixes#3569.
This fixes 2 issues:
* Being added to an invite_only stream did not correctly update the
"streams" key of the initial state.
* Once that's resolved, subscribe_to_stream when called on a
nonexistant stream would both send a "create" event (from
create_stream_if_needed) and an "occupy" event (from
bulk_add_subscriptions).
The second event should just be suppressed in that case, and this
implements that suppression.
In this commit we modify our CSS parser not only to render the text from
a given CSS tokens produced but also enforce 4 space indentation on it.
Also we enforce some basic rules we would like our CSS to follow such as
* Always have "\n" in between the starting of body({) and body itself
and ending of the body and the closing of body(}).
* Use 4 space indents while having but something within the block
structure ( { .... } ).
* Have single space after ',' in between multiple selectors.
* Have only a single space in between selector and the starting of
block structure ({ ... }) if block structure starts on same line as
of selector.
eg. body {
body content here
}
Notice single space between 'body' and '{'.
Fixes: #1659.
The js-dep-visualizer tool now attempts to find a set of edges
to remove from a call graph that would reduce it to having only
trivial mutual dependencies, and it produces a roadmap of the
changes that need to happen.
If the tool can't reduce the graph all the way, it still produces
a DOT file that can be visualized.
This fix also has some significant code cleanup.
This adds a report of nodes, handles some errors better, adds
some helpful output, cleans up some abspath calls, and
updates which modules and/or dependencies we temporarily are
ignoring for the report.
Currently, in the case where `--modified` is not passed, the linter asserts
that it's checking at least 10 files. Removing this (somewhat arbitrary)
check makes it easier to:
- Add support for specifying files to check via command line arguments
- Reason about cases where `check-templates` is called from `lint-all`
Rather than having a bunch of regexes to look for, we just
have a single regex for a function call. And now we process
line by line, which allows us to more easily ignore comments.
This fixes an issue where if you saved a Python file (even just
changing whitespace) while casper tests were running, the Tornado
server being used would restart, triggering a confusing error like
this:
ReferenceError: Can't find variable: $
Traceback:
undefined:2
:4
Suite explicitly interrupted without any message given.
This tools lets us view circular dependencies in our JS
code. It does regex parsing, so it has a few false positives,
but it's an early draft of the tools. Steve Howell helped
with this commit.
Apparently Travis CI has a very strange issue today that causes our
Nagios/E2E tests to have Tornado failing to connect to RabbitMQ.
Causes unknown, but I've spent a day trying to debug this without
luck, and we need our test suites passing in the meantime.
In this commit we add the ability of recognizing comments in
handlebar and Jinja2 templates to the template parser. This
fixes issue of template parser picking up code tags which are
commented out in files.
Also we fix the problem of too much deep nesting in the Jinja2
logic statement. Now only nested Jinja2 logic statements will get
a deeper indentation level.With this another fix was introduced
relating with the tags before a nested handlebar or django tag getting
wrong indent.
We also fix the issue with wrong offsets with closing tags in
cases if they were not matching indent level of the starting
tag intially.
Also we also start to ignore any tags occuring in between 'pre'
tags for deeper indent levels. As well we start to filter out django
non block tags from getting deeper indent level.
Our linter for translation strings shouldn't check test files, since
then we'll end up translating non-user-facing strings.
So we fix that, and actually add the opposite lint rule.
Change applies to both subdomains and non-subdomains case, though we use
just the EXTERNAL_HOST in the non-subdomains case if there is only 1 realm.
Fixes#3903.
- Add new 'missedmessage_email_senders' queue for sending missed messages emails.
- Add the new worker to process 'missedmessage_email_senders' queue.
- Split aggregation missed messages and sending missed messages email
to separate queue workers.
- Adapt tests for sending missed emails to the new logic.
Fixes#2607
- Shell command `sort` depends on system locale and
symbol `_` (underscore) is located after letters
in the default locale of Travis instances. Otherwise,
python sorting uses its own symbols ordering and
underscore is located before letters. Changing the
locale for `sort` shell command with adding environment
variable doesn' t work on Travis instances. That's why
It was decided to apply the same sorting command to both
comparing lists.
This code was added as part of the Django 1.10 migration to make our
tests work with both Django 1.8 and 1.10. Now that we're on 1.10,
it's no longer required.
This is a common mistake (I'll add documentation on why in the next
commit), and this is a great hook for sending folks to the
documentation on our frontend build process.
Like many rare-case code with new tests, it turns out that the logic
for handling null characters in our Zephyr postgres query escaping
never worked, in multiple ways. First, it always changed the second
character in s, not the current one being inspected, and second, the
value it replaced it with was no the correct postgres escape of the
null byte. We fix this and add tests.
This completes the effort to get zerver/views/messages.py to 100%
test coverage.
Fixes#1006.
- Add stamp file creation for the failed templates compilation.
- Add error response to `home` route if stamp file exists. It appears
just for the development environment.
- Add jinja2 template for failed handlebars templates compilation error.
Fixes#3650.
Move zulip-emoji from its current location `images/emoji` to
`images/emoji/unicode` and add a symlink in `images/emoji`
to zulip.png in `images/emoji/unicode`.
This adds to Zulip support for a user changing their own email
address.
It's backed by a huge amount of work by Steve Howell on making email
changes actually work from a UI perspective.
Fixes#734.
In this commit we are modifying pretty print tool to support
Django and html singleton tags. For Addition of html singleton
tags template parser was modified to emit psudeo
html singleton end tags to accompany html singleton tags and
token class was updated to have line_span field.
In this commit we improve the way errors are handled in our
template parser and thus improving the displayed messages in
case of errors. Eg. Errors in case of unbalanced quotes now
makes more sense displaying line and column information
including line where error might be sourced.
* Now queue_workers.py sorts queue names and prints them on their own
line. Previously it's output was nondeterministic.
* Simplified grep strategy for removing the "test" worker.
This list was likely to end up out of date quickly, since it wasn't
documented that you need to update it when adding a queue. The best
solution is to just not require it to be updated.
This adds an assertion, when `test-backend` is run with `--coverage`,
that we have 100% test coverage on a list of files that we expect to.
There's a whitelist/blacklist, managed in tools/test-backend.
Fixes#3363.
In This commit we extend the work being done by @showell in PR#1778
to develop a tool to pretty print html and our handlebar templates
in order to enforce our style convention of 4 Space indentation in
templates.
This commit introduces following changes:
* Fix Py3 Compatibility.
* Add ability to prettify in cases when html tags are not the
starting of a line and addition of test cases for it.
* Add ability to lint handlebar tags and add test cases for it.
* Add {{else}} as special case of indent.
* Add test cases in general to testing new tool.
@showell Helped me throughout and reviewed this commit.
Fixes#1778
This fixes an issue where provision would fail if the user's home
directory was setup in such a way that the postgres user couldn't
access it (and thus the `sudo` command here would throw errors about
having a non-readable current working directory).
The original test was written in shell script which launches a new
django instance for every tests. By doing it in Python, we avoid
the overhead and reduce the test time to <1 second.
Fixes#3620.
Now all the bots that are stored in contrib_bots are in the
same file/directory format.
The format is specified here #3427. Add tests.py file for encrypt_bot as well.
Fixes#3427.
Breaks out the Hello World example to create a new
webhook-walkthrough.md. Includes minor edits so the two docs
read well. Adds the new page, "Webhook walkthrough", to the TOC.
Fixes#3498
This helps make the Zulip development environment somewhat more robust
to new contributors, since it will give them a nice warning if they
try running any of our development tools outside the Zulip virtualenv.
Fixes#3468.
This saves more than 50MB of unnecessary duplicate content in release
tarballs (the static assets that will actually be served have already
been built at this point by tools/minify-js).
Zulip's previous model for managing static asset files via Django
pipeline had some broken behavior around upgrades. In particular, it
was for some reason storing the information as to which static files
should be used in a memcached cache that was shared between different
deployments of Zulip. This means that during the upgrade process,
some clients might be served a version of the static assets that does
not correspond to the server they were connected to.
We've replaced that model with using ManifestStaticFilesStorage, which
instead allows each Zulip deployment directory to have its own
complete copy of the mapping of files to static assets, as it should
be.
We have to do a little bit of hackery with the staticfiles.json path
to make this work, basically because Django expects staticfiles.json
to be under STATIC_ROOT (aka the path nginx is serving to users), but
doing that doesn't really make sense for Zulip, since that directory
is shared between different deployments.
- Add pid file of development processes group, which allows to
manage development processes group with os utils. Also it allows to
kill subprocesses when parent process was closed incorrectly.
- Add tool 'stop_dev_server' to stop development server by pid file.
Fixes#1547
This is a fairly major overhaul of the CSS parser to support
line numbers in error messages.
Basically, instead of passing "slices" of tokens around, we pass
indexes into the token arrays to all of our sub-parsers, which
allows them to have access to previous tokens in certain cases.
This is particularly important for errors where stuff is missing
(vs. being wrong).
In testing this out I found a few more places to catch errors.
Tim Abbott
Rohitt Vashishtha
Steve Howell
Pweaver (Paul Weaver)
Umair Khan
Reid Barton
rht
derAnfaenger
Mehanig
Eeshan Garg
Eklavya Sharma
Jason Michalski
umkay
Jordan Gedney
Carey Metcalfe
Aditya Bansal
Harshit Bansal
Brock Whittaker
Tommy Ip
Rishi Gupta
Cory Lynch
Neeraj Wahi
K.Kanakhin
Joshua Pan
Ayush Jain
Raghav Jajodia
Yago González
Adarsh S
sinwar
Tomasz Kolek
Elliott Jin
wizsid11
Saumya Rawat
Abhijeet Kaur
Feorlen
hackerkid