In this commit we modify our CSS parser not only to render the text from
a given CSS tokens produced but also enforce 4 space indentation on it.
Also we enforce some basic rules we would like our CSS to follow such as
* Always have "\n" in between the starting of body({) and body itself
and ending of the body and the closing of body(}).
* Use 4 space indents while having but something within the block
structure ( { .... } ).
* Have single space after ',' in between multiple selectors.
* Have only a single space in between selector and the starting of
block structure ({ ... }) if block structure starts on same line as
of selector.
eg. body {
body content here
}
Notice single space between 'body' and '{'.
Fixes: #1659.
The js-dep-visualizer tool now attempts to find a set of edges
to remove from a call graph that would reduce it to having only
trivial mutual dependencies, and it produces a roadmap of the
changes that need to happen.
If the tool can't reduce the graph all the way, it still produces
a DOT file that can be visualized.
This fix also has some significant code cleanup.
This adds a report of nodes, handles some errors better, adds
some helpful output, cleans up some abspath calls, and
updates which modules and/or dependencies we temporarily are
ignoring for the report.
Currently, in the case where `--modified` is not passed, the linter asserts
that it's checking at least 10 files. Removing this (somewhat arbitrary)
check makes it easier to:
- Add support for specifying files to check via command line arguments
- Reason about cases where `check-templates` is called from `lint-all`
Rather than having a bunch of regexes to look for, we just
have a single regex for a function call. And now we process
line by line, which allows us to more easily ignore comments.
This fixes an issue where if you saved a Python file (even just
changing whitespace) while casper tests were running, the Tornado
server being used would restart, triggering a confusing error like
this:
ReferenceError: Can't find variable: $
Traceback:
undefined:2
:4
Suite explicitly interrupted without any message given.
This tools lets us view circular dependencies in our JS
code. It does regex parsing, so it has a few false positives,
but it's an early draft of the tools. Steve Howell helped
with this commit.
Apparently Travis CI has a very strange issue today that causes our
Nagios/E2E tests to have Tornado failing to connect to RabbitMQ.
Causes unknown, but I've spent a day trying to debug this without
luck, and we need our test suites passing in the meantime.
In this commit we add the ability of recognizing comments in
handlebar and Jinja2 templates to the template parser. This
fixes issue of template parser picking up code tags which are
commented out in files.
Also we fix the problem of too much deep nesting in the Jinja2
logic statement. Now only nested Jinja2 logic statements will get
a deeper indentation level.With this another fix was introduced
relating with the tags before a nested handlebar or django tag getting
wrong indent.
We also fix the issue with wrong offsets with closing tags in
cases if they were not matching indent level of the starting
tag intially.
Also we also start to ignore any tags occuring in between 'pre'
tags for deeper indent levels. As well we start to filter out django
non block tags from getting deeper indent level.
Our linter for translation strings shouldn't check test files, since
then we'll end up translating non-user-facing strings.
So we fix that, and actually add the opposite lint rule.
Change applies to both subdomains and non-subdomains case, though we use
just the EXTERNAL_HOST in the non-subdomains case if there is only 1 realm.
Fixes#3903.
- Add new 'missedmessage_email_senders' queue for sending missed messages emails.
- Add the new worker to process 'missedmessage_email_senders' queue.
- Split aggregation missed messages and sending missed messages email
to separate queue workers.
- Adapt tests for sending missed emails to the new logic.
Fixes#2607
- Shell command `sort` depends on system locale and
symbol `_` (underscore) is located after letters
in the default locale of Travis instances. Otherwise,
python sorting uses its own symbols ordering and
underscore is located before letters. Changing the
locale for `sort` shell command with adding environment
variable doesn' t work on Travis instances. That's why
It was decided to apply the same sorting command to both
comparing lists.
This code was added as part of the Django 1.10 migration to make our
tests work with both Django 1.8 and 1.10. Now that we're on 1.10,
it's no longer required.
This is a common mistake (I'll add documentation on why in the next
commit), and this is a great hook for sending folks to the
documentation on our frontend build process.
Like many rare-case code with new tests, it turns out that the logic
for handling null characters in our Zephyr postgres query escaping
never worked, in multiple ways. First, it always changed the second
character in s, not the current one being inspected, and second, the
value it replaced it with was no the correct postgres escape of the
null byte. We fix this and add tests.
This completes the effort to get zerver/views/messages.py to 100%
test coverage.
Fixes#1006.
- Add stamp file creation for the failed templates compilation.
- Add error response to `home` route if stamp file exists. It appears
just for the development environment.
- Add jinja2 template for failed handlebars templates compilation error.
Fixes#3650.
Move zulip-emoji from its current location `images/emoji` to
`images/emoji/unicode` and add a symlink in `images/emoji`
to zulip.png in `images/emoji/unicode`.
This adds to Zulip support for a user changing their own email
address.
It's backed by a huge amount of work by Steve Howell on making email
changes actually work from a UI perspective.
Fixes#734.
In this commit we are modifying pretty print tool to support
Django and html singleton tags. For Addition of html singleton
tags template parser was modified to emit psudeo
html singleton end tags to accompany html singleton tags and
token class was updated to have line_span field.
In this commit we improve the way errors are handled in our
template parser and thus improving the displayed messages in
case of errors. Eg. Errors in case of unbalanced quotes now
makes more sense displaying line and column information
including line where error might be sourced.
* Now queue_workers.py sorts queue names and prints them on their own
line. Previously it's output was nondeterministic.
* Simplified grep strategy for removing the "test" worker.
This list was likely to end up out of date quickly, since it wasn't
documented that you need to update it when adding a queue. The best
solution is to just not require it to be updated.
This adds an assertion, when `test-backend` is run with `--coverage`,
that we have 100% test coverage on a list of files that we expect to.
There's a whitelist/blacklist, managed in tools/test-backend.
Fixes#3363.
In This commit we extend the work being done by @showell in PR#1778
to develop a tool to pretty print html and our handlebar templates
in order to enforce our style convention of 4 Space indentation in
templates.
This commit introduces following changes:
* Fix Py3 Compatibility.
* Add ability to prettify in cases when html tags are not the
starting of a line and addition of test cases for it.
* Add ability to lint handlebar tags and add test cases for it.
* Add {{else}} as special case of indent.
* Add test cases in general to testing new tool.
@showell Helped me throughout and reviewed this commit.
Fixes#1778
This fixes an issue where provision would fail if the user's home
directory was setup in such a way that the postgres user couldn't
access it (and thus the `sudo` command here would throw errors about
having a non-readable current working directory).
The original test was written in shell script which launches a new
django instance for every tests. By doing it in Python, we avoid
the overhead and reduce the test time to <1 second.
Fixes#3620.
Now all the bots that are stored in contrib_bots are in the
same file/directory format.
The format is specified here #3427. Add tests.py file for encrypt_bot as well.
Fixes#3427.
Breaks out the Hello World example to create a new
webhook-walkthrough.md. Includes minor edits so the two docs
read well. Adds the new page, "Webhook walkthrough", to the TOC.
Fixes#3498
This helps make the Zulip development environment somewhat more robust
to new contributors, since it will give them a nice warning if they
try running any of our development tools outside the Zulip virtualenv.
Fixes#3468.
This saves more than 50MB of unnecessary duplicate content in release
tarballs (the static assets that will actually be served have already
been built at this point by tools/minify-js).
Zulip's previous model for managing static asset files via Django
pipeline had some broken behavior around upgrades. In particular, it
was for some reason storing the information as to which static files
should be used in a memcached cache that was shared between different
deployments of Zulip. This means that during the upgrade process,
some clients might be served a version of the static assets that does
not correspond to the server they were connected to.
We've replaced that model with using ManifestStaticFilesStorage, which
instead allows each Zulip deployment directory to have its own
complete copy of the mapping of files to static assets, as it should
be.
We have to do a little bit of hackery with the staticfiles.json path
to make this work, basically because Django expects staticfiles.json
to be under STATIC_ROOT (aka the path nginx is serving to users), but
doing that doesn't really make sense for Zulip, since that directory
is shared between different deployments.
- Add pid file of development processes group, which allows to
manage development processes group with os utils. Also it allows to
kill subprocesses when parent process was closed incorrectly.
- Add tool 'stop_dev_server' to stop development server by pid file.
Fixes#1547
This is a fairly major overhaul of the CSS parser to support
line numbers in error messages.
Basically, instead of passing "slices" of tokens around, we pass
indexes into the token arrays to all of our sub-parsers, which
allows them to have access to previous tokens in certain cases.
This is particularly important for errors where stuff is missing
(vs. being wrong).
In testing this out I found a few more places to catch errors.
The goal of this library is to make it a lot easier to prevent bugs
like CVE-2017-0881 by having all of our views logic for fetching a
stream go through a couple carefully tested code paths.
Previously, if you searched for ':offi..' you would see both 🏢 and
:office_building: as possible completions, both of which are shortcodes for
the same unicode codepoint (and hence which have the same image). Also, we
sort the emoji in our emoji pickers alphabetically by shortcode, and so the
images for 🏢 and :office_building: show up next to each other, which
looks like a bug. This removes :office_building: as a shortcode, along with
several hundred other duplicates. It leaves some duplicates in that won't
give autocomplete or alphabetical ordering a problem, like (🚗,
:automobile:).
Replaces the hardcoded list of emoji_names and unicode_emoji_names in
static/js/emoji.js with a list generated from emoji_map.json, both to get
the list out of version control and so we can start modifying it for our
autocomplete. This does not change the contents of emoji_names. It sorts and
removes duplicates from unicode_emoji_names (causes no change in behavior,
since unicode_emoji_names is only used as if it were a set).
Since build_emoji will soon be generated
static/generated/emoji_data.js (containing the emoji data), we need to
generate those JavaScript files before minifying them.
In order to make the layout of all bots consistent, this commit
moves each bot into a folder with its name and modifies 'run.py'
so that only bots with such a structure can be executed. 'lib'
gets renamed to 'bots'.
The script now outputs bullet points to the user when
it fails, and there are some basic comments at the top
of the file. I also fixed the path of the log file.
Fixes#3230
Previously, if a script called by provision threw an error, the
traceback for the called script would be lost far above the traceback
from provision itself in the terminal history, resulting in a great
deal of confusion about what the actual problem was.
Before this commit, provisioning was done by executing provision.py,
which printed the log directly to stdout, making debugging harder.
This commit creates a wrapper bash script 'provision' in tools, which
calls 'zulip/scripts/tools/provision_vm.py' (the new location of
provision.py) and prints all the output to
'zulip/var/log/zulip/zulip_provision.log' via 'tee'.
Travis tests and docs have been modified accordingly.
Contributor visualization showing the avatar, user name and number
of commits for each contributors. The JSON data would be updated
upon deployment, triggered by the `update-prod-static` script.
This new module abstracts the setting up of a test
server for tests to run, pulling existing code from
casper and paving the way for API tests in the future.
In this commit we enhance our current template linter to detect
duplicate ids and report them during lint checks. html_branches.py
was topped up with a new function build_id_dict for the purpose.
Also the get_tag_info function in same file was updated to parse
ids and classes more robustly in cases of template variables.
split_for_id_and_class function was added to serve this purpose.
Unit tests for both the functions were created under
tests/test_html_branches. Also a directory under tests called
test_template_data was created to hold templates for testing under
newly created functionality.
check_templates was modified to print to console any duplicates
detected.
showell reviewed my commit and helped me out.
Fixes#2950.
This script ensures you are starting on master in a pristine
state, and then it creates a branch called review-NNNN for
PR #NNNN in your local repo, along with some basic stats about
the PR.
This checks both that all images under static/images/help/ are used in
the help documentation, and also that none of the image tags are broken.
- Improve documentation spiders and crawler with spider error state.
Fixes#3070.
Previously, we would incorrectly be counting bytes in Python 2, which
meant lines with unicode characters in them appeared to the linter to
be far longer than they actually were.
Apparently `manage.py collectstatic` by default strips files starting
with "." from what it collects. This is a reasonably precaution,
though mostly irrelevant to us, since Zulip primarily runs that as
part of build-release tarball, which runs in a clean directory.
It also breaks our current approach for transferring node_modules to
prod machines via release tarballs; this change fixes that bug.
Eeshan Garg
Yago González
adnrs96
Umair Khan
Steve Howell
Tim Abbott
Elliott Jin
Tommy Ip
Rishi Gupta
Harshit Bansal
K.Kanakhin
sinwar
Raghav Jajodia
Saumya Rawat
Abhijeet Kaur
Brock Whittaker
Feorlen
hackerkid
Brock Whittaker
Tomasz Kolek
Zev Benjamin
Robert Hönig
Rohith Asrk
Sampriti Panda
JefftheBest1
AZtheAsian
Tim Abbott
ausDensk