This code was added as part of the Django 1.10 migration to make our
tests work with both Django 1.8 and 1.10. Now that we're on 1.10,
it's no longer required.
This is a common mistake (I'll add documentation on why in the next
commit), and this is a great hook for sending folks to the
documentation on our frontend build process.
Like many rare-case code with new tests, it turns out that the logic
for handling null characters in our Zephyr postgres query escaping
never worked, in multiple ways. First, it always changed the second
character in s, not the current one being inspected, and second, the
value it replaced it with was no the correct postgres escape of the
null byte. We fix this and add tests.
This completes the effort to get zerver/views/messages.py to 100%
test coverage.
Fixes#1006.
- Add stamp file creation for the failed templates compilation.
- Add error response to `home` route if stamp file exists. It appears
just for the development environment.
- Add jinja2 template for failed handlebars templates compilation error.
Fixes#3650.
Move zulip-emoji from its current location `images/emoji` to
`images/emoji/unicode` and add a symlink in `images/emoji`
to zulip.png in `images/emoji/unicode`.
This adds to Zulip support for a user changing their own email
address.
It's backed by a huge amount of work by Steve Howell on making email
changes actually work from a UI perspective.
Fixes#734.
In this commit we are modifying pretty print tool to support
Django and html singleton tags. For Addition of html singleton
tags template parser was modified to emit psudeo
html singleton end tags to accompany html singleton tags and
token class was updated to have line_span field.
In this commit we improve the way errors are handled in our
template parser and thus improving the displayed messages in
case of errors. Eg. Errors in case of unbalanced quotes now
makes more sense displaying line and column information
including line where error might be sourced.
* Now queue_workers.py sorts queue names and prints them on their own
line. Previously it's output was nondeterministic.
* Simplified grep strategy for removing the "test" worker.
This list was likely to end up out of date quickly, since it wasn't
documented that you need to update it when adding a queue. The best
solution is to just not require it to be updated.
This adds an assertion, when `test-backend` is run with `--coverage`,
that we have 100% test coverage on a list of files that we expect to.
There's a whitelist/blacklist, managed in tools/test-backend.
Fixes#3363.
In This commit we extend the work being done by @showell in PR#1778
to develop a tool to pretty print html and our handlebar templates
in order to enforce our style convention of 4 Space indentation in
templates.
This commit introduces following changes:
* Fix Py3 Compatibility.
* Add ability to prettify in cases when html tags are not the
starting of a line and addition of test cases for it.
* Add ability to lint handlebar tags and add test cases for it.
* Add {{else}} as special case of indent.
* Add test cases in general to testing new tool.
@showell Helped me throughout and reviewed this commit.
Fixes#1778
This fixes an issue where provision would fail if the user's home
directory was setup in such a way that the postgres user couldn't
access it (and thus the `sudo` command here would throw errors about
having a non-readable current working directory).
The original test was written in shell script which launches a new
django instance for every tests. By doing it in Python, we avoid
the overhead and reduce the test time to <1 second.
Fixes#3620.
Now all the bots that are stored in contrib_bots are in the
same file/directory format.
The format is specified here #3427. Add tests.py file for encrypt_bot as well.
Fixes#3427.
Breaks out the Hello World example to create a new
webhook-walkthrough.md. Includes minor edits so the two docs
read well. Adds the new page, "Webhook walkthrough", to the TOC.
Fixes#3498
This helps make the Zulip development environment somewhat more robust
to new contributors, since it will give them a nice warning if they
try running any of our development tools outside the Zulip virtualenv.
Fixes#3468.
This saves more than 50MB of unnecessary duplicate content in release
tarballs (the static assets that will actually be served have already
been built at this point by tools/minify-js).
Zulip's previous model for managing static asset files via Django
pipeline had some broken behavior around upgrades. In particular, it
was for some reason storing the information as to which static files
should be used in a memcached cache that was shared between different
deployments of Zulip. This means that during the upgrade process,
some clients might be served a version of the static assets that does
not correspond to the server they were connected to.
We've replaced that model with using ManifestStaticFilesStorage, which
instead allows each Zulip deployment directory to have its own
complete copy of the mapping of files to static assets, as it should
be.
We have to do a little bit of hackery with the staticfiles.json path
to make this work, basically because Django expects staticfiles.json
to be under STATIC_ROOT (aka the path nginx is serving to users), but
doing that doesn't really make sense for Zulip, since that directory
is shared between different deployments.
- Add pid file of development processes group, which allows to
manage development processes group with os utils. Also it allows to
kill subprocesses when parent process was closed incorrectly.
- Add tool 'stop_dev_server' to stop development server by pid file.
Fixes#1547
This is a fairly major overhaul of the CSS parser to support
line numbers in error messages.
Basically, instead of passing "slices" of tokens around, we pass
indexes into the token arrays to all of our sub-parsers, which
allows them to have access to previous tokens in certain cases.
This is particularly important for errors where stuff is missing
(vs. being wrong).
In testing this out I found a few more places to catch errors.
The goal of this library is to make it a lot easier to prevent bugs
like CVE-2017-0881 by having all of our views logic for fetching a
stream go through a couple carefully tested code paths.
Previously, if you searched for ':offi..' you would see both 🏢 and
:office_building: as possible completions, both of which are shortcodes for
the same unicode codepoint (and hence which have the same image). Also, we
sort the emoji in our emoji pickers alphabetically by shortcode, and so the
images for 🏢 and :office_building: show up next to each other, which
looks like a bug. This removes :office_building: as a shortcode, along with
several hundred other duplicates. It leaves some duplicates in that won't
give autocomplete or alphabetical ordering a problem, like (🚗,
:automobile:).
Replaces the hardcoded list of emoji_names and unicode_emoji_names in
static/js/emoji.js with a list generated from emoji_map.json, both to get
the list out of version control and so we can start modifying it for our
autocomplete. This does not change the contents of emoji_names. It sorts and
removes duplicates from unicode_emoji_names (causes no change in behavior,
since unicode_emoji_names is only used as if it were a set).
Since build_emoji will soon be generated
static/generated/emoji_data.js (containing the emoji data), we need to
generate those JavaScript files before minifying them.
In order to make the layout of all bots consistent, this commit
moves each bot into a folder with its name and modifies 'run.py'
so that only bots with such a structure can be executed. 'lib'
gets renamed to 'bots'.
The script now outputs bullet points to the user when
it fails, and there are some basic comments at the top
of the file. I also fixed the path of the log file.
Fixes#3230
Previously, if a script called by provision threw an error, the
traceback for the called script would be lost far above the traceback
from provision itself in the terminal history, resulting in a great
deal of confusion about what the actual problem was.
Before this commit, provisioning was done by executing provision.py,
which printed the log directly to stdout, making debugging harder.
This commit creates a wrapper bash script 'provision' in tools, which
calls 'zulip/scripts/tools/provision_vm.py' (the new location of
provision.py) and prints all the output to
'zulip/var/log/zulip/zulip_provision.log' via 'tee'.
Travis tests and docs have been modified accordingly.
Contributor visualization showing the avatar, user name and number
of commits for each contributors. The JSON data would be updated
upon deployment, triggered by the `update-prod-static` script.
This new module abstracts the setting up of a test
server for tests to run, pulling existing code from
casper and paving the way for API tests in the future.
In this commit we enhance our current template linter to detect
duplicate ids and report them during lint checks. html_branches.py
was topped up with a new function build_id_dict for the purpose.
Also the get_tag_info function in same file was updated to parse
ids and classes more robustly in cases of template variables.
split_for_id_and_class function was added to serve this purpose.
Unit tests for both the functions were created under
tests/test_html_branches. Also a directory under tests called
test_template_data was created to hold templates for testing under
newly created functionality.
check_templates was modified to print to console any duplicates
detected.
showell reviewed my commit and helped me out.
Fixes#2950.
This script ensures you are starting on master in a pristine
state, and then it creates a branch called review-NNNN for
PR #NNNN in your local repo, along with some basic stats about
the PR.
This checks both that all images under static/images/help/ are used in
the help documentation, and also that none of the image tags are broken.
- Improve documentation spiders and crawler with spider error state.
Fixes#3070.
Previously, we would incorrectly be counting bytes in Python 2, which
meant lines with unicode characters in them appeared to the linter to
be far longer than they actually were.
Apparently `manage.py collectstatic` by default strips files starting
with "." from what it collects. This is a reasonably precaution,
though mostly irrelevant to us, since Zulip primarily runs that as
part of build-release tarball, which runs in a clean directory.
It also breaks our current approach for transferring node_modules to
prod machines via release tarballs; this change fixes that bug.
- Add websocket client to create connection with SockJS websocket server.
It contains callback method to launch after connection setup.
- Add '--websocket' parameter to 'check_send_receive_time' script to
check websocket connection.
- Add testing websocket connection to production installation checking.
- Add cronjob to launch websocket connection nagios test.
This makes it possible for Zulip Nagios monitoring to check for
problems impacting the websockets sending code path, which is what all
web users use.
Previously, we got the following as a part of the output when running
`tools/run-dev.py` without provisioning:
It looks like you checked out a branch that has added
dependencies beyond what you last provisioned. Your tests
are likely to fail until you add dependencies by provisioning.
which is a bit confusing.
This finishes clearing unnecessarily excluded items from the global
tools/lint-all exclude list. The remaining items are third-party code
or generated code that doesn't make sense to lint.
Fixes#822.
This required instead exempting these files:
bots/jabber_mirror_backend.py
tools/deprecated/iframe-bot/show-last-messages
Turning on mypy for the API exposes issues in the annotations
of other files.
We used to create endpoints with Client._register.
Now we now have explicit methods for the endpoints.
This allows us to add docstrings and stricter mypy annotations.
This fix also introduces a call_endpoint() method that avoids
the need for manually building urls with API_VERSTRING when you
know the URL pattern of the endpoint you want to hit (and when
the API doesn't have a convenient wrapper).
I fixed a bug with create_users where it now uses PUT instead
of POST.
I also removed client.export(), which was just broken.
I had to change recent-messages and zulip-export, which were
using client.do_api_query and Client._register.
Now it's easier to just call client.call_endpoint() for
situations where our API doesn't have convenient wrappers,
so that's what I did with those scripts.
- Add rewriting 'X-REAL-IP' header for each request through dev
proxy server.
- Change webpack configuration ip to 0.0.0.0 to fix socket.io build
for launching remote dev server.
The following emojis are colored white (with a transparent background)
and are hard to read in the context of Zulip. This commit changes their
colors to blue. The emojis are:
hash, zero, one, two, three, four, five, six, seven, eight and nine.
This fixes#1969.
- Add spider for documentation crawler, which processes user help
documentation on test development server.
- Add tool script for help documentation checking launching.
Fixes#2639
Previously, we were doing this request to the production server before
waiting for all the supervisord processes to start; it's possible this
could cause failures where we hit the server before the Django uwsgi
processes are up.
Hopefully fixes#2723.
Most of the changes to support this were merged some time ago; what
remains are these changes:
* Update requirements.txt
* Django 1.10: Upgrade success-http-headers.txt file.
- We no longer get the absolute urls, see
https://docs.djangoproject.com/en/1.10/releases/1.9/#http-redirects-no-longer-forced-to-absolute-uris
- The headers are capitalized, previously, they were in upper case.
* Bump PROVISON_VERSION to 3.0 since this is a disruptive change.
Fixes#3.
This is a major change to the /#subscriptions page, converting it to
by a side-by-side list of streams and their settings in an overlay.
There are no new features added/removed, but it's a huge changeset,
because it replaces the old navigation logic and moves the stream
creation modal to appear in the right side of this overlay.
- Replace download-zxcvbn with downloading it from npm.
- Change zxcvbn.js path to node_modules (because npm put it to
`node_modules` directory.
- Bump `PROVISION_VERSION` in `version.py` to 2.4.
Fixes#2423.
When we migrated run-dev.py from Twisted to Tornado a few weeks ago,
the --interface argument wasn't properly ported and thus was ignored.
This restores the original functionality of defaulting to only
listening on localhost.
Ideally, we'd replace the vagrant/zulipdev user check with something
that just checks whether a special file that is created by the
Vagrant/remote-dev-vm creation process exists; that would be more
robust.
(Why is -u needed at all? I’m not sure, but test-run-dev spins forever
“Polling run-dev...” without it.)
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Previously, we didn't install/pin our own node.js version, and thus
had this wrapper script to manage it. Now that install our own node
via nvm, there's no reason we still need this.
Fixes#2409.
We now instrument URL coverage whenever you run the back end tests,
and if you run the full suite and fail to test all endpoints, we
exit with a non-zero exit code and report failures to you.
If you are running just a subset of the test suite, you'll still
be able to see var/url_coverage.txt, which has some useful info.
With some tweaks to the output from tabbott.
Fixes#1441.
This commit adds a basic eslintrc that emulates jslint defaults.
Rules that conflict with our existing code have been switched to
warnings instead of errors. Globals have been added to the eslintrc. The
bundled js file (generated by webpack) and blueslip.js are ignored with
.eslintignore.
To display warnings, run npm run lint-loud. This runs eslint without the
--quiet option on static/js and frontend_tests.
npm run --silent lint is run by tools/lint-all (in addition to jslint).
The --silent option is used to suppress the default output from npm run.
Fixes#535.
This helps automatically detect a common issue where users try to use
the Vagrant development environment on windows without administrator
permissions, which results in errors trying to create symlinks.
We set this up initially with all of the rules that Zulip violates
disabled.
Also, the pep8 linter is substantially slower than the other Zulip
linters, so we've put it behind an option to `tools/lint-all`.
These are not particularly interesting to measure test coverage for,
since migrations are not run as the part of the test suite by
construction, and management commands aren't being tested by this test
suite.
- Use tornado as proxy server for development environment,
replacing twisted (this doesn't support websockets).
- Upgrade tornado version to 4.4.1 (needs to be coupled to the
above since neither change works without the other)
This creates the new topic_list.js module, and the first
function that we extract is topic_list.update_count_in_dom().
This function needed to be decoupled from some non-topic-list
stuff which was overly complicated.
Sometimes remote servers are unavailable or on maintance
what is the cause of broken back-end tests. Such response errors
is excluded from raised exception cases and just notify about
wrong link in test log.
- Add script to compile documentation build and start crawler
to check documentation.
- Add documentation test script to backend travis test case.
- Add log level argument to test-documentation script.
Fixes#1492
- Add scrapy project with rewrited 'crawl' command and spider to check documentation.
Command was rewrited due to return exit status by the exception existing,
it returns exit code 0 if exception happens inside spider in standard behavour.
- Add scrapy requirements to for dev environment. It was added to twisted requirements list.
Fixes#1492
This renames the old `emoji_dump.py` to `build_emoji`, removing the
old shell essentially empty shell script. `emoji_dump.py` was always
a weird name, and this makes it a bit easier to read the code for this
system.
The old "zulip_internal" name was from back when Zulip, Inc. had two
distributions of Zulip, the enterprise distribution in puppet/zulip/
and the "internal" SAAS distribution in puppet/zulip_internal. I
think the name is a bit confusing in the new fully open-source Zulip
work, so we're replacing it with "zulip_ops". I don't think the new
name is perfect, but it's better.
In the following commits, we'll delete a bunch of pieces of Zulip,
Inc.'s infrastructure that don't exist anymore and thus are no longer
useful (e.g. the old Trac configuration), with the goal of cleaning
the repository of as much unnecessary content as possible.
Previously, we checked scripts in a separate run to work around mypy
not supporting multiple scripts with the same name. Since we have
fixed that issue, we can restore the original behavior.
We leave the --scripts-only option available, though I'm not sure it's
particularly useful and we'll probably eventually remove it.
Tim Abbott
Raghav Jajodia
Rishi Gupta
Harshit Bansal
Elliott Jin
K.Kanakhin
Umair Khan
adnrs96
Saumya Rawat
Tommy Ip
Abhijeet Kaur
Brock Whittaker
Feorlen
sinwar
hackerkid
Steve Howell
Yago González
Brock Whittaker
Tomasz Kolek
Zev Benjamin
Robert Hönig
Rohith Asrk
Sampriti Panda
JefftheBest1
AZtheAsian
Tim Abbott
ausDensk
Jackson
Rafid Aslam
anirudhjain75
lonerz
Arpith Siromoney
reallyly
Adarsh S
Zac Pullar-Strecker
Tudor Nazarie
Joy Chen
Jason Le
bulat22101
kevv87
Sidhant Bhavnani
Alex Huang
Vamshi Balanaga
Bickio
Heidi Dong
Anders Kaseorg
sylvan1
Vladislav Manchev
paxapy
Pweaver (Paul Weaver)
Christie Koehler
Shashank
Tomasz Kolek
sonali0901
umkay
deekshaarul
Prabhakar Gupta
Sumana Harihareswara