Fixes#5887. It seems there's some sort of issue in CPython where it
can get confused into thinking a `.pyc` file that's actually stale is
up to date -- perhaps when they date from the same second, say from
the middle of a rebase.
For now, rather than dig further to fix this properly and be sure of
having really done so, just go back to wiping out all `.pyc` files.
The impact is about 1s; that's noticeable when running a short test
file on a quad-core machine (about 8s), but not so much on a more
typical dev machine or on a larger set of tests.
This new module tracks the recent topic names for any given
stream.
The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.
We introduced a minor one-line function called stream_has_topics.
Notably, this adds our checks on translated message strings to
`tools/test-all`, so that they don't cause surprise failures in
CI after a branch is pushed. (Alternately they could have gone
in `tools/lint` to accomplish the same goal, but `makemessages`
which they depend on is quite slow -- on my machine it takes 7s,
compared to 10s for all of `tools/lint`.)
static/ serves static files which get copied around per deploy. Since
the webpack stats files need a consistent name and change per deploy,
they can't live in static/.
This fixes a bug that preventing downgrading a Zulip server to an old
version.
This allow the webbpack dev server to properly reload JavaScript modules
while running in dev without restarting the server. We need to connect
to webpack-dev-server directly because SockJS doesn't support more than
one connection on the same host/port.
This hack saved a lot of time debugging weird issues back in 2016, but
it's no longer needed.
Anyone rebasing a branch from 2016 will need to provision afterwards
regardless, which will fix this issue automatically, and more
importantly, these changes were made obsolete when we moved to the
cached `node_modules` model.
Updates `get_success_stamp()` function to use the `emoji-datasource`
package's version while calculating success stamp so that an emoji
cache rebuild gets triggered automatically if the version is changed.
Apparently, this was missing on Xenial, and we just never noticed
because the package was installed by default on Ubuntu.
It doesn't exist yet in Trusty.
This tool was used for downloading sprite sheets from iamcal's
repository. Since now we have moved to using `emoji-datasource`
npm package, this tool is no longer required.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
`emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
package.
Bumps PROVISION_VERSION.
Fixes: #4730.
NPM packages should be installed at the beginning of the provisioning
process so that later in the provisioning process if a script requires
any NPM package it can use it. Earlier, we were installing NPM packages
in the last as the installation process can fail due to network issues
but since we now retry in case the installation fails, they can be
installed safely at the beginning of the process as well just like apt
packages.
This mostly sets the stage for a subsequent commit to start
using client_message_id as the key into sent_messages.
It has the nice side effect of making it more explicit that
certain things should always happen when transmit_message()
succeeds.
This commit does regress our node test coverage a bit.
This is mostly straightforward moving of code out of compose.js.
The code that was moved currently supports sending time
reports for sent messages, but we intend to grow out the new
module to track more state about sent messages.
The following function names in this commit are new, but their
code was basically pulled over verbatim:
process_success (was process_send_time)
set_timer_for_restarting_event_loop
clear
initialize
All the code in the new module is covered by previous tests that
had been written for compose.js. This commit only modifies
a few things to keep those tests.
The new module has 100% node coverage, so we updated `enforce_fully_covered`.
This redesigns the /help/ page sets to be a single page app that uses
history.pushState to work the same as the old app.
The big new feature is that now we have the index in a nicely designed
left sidebar.
This works around a frequent problem we've been having with Travis
CI's apt repository being busted on startup. This, in turn, caused
`apt-get install moreutils` to fail.
We attempt to fix this by adding our own retry logic.
This optimizes .pyc files removal.
Instead of removing all pyc files, it will only remove pyc files where
corresponding py file is not found. This saves about 1s in the
startup time of `run-dev.py` (and thus also `test-js-with-casper`).
Fixes#5760.
This system hasn't been in active use for several years, and had some
problems with it's design. So it makes sense to just remove it to declutter
the codebase.
Fixes#5655.
Now that we have decided to move our API bindings to a separate
repo and specify them as a pip dependency, we can stop modifying
PATH to include the api/ directory.
The commit is composed of: (1) distill out top-level dependencies in
common.txt, (2) add -e flag to the vcs-based packages because
pip-compile can't do without, (3) pip-compile/generate the locked files
then remove the -e flags from the lockfile, (4) pin pathlib2 to dev.txt
because it turns out it is a direct requirement of
documentation_crawler, (5) document the structure and add an automation
script (6) remove cryptography==1.9 from requirements/scrapy.txt since
cryptography is automatically added from pyopenssl (7) add sed command
to remove future/futures from the generated lock file in python3 (this
should have been automatically handled by pip-compile, so pending for
the feature from pip-compile)
Tweaked by tabbott to update PROVISION_VERSION and add a missing
`first` dependency.
In preparation for the upcoming linter output update, each custom
linter now has a identifier argument. This would be used for separating
each line of lints, eg:
py | Missing space around "%" at analytics/views.py line 485:
py | output += '<hr>%s\n'%(string_id,)
css | 64c64,65
css | < opacity: 1; }
css | ---
css | > opacity: 1;
css | > }
css | static/styles/lightbox.css seems to be broken:
swagger | In static/swagger/zulip.yaml:
swagger | Duplicate operationId: registerQueue
For performance reasons, we spawn each linter in a separate OS thread.
The downside of this is that all lints would end up in stdout without
much visual separation, resulting in confusing error log. This commit
introduce the `print_err` function, which shows which linter each line
of lint is from.
The file ioloop_logging.py is pretty small, but it seems to have
flaky time-sensitive code that creates noise for coverage deltas
when we do builds. Since it's small, we just exclude it.
Tweaked by tabbott to add comments.
In this commit we basically start to override the request method of
httplib2.Http() to raise an exception whenever it is called i.e.
a trial is made to access the network from test suits.
Fixes: #1472.
This brings the total no-op provision time down to around 7.3s!
Tweaked by tabbott to clean up the code, close file descriptions, and
avoid issues with a somehow corrupted file.
Fixes#5185.
This commit adds a new linter which runs from tools/travis/backend.
It runs over the translations.json file and checks if any of the
translatable string contains handlebars in it.
Fixes#5544
The Swagger specification indicates that all operationId values should be
unique. However, SwaggerParser doesn't complain during validation if that
doesn't happen, so this commit adds our own method to identify these
cases.
Also, the violations of this rule have been fixed.
Handlebars allows putting tildes (~) by the braces in mustache tags to
strip whitespaces on that side of the tag.
{{#if foo~}}
<p>Bar</p>
{{~/if}}
This way, the linter ignores any potential tilde at the ends of Handlebars
tags so we can use this feature without lint fails.
In this commit, we are adding a new tool which can be used to
clear out the emoji cache. Cache cleanup is a good thing to in
general and as well helps keep travis cache small (meaning faster
builds).
Credits to @HarshitOnGitHub as well for suggesting the use of
emoji symlink.
In order for the `include_only` linter rule to not have
any side effects, we need to explicitly add a trailing '/'
after every directory we want to include.
The main thing here is to make looping over lines be the inner
loop, instead of looping over rules. This keeps regexes in
cache, and it also avoids some O(N) checks.
This is a significant speedup for me, reducing time from 16s
to 11s.
The problem this commit solves is related to how we search for
testcases in the test files. We use a simple 'search_key' in file_data.
This will return a false positive if there is a 'longer_search_key' in
file_data.
While searching for TestCases we should use a longer key to remove the
possibility of collision. Using 'class TestCase(' should be precise
enough.
Fixes#4983
In this commit we add a wrapper around the provisioning inside
setup-backend to trigger another try at provisioning if it fails
in hopes of it will run this time fine if it was interuppted by a
network issue the first time.
Fixes: #2034.
This commit also makes the following changes:
* Bumps the version of Django to 1.11.2.
* Fixes the HTTP response headers. Now CommonMiddleware sets Content-Length
header for non-streaming response, see
https://docs.djangoproject.com/en/1.11/ref/middleware/#module-django.middleware.common.
Due to this, 'Transfer-Encoding: chunked' header is removed, which signifies
a streaming response.
In this commit we basically stop ignoring 'base.html' from linting
point of view. Basically the condition was implemented in a bad way
resulting in ignorance of every template ending with `base.html`.
As well since removing it doesn't complain about the base.html
non parsable(We have made progress with our linter) we can safely
remove the filter condition.
As part of extracting this, we exempt the library from all custom
checks on itself. This is expedient, since a lot of our
custom checks are naive about whether things are in strings, and
it is also a pain to configure individual rules.
In this commit we start to check for violations to PEP-E261 in our
codebase by default in our linters. Also we have introduced a
ignore list which has around 20 files still in violation to
PEP-E261 which we intend to clear up soon.
Now that we're not using the copy_modules functionality (basically
because including node_modules in production tarballs was a huge disk
sink), the production Zulip code isn't using `zulip-npm-cache` anyway.
And deleting that cache had a huge impact on the performance of the
development environment provisioning that we do as part of this suite.
Commit message rewritten by tabbott.
This reuses the work we did some time ago to avoid regenerating the
test database unnecessarily.
In addition to being a nice convenience for developers (since any
accomulated test data is still available), this also saves about half
the time consumed in a no-op provision.
Fixes#5182.
This commit adds mypy annotations for both the main
bots and the bots testing runner. It involves a change
to the BotHandlerApi send_message and update_message
funtions, which is compatible with every bot.
Tweaked by tabbott to use more expressive annotations.
Greg Price
Jack Zhang
Vaida Plankyte
Tim Abbott
Pweaver (Paul Weaver)
Steve Howell
Robert Hönig
Vishnu Ks
rht
Aditya Bansal
Harshit Bansal
Yago González
Brock Whittaker
Abdul Qabiz
neiljp (Neil Pilgrim)
sinwar
Rishi Gupta
James Rowan
Eeshan Garg
Tommy Ip
Cory Lynch
Umair Khan
Joshua Pan
Cynthia Lin
vaibhav
Mehanig
Rafid Aslam
Rohitt Vashishtha
Reid Barton