py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2. In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.
One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2. See discussion on the respective previous
commits that made those explicit. There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-02 23:15:16 +02:00
|
|
|
#!/usr/bin/env python3
|
2013-01-30 23:11:34 +01:00
|
|
|
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
# Updates static files for production.
|
2013-01-30 23:11:34 +01:00
|
|
|
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
|
|
|
|
import os
|
2016-08-29 00:56:05 +02:00
|
|
|
import argparse
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
import sys
|
|
|
|
|
|
|
|
# We need settings so we can figure out where the prod-static directory is.
|
|
|
|
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
|
2016-07-19 08:57:12 +02:00
|
|
|
import scripts.lib.setup_path_on_import
|
2013-08-06 22:51:47 +02:00
|
|
|
os.environ['DJANGO_SETTINGS_MODULE'] = 'zproject.settings'
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
from django.conf import settings
|
2016-08-18 13:55:13 +02:00
|
|
|
from scripts.lib.node_cache import setup_node_modules
|
2018-06-01 01:45:18 +02:00
|
|
|
from scripts.lib.zulip_tools import run
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
|
2017-02-07 20:49:12 +01:00
|
|
|
# check for the venv
|
|
|
|
from lib import sanity_check
|
|
|
|
sanity_check.check_venv(__file__)
|
|
|
|
|
2016-08-29 00:56:05 +02:00
|
|
|
parser = argparse.ArgumentParser()
|
|
|
|
parser.add_argument('--prev-deploy', metavar='DIR',
|
2016-12-11 14:30:45 +01:00
|
|
|
help='a previous deploy from which to reuse files if possible')
|
2017-01-24 07:19:25 +01:00
|
|
|
parser.add_argument('--authors-not-required', action='store_true', default=False,
|
|
|
|
help='Authors files need not be updated')
|
2016-08-29 00:56:05 +02:00
|
|
|
args = parser.parse_args()
|
|
|
|
prev_deploy = args.prev_deploy
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
|
|
|
|
os.chdir(settings.DEPLOY_ROOT)
|
2013-01-30 23:11:34 +01:00
|
|
|
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
# Redirect child processes' output to a log file (most recent run only).
|
2018-07-18 23:50:16 +02:00
|
|
|
os.makedirs("var/log", exist_ok=True)
|
2016-07-19 15:52:15 +02:00
|
|
|
fp = open('var/log/update-prod-static.log', 'w')
|
2013-06-27 18:56:02 +02:00
|
|
|
|
2015-10-26 17:16:26 +01:00
|
|
|
# Install node packages
|
2017-07-22 01:19:53 +02:00
|
|
|
setup_node_modules(production=True, stdout=fp, stderr=fp)
|
2015-10-26 17:16:26 +01:00
|
|
|
|
2017-01-28 21:12:01 +01:00
|
|
|
# Build emoji
|
2018-06-01 01:45:18 +02:00
|
|
|
run(['./tools/setup/emoji/build_emoji'], stdout=fp, stderr=fp)
|
2017-01-28 21:12:01 +01:00
|
|
|
|
2017-09-16 14:30:57 +02:00
|
|
|
# Inline CSS in emails
|
2019-11-16 02:38:42 +01:00
|
|
|
run(['./scripts/setup/inline-email-css'], stdout=fp, stderr=fp)
|
2017-09-16 14:30:57 +02:00
|
|
|
|
2017-09-17 02:14:56 +02:00
|
|
|
# Copy over static files from the zulip_bots package
|
2018-12-09 16:13:46 +01:00
|
|
|
run(['./tools/setup/generate_zulip_bots_static_files.py'], stdout=fp, stderr=fp)
|
2017-09-17 02:14:56 +02:00
|
|
|
|
2017-04-26 09:32:46 +02:00
|
|
|
# Build pygment data
|
2018-06-01 01:45:18 +02:00
|
|
|
run(['./tools/setup/build_pygments_data'], stdout=fp, stderr=fp)
|
2017-04-26 09:32:46 +02:00
|
|
|
|
2019-06-29 10:26:57 +02:00
|
|
|
# Create webpack bundle
|
|
|
|
run(['./tools/webpack'], stdout=fp, stderr=fp)
|
2013-01-30 23:11:34 +01:00
|
|
|
|
2019-07-18 23:39:36 +02:00
|
|
|
# Generate /team page markdown for authors
|
|
|
|
authors_cmd = ['./tools/update-authors-json']
|
|
|
|
if os.environ.get("TRAVIS"):
|
|
|
|
authors_cmd.append("--use-fixture")
|
|
|
|
if args.authors_not_required:
|
|
|
|
authors_cmd.append("--not-required")
|
|
|
|
run(authors_cmd, stdout=fp, stderr=fp)
|
|
|
|
|
2017-01-07 05:09:10 +01:00
|
|
|
# Collect the files that we're going to serve; this creates prod-static/serve.
|
2020-02-21 00:20:39 +01:00
|
|
|
run([
|
|
|
|
'./manage.py', 'collectstatic', '--no-default-ignore',
|
|
|
|
'--noinput',
|
|
|
|
'-i', 'assets',
|
|
|
|
'-i', 'emoji-styles',
|
|
|
|
'-i', 'html',
|
|
|
|
'-i', 'js',
|
|
|
|
'-i', 'styles',
|
|
|
|
'-i', 'templates',
|
|
|
|
], stdout=fp, stderr=fp)
|
2018-06-01 01:45:18 +02:00
|
|
|
|
2016-05-02 10:20:00 +02:00
|
|
|
# Compile translation strings to generate `.mo` files.
|
2018-06-01 01:45:18 +02:00
|
|
|
run(['./manage.py', 'compilemessages'], stdout=fp, stderr=fp)
|
2016-05-02 10:20:00 +02:00
|
|
|
|
2018-06-01 01:45:18 +02:00
|
|
|
# Needed if PRODUCTION
|
2018-07-18 23:50:16 +02:00
|
|
|
os.makedirs('prod-static', exist_ok=True)
|
2018-06-01 01:45:18 +02:00
|
|
|
|
Reuse minified JS from previous deploys
This is a big change affecting lots of areas:
* Pipeline no longer deals with JS (though it still minifies CSS)
* A new script, tools/minify-js (called from update-prod-static),
minifies JavaScripts
* A command-line argument --prev-deploy, if passed to minify-js or
update-prod-static, is used to copy minified JS from a previous
deploy (i.e., a previous git checkout), if the source files have
not changed
* update-deployment passes --prev-deploy
* Scripts are now included with the minified_js template tag, rather
than Pipeline's compressed_js
Also, as a side benefit of this commit, our Handlebars templates will
no longer be copied into prod-static/ and accessible in production.
Unminification is probably broken, but, per Zev and Trac ticket #1377,
it wasn't working perfectly before this change either.
(Based on code review, this commit has been revised to:
* Warn if git returns an error in minify-js
* Add missing output redirects in update-prod-static
* Use DEPLOY_ROOT instead of manually constructing that directory
* Use old style formatting)
(imported from commit e67722ea252756db8519d5c0bd6a421d59374185)
2013-07-03 22:42:25 +02:00
|
|
|
fp.close()
|