Commit Graph

86 Commits

Author SHA1 Message Date
Tim Abbott 4e421ebe12 scripts: Move inline-email-css from tools to scripts.
We'll be soon documenting a production workflow that involves using
it, and that means it needs to live under scripts/ (since tools/ isn't
present in release tarballs).
2019-11-15 17:39:42 -08:00
Anders Kaseorg ee9a6071fd 5xx.html: Build with webpack.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:53:15 -07:00
Anders Kaseorg a7eb3faf96 storage: Move the staticfiles.json hack to ZulipStorage.
There’s no reason to monkey-patch something that we were already
subclassing.

Removing the PRODUCTION conditional causes us to generate
staticfiles.json in the right place to begin with so we don’t need to
move it later.  It also allows Django to find staticfiles.json if
running the dev server with PIPELINE_ENABLED.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:47:51 -07:00
Anders Kaseorg 5f712b960f update-authors-json: Write to static, not STATIC_ROOT.
Otherwise the file isn’t processed by collectstatic and doesn’t end up
in the staticfiles.json manifest.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-19 11:33:08 -07:00
Anders Kaseorg ab89f40a66 generate-custom-icon-webfont: Replace with webpack webfonts-loader.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-18 12:00:00 -07:00
Anders Kaseorg b0a30aa01a storage: Remove RemoveUnminifiedFilesMixin.
This can be done much more easily with the right options to
collectstatic.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 14:16:07 -07:00
Anders Kaseorg 0cd6c75ab1 update-prod-static: Remove unused pre-webpack files.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 13:58:21 -07:00
Anders Kaseorg 079ddae4c8 minify-js: Remove; everything has been migrated to Webpack.
min/sockjs-0.3.4.min.js is not used.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-03 13:58:21 -07:00
Anders Kaseorg bbb56df6b0 i18n: Move static/locale back to locale.
As of commit cff40c557b (#9300), these
files are no longer served directly to the browser.  Disentangle them
from the static asset pipeline so we can refactor it without worrying
about them.

This has the side effect of eliminating the accidental duplication of
translation data via hash-naming in our release tarballs.

This reverts commit b546391f0b (#1148).

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 14:57:55 -07:00
Anders Kaseorg 141088586b Completely replace perfect-scrollbar with SimpleBar.
perfect-scrollbar replaces both the appearance and the behavior of the
scrollbar, and its emulated behavior will never feel native on most
platforms.  SimpleBar customizes the appearance while preserving the
native behavior.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-17 12:06:51 -07:00
Tim Abbott 500b5cdaf1 tools: Fix missing comma in collectstatic configuration.
It appears that this code did the right thing despite being written
wrong, probably due to whatever `manage.py collectstatic` does in its
argument parsing.  But in any case, we should make the code read how
it's intended.
2019-03-04 08:31:12 -08:00
Sumanth V Rao cae718f184 provision: Rename generate_zulip_bots_static_files to .py.
This should make it possible to call this script as a function without
starting up a new Python interpreter.
2018-12-09 21:42:40 -08:00
Anders Kaseorg 0bfcd2294e tools/update-prod-static: Avoid shelling out for mkdir, cp, mv, rm.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Tim Abbott c2e5cc99a4 tools: Move generate-custom-icon-webfont to a subdirectory. 2018-08-09 15:39:57 -07:00
Cynthia Lin 7d9281fa6c simplebar: Add simplebar dependency for help page scrolling. 2018-07-11 20:04:55 +05:30
Tim Abbott 751c602a0b update-prod-static: Don't copy/minify templates and styles.
This is a performance optimization: Rather than copying these files
into the `prod-static` directory and then deleting them, we just don't
copy them over in the first place.

For styles, it might have once been the case that this did something,
but we've moved them all to being managed by webpack some time ago.

For the js directory, I think it was never useful to copy and then
delete them; these files were always compiled via tools/minify-js,
and the raw JS files weren't needed, anyway.
2018-06-01 16:39:35 -07:00
Tim Abbott 5b5d1f3e87 update-prod-static: Rewrite to use the run() scripting tool.
This provides nice `set -x` style logging of which commands are
running when.
2018-05-31 16:45:51 -07:00
Gooca 2363c9955a ui scrollbar: Update perfect-scrollbar to 1.3.0.
Update perfect-scrollbar to fix stutter space-scrolling in #8544. Also
reworked deprecated `element.perfectScrollbar` to `new
PerfectScrollbar(element)`.  Lastly, updated provision version and
changed node module path to new path.

This also refactors perfect-scrollbar in help.js to work with updated
version of perfect-scrollbar. Because the update also changed
perfect-scrollbar's css selectors for all scrollbars in zulip, we
update those too.

Fixes #8544.
2018-03-20 21:59:11 -07:00
Aditya Bansal 76f6f7cb47 datetimepicker: Add flatpickr lib as dependancy. 2018-01-19 11:33:11 -05:00
rht bff736868e Generate custom-icon-webfont on each provision or update-prod-static.
Fixes #7354.
2017-11-20 16:36:49 -08:00
Tim Abbott d27002ba21 landing-page: Replace /about with /team and /history in links. 2017-10-31 12:44:05 -07:00
rht bf4eda7374 tools: Remove absolute_import in most tools.
Tweaked by tabbott to not remove it from lister.py, linter_lib, and
friends, since those are intended to support both Python 2 and 3
(we're planning to extract them from the repository).
2017-09-29 12:28:43 -07:00
Eeshan Garg bb34ba37b5 tools: Rename build_pygments_data.py -> build_pygments_data.
Dropping the file extension (.py) conforms to our naming
convention for such scripts.
2017-09-26 16:00:41 -07:00
Eeshan Garg 390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Vishnu Ks f4c69bb126 tools: Create tool for inlining email CSS during provision. 2017-09-16 07:17:48 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Brock Whittaker 83184d5c60 /about/: Merge with /authors, redesign page.
This adds the authors to the Zulip repository on GitHub from
/authors/ along with re-styling the page to fit the same
aesthetic as /for/open-source/ and other product-pages.
2017-08-15 16:09:01 -07:00
Greg Price b6d90808a5 prod: Make `cp` copy a directory to its target, not into it.
This was causing `language_options.json` and the `django.mo` files
never to get updated by `upgrade-zulip-from-git` after first deploy.
The stale `django.mo` files meant that any message string which wasn't
in the original from-tarball installation of the server would never
get translated.

The same bug exists in several other spots in this file.  Some of
our repetitive-yet-subtly-different uses of `cp` should probably
turn into just a list of arguments to one `rsync -R` command.
Also this should probably really use `rsync --delete` -- for example,
in prod on zulipchat.com we still have lying around a `locale/zh_CN`
directory, even though in a freshly-built static tree there's only
`locale/zh_Hans` instead.
2017-08-07 19:23:27 -07:00
Tim Abbott e1e5e15797 node_cache: Move npm_args inside the interface. 2017-07-21 17:28:20 -07:00
Harshit Bansal 92f6613bdd tools: Remove `tools/setup/emoji/download-emoji-data` tool.
This tool was used for downloading sprite sheets from iamcal's
repository. Since now we have moved to using `emoji-datasource`
npm package, this tool is no longer required.
2017-07-13 21:52:01 -07:00
Rafid Aslam ef2c0f806e update-prod-static: Add code to copy CSS from node_modules 2017-06-11 10:15:48 -07:00
Reid Barton 2e4078cbd2 update-prod-static: Redirect update-authors-json stderr to log file. 2017-05-24 16:44:02 -07:00
Cory Lynch 0965c43238 Add typeahead for syntax highlighting languages.
Modified composebox_typeahead.js to recognize the triple backtick
and tilde for code blocks, and added appropriate typeahead functions
in that file and in typeahead_helper.js.

Additionally, a new file pygments_data.js contains a dictionary of
the supported languages, mapping to relative popularity
rankings. These rankings determine the order of sort of the
languages in the typeahead.

This JavaScript file is actually in static/generated/pygments_data.js, as it
is generated by a Python script, tools/build_pymgents_data.py. This is
so that if Pygments adds support for new languages, the JavaScript file
will be updated appropriately. This python script uses a set of popularity
rankings defined in lang.json.

Corresponding unit tests were also added.

Fixes #4111.
2017-04-28 17:22:59 -07:00
Harshit Bansal 4470a3947b provision: Download emoji-data during provisioning.
Add code to download iamcal's emoji_pretty.json
(emoji-data/emoji.json) during provisioning.
2017-04-18 12:46:52 -07:00
Yago González 34a9e1ae11 markdown: Add TeX typesetting support.
Co-authored-by: Reid Barton <rwbarton@gmail.com>

Fixes #2056.
2017-03-21 16:40:00 -07:00
Tim Abbott 1005d70ff5 update-prod-static: Fix incorrect ordering of venv setup code.
We should check if we succeeded in using a venv only after we've
actually setup sys.path to use it.
2017-02-07 11:49:31 -08:00
sinwar eab355b0cd tools: Create more consistent checks for venv.
This helps make the Zulip development environment somewhat more robust
to new contributors, since it will give them a nice warning if they
try running any of our development tools outside the Zulip virtualenv.

Fixes #3468.
2017-02-06 21:50:32 -08:00
Tim Abbott d71f2e7b9b update-prod-static: Don't ship node_modules.
This saves more than 50MB of unnecessary duplicate content in release
tarballs (the static assets that will actually be served have already
been built at this point by tools/minify-js).
2017-02-06 17:21:34 -08:00
Tim Abbott e2ee1951e0 storage: Fix static files storage reuse issues.
Zulip's previous model for managing static asset files via Django
pipeline had some broken behavior around upgrades.  In particular, it
was for some reason storing the information as to which static files
should be used in a memcached cache that was shared between different
deployments of Zulip.  This means that during the upgrade process,
some clients might be served a version of the static assets that does
not correspond to the server they were connected to.

We've replaced that model with using ManifestStaticFilesStorage, which
instead allows each Zulip deployment directory to have its own
complete copy of the mapping of files to static assets, as it should
be.

We have to do a little bit of hackery with the staticfiles.json path
to make this work, basically because Django expects staticfiles.json
to be under STATIC_ROOT (aka the path nginx is serving to users), but
doing that doesn't really make sense for Zulip, since that directory
is shared between different deployments.
2017-02-06 16:10:24 -08:00
Tim Abbott f6f8168f4d emoji: Build emoji before minifying JS.
Since build_emoji will soon be generated
static/generated/emoji_data.js (containing the emoji data), we need to
generate those JavaScript files before minifying them.
2017-01-28 12:15:34 -08:00
Tim Abbott 126b1c4edc upgrade: Don't require authors updates deploying from Git.
Fixes #3392.
2017-01-23 23:00:09 -08:00
Tim Abbott 22d1aa396b lint: Clean up W503 PEP-8 warning. 2017-01-23 20:50:04 -08:00
Tim Abbott 4f67b9802d travis: Use fixture for authors page in CI. 2017-01-17 13:42:37 -08:00
Tommy Ip c407919db3 Add /authors page.
Contributor visualization showing the avatar, user name and number
of commits for each contributors. The JSON data would be updated
upon deployment, triggered by the `update-prod-static` script.
2017-01-17 13:35:55 -08:00
Tim Abbott 246bdbd829 update-prod-static: Fix node_modules/.bin being unavailable in prod.
Apparently `manage.py collectstatic` by default strips files starting
with "." from what it collects.  This is a reasonably precaution,
though mostly irrelevant to us, since Zulip primarily runs that as
part of build-release tarball, which runs in a clean directory.

It also breaks our current approach for transferring node_modules to
prod machines via release tarballs; this change fixes that bug.
2017-01-06 20:23:46 -08:00
Tim Abbott 08dae5108e emoji: Move tools/setup/emoji_dump to tools/setup/emoji. 2016-12-27 19:58:41 -08:00
reallyly 94b28e0afb pep8: Fix E128 violations.
With some line-wrapping tweaks by tabbott.
2016-12-13 10:50:14 -08:00
Rafid Aslam ce94fb2eb1 deps: Change zxcvbn download method from download-zxcvbn to npm.
- Replace download-zxcvbn with downloading it from npm.
- Change zxcvbn.js path to node_modules (because npm put it to
`node_modules` directory.
- Bump `PROVISION_VERSION` in `version.py` to 2.4.

Fixes #2423.
2016-12-06 17:54:56 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Prabhakar Gupta bc6421fbfb tools: Remove empty wrapper script build_emoji.
This renames the old `emoji_dump.py` to `build_emoji`, removing the
old shell essentially empty shell script.  `emoji_dump.py` was always
a weird name, and this makes it a bit easier to read the code for this
system.
2016-10-20 22:57:45 -07:00