Mirror
/
zulip
mirror of https://github.com/zulip/zulip.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zulip/tools/circleci/Dockerfile.template

107 lines
4.7 KiB
Docker
Raw Normal View History

circleci: Create script for generating Dockerfiles. [greg: updated Dockerfile comment]
2018-01-30 20:17:30 +01:00
# Dockerfile for a generic Ubuntu image with just the basics we need
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
# to make it suitable for CI. In particular:
# * a non-root user to run as (a pain to try to do in setup,
# because by then we've already cloned the repo);
# * Git and other basic utilities.
#
# Based on CircleCI's provided images, but those are on Debian Jessie
# and we want Ubuntu, to match our supported environments in production.
# See these templates and code:
# https://github.com/circleci/circleci-images/blob/1949c44df/shared/images/
# which we've borrowed from, chiefly Dockerfile-basic.template.
#
# The CircleCI `python` images are based on upstream's `python` (i.e.,
# https://hub.docker.com/_/python/), which also come only for Debian
# (and various obscure distros, and Windows) and not Ubuntu. Those
# are in turn based on upstream's `buildpack-deps`, which do come in
# Ubuntu flavors.
#
# So this image starts from `buildpack-deps`, does the job of `python`
# (but much simpler, with a couple of packages from the distro), and
# then borrows from the CircleCI Dockerfile.
circleci: Create script for generating Dockerfiles. [greg: updated Dockerfile comment]
2018-01-30 20:17:30 +01:00
# To rebuild from this file for a given release, say trusty:
# 0. $ tools/circleci/generate-dockerfiles
# 1. pick a new image name, like `gregprice/circleci:trusty-python-3.test`
# 2. $ sudo docker build tools/circleci/images/$RELEASE/ --tag $NAME
# 3. $ sudo docker push $NAME
# 4. update .circleci/config.yml to refer to the new name
circle-docker: Describe how to build the image. These commands are super boring standard Docker commands, so this probably isn't helpful for anyone who is familiar with building Docker images... but I had to consult docs to work out the right commands again today, so they'd help me.
2017-12-02 02:13:26 +01:00
circleci: Create script for generating Dockerfiles. [greg: updated Dockerfile comment]
2018-01-30 20:17:30 +01:00
FROM {base_image}
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
RUN echo 'APT::Get::Assume-Yes "true";' > /etc/apt/apt.conf.d/90circleci \
&& echo 'DPkg::Options "--force-confnew";' >> /etc/apt/apt.conf.d/90circleci
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install -y \
circleci: Generate xenial dockerfiles along with trusty.
2018-01-30 22:01:54 +01:00
sudo \
locales \
circle-docker: Use -scm variant of base image to save a bit of work.
2017-09-20 01:42:18 +02:00
xvfb \
circle-docker: Prune redundant packages from install list for clarity.
2017-09-20 01:55:30 +02:00
parallel \
netcat unzip zip jq \
python3-pip \
circle-docker: Simplify a bit for clarity and efficiency. Install `jq` with APT -- that's a lot simpler to read than this explicit download. And coalesce several commands, following Docker upstream's recommendation and avoiding unnecessary overhead.
2017-09-20 01:37:19 +02:00
&& ln -sf /usr/share/zoneinfo/Etc/UTC /etc/localtime \
circleci: Create script for generating Dockerfiles. [greg: updated Dockerfile comment]
2018-01-30 20:17:30 +01:00
&& {{ locale-gen en_US.UTF-8 || true; }} \
circleci: Set up locale en_US.UTF-8. See comment for motivation and what's known. [greg: Added comment, summarizing Vishnu's results from testing.]
2017-12-13 20:52:16 +01:00
&& echo "LC_ALL=en_US.UTF-8" | sudo tee -a /etc/default/locale
# Set the locale, together with the locale-related steps above.
# It's not entirely clear why, but alternatives that don't work include
# * using `C.UTF-8` instead of `en_US.UTF-8`, here and above
# (causes mysterious failures in zerver.tests.test_narrow)
# * skipping the /etc/default/locale step above (ditto)
# * skipping this ENV instruction (causes provision to fail,
# because Python tries to use the `ascii` codec)
# Details in https://github.com/zulip/zulip/pull/7762#issuecomment-353197289
# and particularly https://circleci.com/gh/hackerkid/zulip/80 .
ENV LC_ALL en_US.UTF-8
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
# Install Docker. This logic comes from Circle's Dockerfile; it's probably
# faster than the upstream-recommended approach of using their apt repo,
# and fine for an image that will be rebuilt rather than upgraded.
# Docker core...
circle-docker: Silence some spam output.
2017-12-02 01:35:42 +01:00
RUN set -e \
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
&& export DOCKER_VERSION=$(curl --silent --fail --retry 3 https://download.docker.com/linux/static/stable/x86_64/ | grep -o -e 'docker-[.0-9]*-ce\.tgz' | sort -r | head -n 1) \
circleci: Create script for generating Dockerfiles. [greg: updated Dockerfile comment]
2018-01-30 20:17:30 +01:00
&& DOCKER_URL="https://download.docker.com/linux/static/stable/x86_64/${{DOCKER_VERSION}}" \
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
&& echo Docker URL: $DOCKER_URL \
circleci: Create script for generating Dockerfiles. [greg: updated Dockerfile comment]
2018-01-30 20:17:30 +01:00
&& curl --silent --show-error --location --fail --retry 3 --output /tmp/docker.tgz "${{DOCKER_URL}}" \
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
&& ls -lha /tmp/docker.tgz \
&& tar -xz -C /tmp -f /tmp/docker.tgz \
&& mv /tmp/docker/* /usr/bin \
&& rm -rf /tmp/docker /tmp/docker.tgz \
&& which docker \
circle-docker: Silence some spam output.
2017-12-02 01:35:42 +01:00
&& (docker version 2>/dev/null || true)
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
# ...docker-compose...
RUN COMPOSE_URL="https://circle-downloads.s3.amazonaws.com/circleci-images/cache/linux-amd64/docker-compose-latest" \
&& curl --silent --show-error --location --fail --retry 3 --output /usr/bin/docker-compose $COMPOSE_URL \
&& chmod +x /usr/bin/docker-compose \
&& docker-compose version
# ... and dockerize.
RUN DOCKERIZE_URL="https://circle-downloads.s3.amazonaws.com/circleci-images/cache/linux-amd64/dockerize-latest.tar.gz" \
&& curl --silent --show-error --location --fail --retry 3 --output /tmp/dockerize-linux-amd64.tar.gz $DOCKERIZE_URL \
&& tar -C /usr/local/bin -xzvf /tmp/dockerize-linux-amd64.tar.gz \
&& rm -rf /tmp/dockerize-linux-amd64.tar.gz \
&& dockerize --version
circle-docker: Add all the standard APT deps we provision. This is only an optimization -- if this list is missing anything, we'll get to it in `provision` in the actual build. That's important, because we want an existing image to work fine for testing new versions of our codebase, including changes that may install more packages in `provision`. What this does accomplish is keeping provision's `apt-get install` fast, by leaving it very little work to do. The list comes from looking at the APT output during provision in an actual run without this step, and leaving out two packages which aren't available at this stage, because we get them from PPAs: postgresql-9.3-pgroonga and postgresql-9.3-tsearch-extras.
2017-09-20 03:37:30 +02:00
# Extra packages used by Zulip.
RUN apt-get update \
&& apt-get install --no-install-recommends \
closure-compiler memcached rabbitmq-server redis-server \
hunspell-en-us supervisor libssl-dev yui-compressor puppet \
deps: Replace libz-dev with zlib1g-dev since the former was renamed.
2018-02-09 22:28:20 +01:00
gettext libffi-dev libfreetype6-dev zlib1g-dev libjpeg-dev \
circle-docker: Add all the standard APT deps we provision. This is only an optimization -- if this list is missing anything, we'll get to it in `provision` in the actual build. That's important, because we want an existing image to work fine for testing new versions of our codebase, including changes that may install more packages in `provision`. What this does accomplish is keeping provision's `apt-get install` fast, by leaving it very little work to do. The list comes from looking at the APT output during provision in an actual run without this step, and leaving out two packages which aren't available at this stage, because we get them from PPAs: postgresql-9.3-pgroonga and postgresql-9.3-tsearch-extras.
2017-09-20 03:37:30 +02:00
libldap2-dev libmemcached-dev python-dev python-pip \
circleci: Generate xenial dockerfiles along with trusty.
2018-01-30 22:01:54 +01:00
python-six libxml2-dev libxslt1-dev libpq-dev \
{extra_packages}
circle-docker: Add all the standard APT deps we provision. This is only an optimization -- if this list is missing anything, we'll get to it in `provision` in the actual build. That's important, because we want an existing image to work fine for testing new versions of our codebase, including changes that may install more packages in `provision`. What this does accomplish is keeping provision's `apt-get install` fast, by leaving it very little work to do. The list comes from looking at the APT output during provision in an actual run without this step, and leaving out two packages which aren't available at this stage, because we get them from PPAs: postgresql-9.3-pgroonga and postgresql-9.3-tsearch-extras.
2017-09-20 03:37:30 +02:00
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
RUN groupadd --gid 3434 circleci \
&& useradd --uid 3434 --gid circleci --shell /bin/bash --create-home circleci \
circle-docker: Fix getting a sudo prompt. Provision was failing at tools/setup/postgres-init-dev-db with this in the log: Sep 20 02:27:01 + sudo -i -u postgres psql '' [sudo] password for circleci: The issue is that the old version of this line (from Circle upstream) only lets the `circleci` user sudo to root -- not to other users, or not directly anyway -- because sudoers syntax is complicated. Fix it, after studying `man sudoers`.
2017-09-23 00:00:21 +02:00
&& echo 'circleci ALL = (ALL) NOPASSWD: ALL' >> /etc/sudoers.d/50-circleci \
circle: First, near-upstream cut at a Dockerfile. This is nearly the same as Circle's version, linked in the comment. I've * changed the FROM line to get Ubuntu, * added a couple of distro packages to compensate, and * revised the comments.
2017-09-20 01:36:00 +02:00
&& echo 'Defaults env_keep += "DEBIAN_FRONTEND"' >> /etc/sudoers.d/env_keep
USER circleci
CMD ["/bin/sh"]