How to quickly restore a dockerfile using a cache?

Question

I want to optimize my Docker file. And I want to save the cache file to disk. But I found that when starting docker build . He always tries to get every file from the network.

I want to split my cached directory during build (e.g.. / Var / cache / yum / x86_64 / 6). But it only works on docker run -v ...

Any suggestion? (In this example, only 1 rpm is set, in the real case I need to install hundreds of rpms)

My Dockerfile Project

 FROM centos:6.4 RUN yum update -y RUN yum install -y openssh-server RUN sed -i -e 's:keepcache=0:keepcache=1:' /etc/yum.conf VOLUME ["/var/cache/yum/x86_64/6"] EXPOSE 22 

The second time I want to create a similar image

 FROM centos:6.4 RUN yum update -y RUN yum install -y openssh-server vim 

I don't want to fetch openssh-server from the inside again (it's slow). In my real case, this is not one package, it is about 100 packages.

Answer 1

Updating previous answers, the current build docker accepts --build-arg , which pass environment variables like http_proxy without saving in the resulting image.

Example:

 # get squid docker run --name squid -d --restart=always \ --publish 3128:3128 \ --volume /var/spool/squid3 \ sameersbn/squid:3.3.8-11 # optionally in another terminal run tail on logs docker exec -it squid tail -f /var/log/squid3/access.log # get squid ip to use in docker build SQUID_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' squid) # build your instance docker build --build-arg http_proxy=http://$SQUID_IP:3128 . 

Answer 2

You should use a caching proxy (fe Http Replicator , squid-deb-proxy ...) or apt-cacher-ng for Ubuntu to cache installation packages. I think you can install this software on the host machine.

EDIT:

Option 1 - http proxy caching is a simpler method with a modified Docker file:

 > cd ~/your-project > git clone https://github.com/gertjanvanzwieten/replicator.git > mkdir cache > replicator/http-replicator -r ./cache -p 8080 --daemon ./cache/replicator.log --static 

add to your Docker file (before the first RUN line):

 ENV http_proxy http://172.17.42.1:8080/ 

You should clear the cache from time to time.

Option 2 - transparent proxy caching, without modifying the Docker file:

 > cd ~/your-project > curl -o r.zip https://codeload.github.com/zahradil/replicator/zip/transparent-requests > unzip r.zip > rm r.zip > mv replicator-transparent-requests replicator > mkdir cache > replicator/http-replicator -r ./cache -p 8080 --daemon ./cache/replicator.log --static 

You need to run the replicator as a user (not root!).

Set up transparent redirection:

 > iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner <replicator-user> --dport 80 -j REDIRECT --to-port 8080 

Disable redirection:

 > iptables -t nat -D OUTPUT -p tcp -m owner ! --uid-owner <replicator-user> --dport 80 -j REDIRECT --to-port 8080 

This method is the most transparent and general, and your Dockerfile does not need to be changed. You should clear the cache from time to time.

Answer 3

Just use the intermediate / base image:

Base Dockerfile, build it with docker build -t custom-base or something:

 FROM centos:6.4 RUN yum update -y RUN yum install -y openssh-server vim RUN sed -i -e 's:keepcache=0:keepcache=1:' /etc/yum.conf 

Dockerfile app:

 FROM custom-base VOLUME ["/var/cache/yum/x86_64/6"] EXPOSE 22