Introduction
I was having a private artifactory, and has my custom modules deployed there. I wanted to build a docker image, where I was using my private artifacts. I needed to pass credentials to build process, so that I could install those private artifacts.
I tried using ARG and ENV in Dockerfile, and it did successfully built. But, when I tried running docker image history --no-trunc <image>, I could see the credentials naked.
So, the goal is to build docker image, by passing secrets. But, not exposing it in docker image history.
I will describe two solutions:
- Multi-stage Docker build
- Docker buildkit
Example Which Exposes Secret in Docker Image History
FROM YOUR_IMAGE:1.0 as intermediate
WORKDIR /var/www/html/
COPY . /var/www/html/
ARG ARTIFACTORY_USER
ARG ARTIFACTORY_TOKEN
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& python3.9 -m pip install -r /var/www/html/requirements.txt -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple \
&& python3.9 -m pip install mod_wsgi -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& mod_wsgi-express install-module
COPY ./docker/httpd.conf /etc/httpd/conf/httpd.conf
COPY ./docker/sites-enabled /etc/httpd/sites-enabled
ENV CONFIG "/var/www/html"
EXPOSE 8080
USER root
CMD ["/usr/sbin/httpd","-D","FOREGROUND"]Now build and run docker image history --no-trunc <YOUR_IMAGE>
You could see somewhere in between the step and which exposes secrets in plain text:
<missing> 13 seconds ago RUN |2 ARTIFACTORY_USER=USER ARTIFACTORY_TOKEN=TOKEN /bin/sh -c virtualenv env --python python3.9 && source /var/www/html/env/bin/activate && python3.9 -m pip install -r /var/www/html/requirements.txt -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple && python3.9 -m pip install mod_wsgi -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple # buildkit 102MB buildkit.dockerfile.v0
<missing> 13 seconds ago ARG ARTIFACTORY_TOKEN Multi-stage Docker Build
Lets have a look at one of example I made,
FROM YOUR_IMAGE:1.0 as intermediate
WORKDIR /var/www/html/
COPY . /var/www/html/
ARG ARTIFACTORY_USER
ARG ARTIFACTORY_TOKEN
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& python3.9 -m pip install -r /var/www/html/requirements.txt -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/pypi-MY-release/simple \
&& python3.9 -m pip install mod_wsgi -i https://${ARTIFACTORY_USER}:${ARTIFACTORY_TOKEN}@artifactory.MY.com/artifactory/api/pypi/MY/simple
FROM YOUR_IMAGE:1.0
WORKDIR /var/www/html/
COPY /var/www/html /var/www/html
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& mod_wsgi-express install-module
COPY ./docker/httpd.conf /etc/httpd/conf/httpd.conf
COPY ./docker/sites-enabled /etc/httpd/sites-enabled
ENV CONFIG "/var/www/html"
EXPOSE 8080
USER root
CMD ["/usr/sbin/httpd","-D","FOREGROUND"]Notice the two FROM keyword in above Dockerfile. The command to build this is:
docker build -t data-ws --build-arg ARTIFACTORY_USER=<USER> --build-arg ARTIFACTORY_TOKEN=<ARTIFACTORY_TOKEN> .I’m passing two arguments to dockerfile: ARTIFACTORY_USER, and ARTIFACTORY_TOKEN.
Now, if you do: docker image history --no-trunc <image>. You will not see the first stage command history. And, you are able to successfully hide your secrets from the build process.
Buildkit from Docker
In this solution, I will pass complete pip.conf file to docker build process. And, I will copy that file inside image before running pip install. Also I need to set PIP_CONFIG_FILE environment variable to set path of custom pip.conf file.
FROM YOUR_IMAGE:1.0
WORKDIR /var/www/html/
COPY . /var/www/html/data
ENV PIP_CONFIG_FILE /root/.pip.conf
RUN virtualenv env --python python3.9 \
&& source /var/www/html/env/bin/activate \
&& python3.9 -m pip install -r /var/www/html/data/requirements.txt \
&& python3.9 -m pip install mod_wsgi \
&& mod_wsgi-express install-module
COPY ./docker/httpd.conf /etc/httpd/conf/httpd.conf
COPY ./docker/sites-enabled /etc/httpd/sites-enabled
EXPOSE 80
USER root
CMD ["/usr/sbin/httpd","-D","FOREGROUND"]Build command:
docker build -f Dockerfile --secret id=pip.conf,src=/path/to/pip.conf -t MY_IMAGE .Note the --mount=type=secret,id=pip.conf,dst=/root/.pip.conf.
From the build command, I’m passing source file information in src, and I’m setting its id as id.
In the Dockerfile, I’m receiving it in same id. And, copying it with dst path inside the docker image.
Note, the file used in dst will not be copied in the image produced. The magic :)
It will not be present even in the intermediate layers.
Hope you enjoy the post.
$ subscribe --weekly
One deep dive a week, in your inbox
System design, AI engineering, trading systems and interview prep. Tested code, real numbers. Free, and you can unsubscribe any time.
or read it on gyanbyte.substack.com
