How to set docker registry mirror with --buildkitd-flags for builder

[OCram85]

I'm currently contributing to a Woodpecker CI plugin for building docker images. I'm focused on using it in an enterprise environment behind a http proxy.
I managed to add missing args like --driver-opt , but now ran into an issue where buildkit or the builder doesn't use the internal mirror from the docker daemon.

I found some issues and discussions about the --buildkitd-flags. Is there any way to set an registry mirror with this flag?

So i could also implement the flag into the plugin step which creates the builder like this:

/usr/local/bin/docker buildx create --use --driver-opt env.http_proxy=http://swarmproxy:8888 --driver-opt env.https_proxy=http://swarmproxy:8888 --driver-opt env.no_proxy=ocram85.com --buildkitd-flags <registry mirror config>

As workaround I have to change all the FROM statements in the Dockerfiles and prepend the mirror url like this:

FROM mirror.ocram85.com/node:18.18-slim as node
# instead of
# FROM node:18.18-slim as node

[crazy-max]

Is there any way to set an registry mirror with this flag?

Registry mirror can be set through the BuildKit configuration: https://docs.docker.com/build/buildkit/configure/#registry-mirror

[OCram85]

Awesome,
I've added the BuildKit config like show in the docs example and it seems to work now. I see the image in the mirror store but the buildx output confuses me now:

#8 [base 1/7] FROM docker.io/library/ubuntu:jammy-20231004@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f
#8 resolve docker.io/library/ubuntu:jammy-20231004@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f 0.1s done
#8 sha256:aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e 18.87MB / 29.54MB 0.2s
#8 sha256:aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e 29.54MB / 29.54MB 0.3s done
#8 DONE 0.5s

The download seems to work I'm not sure if the parsed FROM statement docker.io/library/ubuntu:jammy-20231004 means it's downloading it from docker.io or the given mirror.

[crazy-max]

I'm not sure if the parsed FROM statement docker.io/library/ubuntu:jammy-20231004 means it's downloading it from docker.io or the given mirror.

Log output is not altered with the registry mirror configuration. If you want to make sure, you can check the BuildKit logs as shown in https://docs.docker.com/build/buildkit/configure/#registry-mirror. Otoh that might be something we could indicate in the log output as well in the future.

[OCram85]

Thanks for the help. So then I have to find a way getting the logs after running the build command. Thus the plugin spawns a docker-dind instance + a buildkit builder, it's really hard to get the logs afterwards, because they will be deleted after the run.

[sitsofe]

In case anyone stumbles across this: as alluded to, you can't set a docker builder mirror via --buildkitd-flags - you must set it in a configuration file that is then loaded using --buildkitd-config (as mentioned in https://docs.docker.com/build/buildkit/configure/#registry-mirror). Even though it is possible for TOML to be squeezed into one line via inline tables, for this particular situation there lies madness because --buildkitd-flags is only for setting flags that the buildkitd lists on its command line.

[fenollp]

@crazy-max Any plans on allowing to pass these flags without having to write a file to disk?

It's a bit weird having to pass args through via a file on disk: https://github.com/docker/actions-toolkit/blob/779136b849fb01373e53c81156c2fcbdc2f7b95b/src/buildkit/config.ts#L38