Recently CI/CD Pipelines managed by me in GitLab started failing one by one. In logs, it said:
ERROR: Preparation failed: Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit (docker.go:142:1s)I went to the Docker website link provided in error to read more. As it turns out Docker Inc changed its policy on November 20, 2020. Now Anonymous and Free Docker Hub users are limited to 100 and 200 container image pull requests per six hours. It meant I had to migrate CI/CD images to our hosted GitLab or create Docker Pro or Team account.
I decided to go for the first option. After pulling locally all necessary images I pushed them to our GitLab Docker Registry.
Next, I had to change CI/CD definitions. I modified .gitlab-ci.yml files. Every image directive has changed to:
image: "registry.example.com/my/image:latest"Now build docker images are pulled from our private Docker Registry. But for this to work we need to finish the last step which is configuring the access to the private registry. There are 3 options to do that:
- DOCKER_AUTH_CONFIG variable provided as either:
1. A variable in .gitlab-ci.yml.
2. A project’s variables are stored on the project's Settings > CI/CD page. -
DOCKER_AUTH_CONFIG variable provided as an environment variable in config.toml of the runner.
- config.json file placed in the $HOME/.docker directory of the user running the GitLab Runner process. If the --user flag is provided to run the GitLab Runner child processes as an unprivileged user, the home directory of the main GitLab Runner process user is used.
Since I decided to expand this approach to every repository and pipeline in the future I chose the second option. We need to set the Docker Authentication string in the config.toml of GitLab Runner. Just log in to docker with the terminal and user you want to use. After the login is successful you can cat $HOME/.docker/config.json content and escape lines and double-quotes. Then edit the GitLab Runner config.toml file and add the following variable:
[[runners]]
environment = ["DOCKER_AUTH_CONFIG={"auths":{"registry.example.com:5000":{"auth":"auttoken"}}}"]Restart the GitLab Runner and now your pipelines will pull build images from your private repositories and you don't need to pay Docker Inc for free, open-source stuff. I understand they introduced this functionality mostly for security purposes, but on the other hand, we had to expect new monetization techniques after Mirantis acquired Docker Inc in 2019.