Error response from daemon: No such image: quay.io/helmpack/chart-testing:v3.7.1

General
1

All of my jenkins job is failing at last step(docker rmi quay.io/helmpack/chart-testing:v3.7.1) which is clearing image with error message - “Error response from daemon: No such image: quay.io/helmpack/chart-testing:v3.7.1”. Does anyone know why I am getting this error?

Jenkinsfile

pipeline {
agent {
// Build global agent definition
label 'DOCKER-LINUX'
}
// Setting up parameters
parameters {
string(name: 'kubeconfigFile', defaultValue: '****',
description: 'Enter the kubeconfig jenkins credentialID, this set the k8s cluster:')
string(name: 'scmCredentials', defaultValue: '*****',
description: 'Enter the SCM jenkins credentialID')
string(name: 'ageKey', defaultValue: '****',
description: 'Enter the Jenkins credenitals id that contains the age key used to de-crypt the helm chart secrets. This needs to be the same key used to encrypted')
string(name: 'SERVICE_NAME', defaultValue: 'ml****',
description: 'Enter a service name e.g. ml****')
string(name: 'DOCKER_TAG', defaultValue: '1.8.129',
description: 'Enter a Docker Tag e.g. latest')
string(name: 'HELM_CHART_VERSION', defaultValue: '***',
description: 'Enter a valid Helm Chart version e.g. 0.1.0')
choice(choices: ['','dev', 'qa', 'test'],
description: 'Enter the ENVIRONMENT', name: 'ENVIRONMENT')
string(name: 'NAMESPACE', defaultValue: 'dev',
description: 'Enter a valid Kubernetes namespace e.g. dev')
string(name: 'helmCredentials', defaultValue: '******',
description: 'Enter name of the helm private respository Jenkins credentials')

}

//keeping last 5 build log and artifacts
options { buildDiscarder(logRotator(numToKeepStr: "5", artifactNumToKeepStr: "5")) }

stages
{
stage ("Install & Deploy")
{
agent {
// Build stage agent definition
dockerfile {
args '-u root --dns 10.10.16.201 --dns 10.10.16.202 --dns-search corp.****.com'
filename 'dockerfile/Dockerfile'
label 'DOCKER-LINUX'
reuseNode true
}
}
when {
expression {
"${SERVICE_NAME}"!= '****' && "${SERVICE_NAME}"!= 'ml***'
}
}

steps
{
// Add repo
withCredentials([usernamePassword(credentialsId: "${params.helmCredentials}", passwordVariable: 'KEY', usernameVariable: 'USER'),
file(credentialsId: "${params.ageKey}", variable: 'ageKey'),
file(credentialsId: "${params.kubeconfigFile}", variable: 'KUBECONFIG')])
{

sh ('helm repo add helm-repository https://artifacts.corp.com/artifactory/****-helm-repository --username ${USER} --password ${KEY} --pass-credentials')
sh ('helm repo update')

// Copy the Age key
sh ('cp \$ageKey /tmp/.config/sops/age/keys.txt')

// create namespace
sh ('kubectl create namespace ml****\${ENVIRONMENT} > /dev/null 2>&1 || true')

//decrypt secret key
sh ('KUBECONFIG=\$KUBECONFIG helm secrets upgrade --install ${SERVICE_NAME} ***-helm-repository/${SERVICE_NAME} \
-f secrets/\${NAMESPACE}/${SERVICE_NAME}/secrets.yaml \
-f environments/\${NAMESPACE}/${SERVICE_NAME}/values.yaml \
--version="${HELM_CHART_VERSION}" --set image.tag="${DOCKER_TAG}" -n ${NAMESPACE} --username "${USER}" --debug --wait --timeout 20m0s')

}
}
}

stage ("Install & Deploy either cassandra or ml*** only")
{
agent {
// Build stage agent definition
dockerfile {
args '-u root --dns 10.10.16.201 --dns 10.10.16.202 --dns-search corp.****.com'
filename 'dockerfile/Dockerfile'
label 'DOCKER-LINUX'
reuseNode true
}
}
when {
expression {
"${SERVICE_NAME}" == '***' || "${SERVICE_NAME}" == 'ml***'
}
}

steps
{
// Add repo
withCredentials([usernamePassword(credentialsId: "${params.helmCredentials}", passwordVariable: 'KEY', usernameVariable: 'USER'),
file(credentialsId: "${params.ageKey}", variable: 'ageKey'),
file(credentialsId: "${params.kubeconfigFile}", variable: 'KUBECONFIG')])
{

sh ('helm repo add helm-repository https://artifacts.corp.com/artifactory/***-helm-repository --username ${USER} --password ${KEY} --pass-credentials')
sh ('helm repo update')

// Copy the Age key
sh ('cp \$ageKey /tmp/.config/sops/age/keys.txt')

// create namespace
sh ('kubectl create namespace ml-datapipeline-\${ENVIRONMENT} > /dev/null 2>&1 || true')

//decrypt secret key
sh ('KUBECONFIG=\$KUBECONFIG helm secrets upgrade --install ${SERVICE_NAME} ****-helm-repository/${SERVICE_NAME} \
-f secrets/\${NAMESPACE}/${SERVICE_NAME}/secrets.yaml \
-f environments/\${NAMESPACE}/${SERVICE_NAME}/values.yaml \
--version=${HELM_CHART_VERSION} --set image.tag=${DOCKER_TAG} -n ${NAMESPACE} --debug --username "${USER}" --wait --timeout 20m0s')

}
}
}

//Stage 2: Remove the Docker image from the jenkins box after build
stage('Remove Unused Docker Image') {
// Delete the daingling image
agent {
// stage agent
docker {
image "docker"
label 'DOCKER-LINUX'
reuseNode true
args '-u root -v /var/run/docker.sock:/var/run/docker.sock'
}
}
steps {
sh('docker rmi [quay.io/helmpack/chart-testing:v3.7.1](http://quay.io/helmpack/chart-testing:v3.7.1)')
}
}
}
post {
always {
cleanWs deleteDirs: true, notFailBuild: true
}
}
}

Dockerfile

FROM [quay.io/helmpack/chart-testing:v3.7.1](http://quay.io/helmpack/chart-testing:v3.7.1)
LABEL Name=Deployment Version=0.0.1

# Used to se the HELM working directory
ENV HOME=/tmp

# Install the necessary tools
RUN apk add go \
&& go install [filippo.io/age/cmd/...@latest](http://filippo.io/age/cmd/...@latest) \
&& wget -O /usr/local/bin/sops https://github.com/mozilla/sops/releases/download/v3.7.3/sops-v3.7.3.linux \
&& chmod 700 /usr/local/bin/sops \
&& mkdir -p /tmp/.config/sops/age \
&& helm plugin install https://github.com/jkroepke/helm-secrets --version v3.12.0 \
&& helm plugin install https://github.com/databus23/helm-diff --version v3.5.0

JekinsErrorlog

21:55:47 kubectl --namespace dev port-forward $POD_NAME 8080:$CONTAINER_PORT
21:55:47
21:55:47 [helm-secrets] Removed: secrets/dev/uibackend/secrets.yaml.dec
21:55:47 [Pipeline] }
21:55:47 [Pipeline] // withCredentials
21:55:47 [Pipeline] }
21:55:22 $ docker stop --time=1 97be5aa5efe1ccca00d88400b546cf4b357181c2573d0ed72f82623622568822
21:55:27 $ docker rm -f --volumes 97be5aa5efe1ccca00d88400b546cf4b357181c2573d0ed72f82623622568822
21:55:53 [Pipeline] // withDockerContainer
21:55:53 [Pipeline] }
21:55:53 [Pipeline] // stage
21:55:53 [Pipeline] stage
21:55:53 [Pipeline] { (Install & Deploy either cassandra or ml*** only)
21:55:53 [Pipeline] getContext
21:55:53 [Pipeline] isUnix
21:55:53 [Pipeline] readFile
21:55:53 [Pipeline] sh
21:55:53 + docker build -t 0b9765db7e81162633c032e492e32c24e8073bd0 -f dockerfile/Dockerfile .
21:55:54 #1 [internal] load .dockerignore
21:55:54 #1 transferring context: 2B done
21:55:54 #1 DONE 0.0s
21:55:54
21:55:54 #2 [internal] load build definition from Dockerfile
21:55:54 #2 transferring dockerfile: 620B done
21:55:54 #2 DONE 0.0s
21:55:54
21:55:54 #3 [internal] load metadata for [quay.io/helmpack/chart-testing:v3.7.1](http://quay.io/helmpack/chart-testing:v3.7.1)
21:55:54 #3 DONE 0.2s
21:55:54
21:55:54 #4 [1/2] FROM [quay.io/helmpack/chart-testing:v3.7.1@sha256:ef453de0be68d5ded26f3b3ea0c5431b396c8c48f89e2a07be7b19c4c9a68b31](http://quay.io/helmpack/chart-testing:v3.7.1@sha256:ef453de0be68d5ded26f3b3ea0c5431b396c8c48f89e2a07be7b19c4c9a68b31)
21:55:54 #4 DONE 0.0s
21:55:54
21:55:54 #5 [2/2] RUN apk add go && go install [filippo.io/age/cmd/...@latest](http://filippo.io/age/cmd/...@latest) && wget -O /usr/local/bin/sops https://github.com/mozilla/sops/releases/download/v3.7.3/sops-v3.7.3.linux && chmod 700 /usr/local/bin/sops && mkdir -p /tmp/.config/sops/age && helm plugin install https://github.com/jkroepke/helm-secrets --version v3.12.0 && helm plugin install https://github.com/databus23/helm-diff --version v3.5.0
21:55:54 #5 CACHED
21:55:54
21:55:54 #6 exporting to image
21:55:54 #6 exporting layers done
21:55:54 #6 writing image sha256:f3f6c79fd5ee2d941285c2e75932b37740c6fcbaa7ab47ae64b59c82ed0fa9ab done
21:55:54 #6 naming to [docker.io/library/0b9765db7e81162633c032e492e32c24e8073bd0](http://docker.io/library/0b9765db7e81162633c032e492e32c24e8073bd0) done
21:55:54 #6 DONE 0.0s
21:55:54 [Pipeline] isUnix
21:55:54 [Pipeline] withEnv
21:55:54 [Pipeline] {
21:55:54 [Pipeline] sh
21:55:54 + docker inspect -f . 0b9765db7e81162633c032e492e32c24e8073bd0
21:55:54 .
21:55:54 [Pipeline] }
21:55:54 [Pipeline] // withEnv
21:55:54 [Pipeline] withDockerContainer
21:55:55 gpu-envy does not seem to be running inside a container
21:55:29 $ docker run -t -d -u 1000:1000 -u root --dns 10.10.16.201 --dns 10.10.16.202 --dns-search [corp.com](http://corp.com/) -w /ssd2/jenkins/workspace/Penni/CICD/deployment-v2 -v /ssd2/jenkins/workspace/Penni/CICD/deployment-v2:/ssd2/jenkins/workspace/Penni/CICD/deployment-v2:rw,z -v /ssd2/jenkins/workspace/Penni/CICD/deployment-v2@tmp:/ssd2/jenkins/workspace/Penni/CICD/deployment-v2@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** 0b9765db7e81162633c032e492e32c24e8073bd0 cat
21:55:30 $ docker top 397544ec68070b2168327dde20b492f0e2cda6d79e9136682109130cd47dd635 -eo pid,comm
21:55:55 [Pipeline] {
21:55:55 [Pipeline] }
21:55:30 $ docker stop --time=1 397544ec68070b2168327dde20b492f0e2cda6d79e9136682109130cd47dd635
21:55:31 $ docker rm -f --volumes 397544ec68070b2168327dde20b492f0e2cda6d79e9136682109130cd47dd635
21:55:57 [Pipeline] // withDockerContainer
21:55:57 Stage "Install & Deploy either cassandra or ml*** only" skipped due to when conditional
21:55:57 [Pipeline] }
21:55:57 [Pipeline] // stage
21:55:57 [Pipeline] stage
21:55:57 [Pipeline] { (Remove Unused Docker Image)
21:55:57 [Pipeline] getContext
21:55:57 [Pipeline] isUnix
21:55:57 [Pipeline] withEnv
21:55:57 [Pipeline] {
21:55:57 [Pipeline] sh
21:55:58 + docker inspect -f . docker
21:55:58 .
21:55:58 [Pipeline] }
21:55:58 [Pipeline] // withEnv
21:55:58 [Pipeline] withDockerContainer
21:55:58 gpu-envy does not seem to be running inside a container
21:55:32 $ docker run -t -d -u 1000:1000 -u root -v /var/run/docker.sock:/var/run/docker.sock -w /ssd2/jenkins/workspace/Penni/CICD/deployment-v2 -v /ssd2/jenkins/workspace/Penni/CICD/deployment-v2:/ssd2/jenkins/workspace/Penni/CICD/deployment-v2:rw,z -v /ssd2/jenkins/workspace/Penni/CICD/deployment-v2@tmp:/ssd2/jenkins/workspace/Penni/CICD/deployment-v2@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** docker cat
21:55:33 $ docker top 18e413b28cf4158d85cb88fc5a63a1999244f24b128443508d8787d37f4bc350 -eo pid,comm
21:55:59 [Pipeline] {
21:55:59 [Pipeline] sh
21:55:59 + docker rmi [quay.io/helmpack/chart-testing:v3.7.1](http://quay.io/helmpack/chart-testing:v3.7.1)
21:55:59 Error response from daemon: No such image: [quay.io/helmpack/chart-testing:v3.7.1](http://quay.io/helmpack/chart-testing:v3.7.1)
21:55:59 [Pipeline] }
21:55:34 $ docker stop --time=1 18e413b28cf4158d85cb88fc5a63a1999244f24b128443508d8787d37f4bc350
21:55:35 $ docker rm -f --volumes 18e413b28cf4158d85cb88fc5a63a1999244f24b128443508d8787d37f4bc350
21:56:01 [Pipeline] // withDockerContainer
21:56:01 [Pipeline] }
21:56:01 [Pipeline] // stage
21:56:01 [Pipeline] stage
21:56:01 [Pipeline] { (Declarative: Post Actions)
21:56:01 [Pipeline] cleanWs
21:56:01 [WS-CLEANUP] Deleting project workspace...
21:56:01 [WS-CLEANUP] Deferred wipeout is used...
21:56:01 [WS-CLEANUP] done
21:56:01 [Pipeline] }
21:56:01 [Pipeline] // stage
21:56:01 [Pipeline] }
21:56:01 [Pipeline] // withEnv
21:56:01 [Pipeline] }
21:56:01 [Pipeline] // node
21:56:01 [Pipeline] End of Pipeline
21:56:01 ERROR: script returned exit code 1
21:56:01 Finished: FAILURE
2

What is your docker version?

All I can see is that there is no size information next in the line of v3.7.1, but I can pull it on my machine

Can you try with v3.7.1-amd64 to see if it matters?

3

Hi @rimelek
Thanks for your response. I tried with v3.7.1-amd64 but facing same issue again.
Any other suggestion?

4

Also, Docker version 23.0.5, build bc4487a being installed on our system.

5

I completely missed the fact that the error message is not because of pulling an image but because of trying to remove it. I am sorry, I must have been confused by other topics where the issue was pulling images.

In this case it doesn’t seem to be a docker issue. For some reason the docker image does not exist that you want to delete. Either because something else already deleted it or you just used it as a base image so the tag of the base image was not pulled at all, only the layers, so you can’t delete the tag.

Other reason could be (since you are using Kubernetes) that Kubernetes is not using Docker as container runtime and the images are not stored in Docker’s data directory but somewhere in /var/lib/containerd managed by containerd.

Based on the following line:

It seems you are using DOCKER_TAG as a helm chart parameter so I guess it is not just a base image in a Dockerfile.

Note that Kubernetes does not require Docker to be installed and the default container runtime is containerd, not Docker. If you are still using Docker as a container runtime, you have to find out what removes the image before docker rmi. You can also try to add a new step which lists available images and see if there are any.

If Kubernetes is using containerd as container runtime, you can use the ctr or crictl commands instead of docker