How do I force Kubernetes to re-pull an image?

I have the following replication controller in Kubernetes on GKE:

apiVersion: v1
kind: ReplicationController
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  replicas: 2
  selector:
    app: myapp
    deployment: initial
  template:
    metadata:
      labels:
        app: myapp
        deployment: initial
    spec:
      containers:
      - name: myapp
        image: myregistry.com/myapp:5c3dda6b
        ports:
        - containerPort: 80
      imagePullPolicy: Always
      imagePullSecrets:
        - name: myregistry.com-registry-key

Now, if I say

kubectl rolling-update myapp --image=us.gcr.io/project-107012/myapp:5c3dda6b

the rolling update is performed, but no re-pull. Why?

Kubernetes will pull upon Pod creation if either (see updating-images doc):

• Using images tagged :latest
• imagePullPolicy: Always is specified

This is great if you want to always pull. But what if you want to do it on demand: For example, if you want to use some-public-image:latest but only want to pull a newer version manually when you ask for it. You can currently:

• Set imagePullPolicy to IfNotPresent or Never and pre-pull: Pull manually images on each cluster node so the latest is cached, then do a kubectl rolling-update or similar to restart Pods (ugly easily broken hack!)
• Temporarily change imagePullPolicy, do a kubectl apply, restart the pod (e.g. kubectl rolling-update), revert imagePullPolicy, redo a kubectl apply (ugly!)
• Pull and push some-public-image:latest to your private repository and do a kubectl rolling-update (heavy!)

No good solution for on-demand pull. If that changes, please comment; I'll update this answer.

One has to group imagePullPolicy inside the container data instead of inside the spec data. However, I filed an [issue][1] about this because I find it odd. Besides, there is no error message.

So, this spec snippet works:

spec:
  containers:
  - name: myapp
    image: myregistry.com/myapp:5c3dda6b
    ports:
    - containerPort: 80
    imagePullPolicy: Always
  imagePullSecrets:
    - name: myregistry.com-registry-key

[1]: https://github.com/kubernetes/kubernetes/issues/15609 "issue"

There is a comand to directly do that:

Create a new kubectl rollout restart command that does a rolling restart of a deployment.

The pull request got merged. It is part of the version 1.15 (changelog) or higher.

My hack during development is to change my Deployment manifest to add the latest tag and always pull like so

image: etoews/my-image:latest
imagePullPolicy: Always

Then I delete the pod manually

kubectl delete pod my-app-3498980157-2zxhd

Because it's a Deployment, Kubernetes will automatically recreate the pod and pull the latest image.

A popular workaround is to patch the deployment with a dummy annotation (or label):

kubectl patch deployment <name> -p \
  "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"date\":\"`date +'%s'`\"}}}}}"

Assuming your deployment meets these requirements, this will cause K8s to pull any new image and redeploy.

Now, the command kubectl rollout restart deploy YOUR-DEPLOYMENT combined with a imagePullPolicy: Always policy will allow you to restart all your pods with a latest version of your image.

1. Specify strategy as:

  strategy: 
    type: Recreate
    rollingUpdate: null

2. Make sure you have different annotation for each deployment. Helm does it like:

  template:
    metadata:
      labels:
        app.kubernetes.io/name: AppName
        app.kubernetes.io/instance: ReleaseName
      annotations:
        rollme: {{ randAlphaNum 5 | quote }}

3. Specify image pull policy - Always

      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: Always

# Linux

kubectl patch deployment <name> -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"date\":\"`date +'%s'`\"}}}}}"

# windows

kubectl patch deployment <name> -p (-join("{\""spec\"":{\""template\"":{\""metadata\"":{\""annotations\"":{\""date\"":\""" , $(Get-Date -Format o).replace(':','-').replace('+','_') , "\""}}}}}"))

This answer aims to force an image pull in a situation where your node has already downloaded an image with the same name, therefore even though you push a new image to container registry, when you spin up some pods, your pod says "image already present".

For a case in Azure Container Registry (probably AWS and GCP also provides this):

1. You can look to your Azure Container Registry and by checking the manifest creation date you can identify what image is the most recent one.

2. Then, copy its digest hash (which has a format of sha256:xxx...xxx).

3. You can scale down your current replica by running command below. Note that this will obviously stop your container and cause downtime.

kubectl scale --replicas=0 deployment <deployment-name> -n <namespace-name>

4. Then you can get the copy of the deployment.yaml by running:

kubectl get deployments.apps <deployment-name> -o yaml > deployment.yaml

5. Then change the line with image field from <image-name>:<tag> to <image-name>@sha256:xxx...xxx, save the file.

6. Now you can scale up your replicas again. New image will be pulled with its unique digest.

Note: It is assumed that, imagePullPolicy: Always field is present in the container.

Apparently now when you run a rolling-update with the --image argument the same as the existing container image, you must also specify an --image-pull-policy. The following command should force a pull of the image when it is the same as the container image:

kubectl rolling-update myapp --image=us.gcr.io/project-107012/myapp:5c3dda6b --image-pull-policy Always

Having gone through all the other answers and not being satisfied, I found much better solution here: https://cloud.google.com/kubernetes-engine/docs/how-to/updating-apps

It works without using latest tag or imagePullPolicy: Always. It also works if you push new image to the same tag by specifying image sha256 digest.

Steps:

1. get image SHA256 from docker hub (see image below)
2. kubectl set image deployment/<your-deployment> <your_container_name>=<some/image>@sha256:<your sha>
3. kubectl scale deployment <your-deployment>--replicas=0
4. kubectl scale deployment <your-deployment>--replicas=original replicas count

Note: Rollout might also work instead of scale but in my case we don't have enough hardware resources to create another instance and k8s gets stuck.

The rolling update command, when given an image argument, assumes that the image is different than what currently exists in the replication controller.

You can define imagePullPolicy: Always in your deployment file.

The Image pull policy will always actually help to pull the image every single time a new pod is created (this can be in any case like scaling the replicas, or pod dies and new pod is created)

But if you want to update the image of the current running pod, deployment is the best way. It leaves you flawless update without any problem (mainly when you have a persistent volume attached to the pod) :)

if you want to perform a direct image update on a specific pod, you can use kubectl set image also.

https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

either you have deleted all the pods manually to get it recreated with pulling the image again.

or

run this below command kubectl rollout restart deployment/deployment_name kubectl rollout restart deployment/nginx

this command should recreate all the pods.

for both scenarios imagepullPolicy should be set as Always.