Incident began at 2021-09-23 18:14 (all times are US/Pacific).
Summary: Specific GKE Clusters upgraded from 1.19 to 1.20 (lower than 1.20.9-gke.2100) may experience pods restarting when Docker restarts.
Description: We are experiencing an issue with GKE clusters upgraded from 1.19 to 1.20(lower than 1.20.9-gke.2100). Clusters may see pods restarting when Dockers are restarted.
Problematic versions:
- All 1.20.0.* – 1.20.8.*
- 1.20.9.* below 1.20.9-gke.1600
Action required:
Do not upgrade from 1.19 -> 1.20.x (lower than 1.20.9-gke.2100)
To mitigate a cluster already upgraded to a problematic version, upgrade master and nodes to 1.20.10-gke.301.
Available versions with fix:
- 1.20.9-gke.210
- 1.20.10+
Our engineering team continues to investigate the issue at the back end.
We will provide an update by Thursday, 2021-09-23 22:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers may see GKE Cluster pods restarting when Docker restarts.
Workaround: Do not upgrade from 1.19 -> 1.20.x (lower than 1.20.9-gke.2100)
To mitigate a cluster already upgraded, to a problematic version, upgrade master and nodes to 1.20.10-gke.301. Versions above 1.20.9-gke.2100 is not impacted.
Affected products: Google Kubernetes Engine