内容纲要
概要描述
本文描述容器重启挂载失败报错的排查思路,以及解决方案:
error: Put "http://warpdrive-manager-service:9500/v1/volumes?action=update": EOF
unable to attach or mount volumes: unmounted volumes=[mp-user-local-tomcat-logs mp-usr-local-tomcat-webapps], unattached volumes=[default-token-8697s mp-usr-local-tomcat-logs mp-usr-local-tomcat-webapps]: timed out waiting for the condition
详细描述
问题现象
TDC3.0.0 环境,pod处于containercreating状态
kubectl describe pod xxxx -n yyy
查看pod的events 报错
error: Put "http://warpdrive-manager-service:9500/v1/volumes?action=update": EOF
排查思路
1 检查 warpdrive-operator 的pod状态
kubectl get pod -A | grep warpdrive-operator
2 查看cert状态
kubectl get pod -A | grep cert
3 输出caBundle 和ca.crt内容进行比对
kubectl get mutatingwebhookconfiguration warpdrive-operator-mutating-webhook-configuration -oyaml | grep caBundle
kubectl get secret warpdrive-operator-webhook-server-cert -nkube-system -oyaml | grep ca.crt
对比发现签发webhook的ca和secret的ca不一致
4 解决方案
kubectl edit mutatingwebhookconfiguration warpdrive-operator-mutating-webhook-configuration
kubectl edit validatingwebhookconfiguration warpdrive-operator-validating-webhook-configuration
把所有的caBundle字段,都替换成以下命令输出的 ca.crt 对应的字段:
kubectl get secret warpdrive-operator-webhook-server-cert -nkube-system -oyaml | grep ca.crt
重启 cert-manager pod