问题现象
master节点显示NotReady,不可调度,原本在master节点上运行的corecdn,各类DeamonSet服务都显示异常,查看kubelet服务发现启动报错.
通过命令journalctl -u kubelet -f查看kubelet启动报错的详细日志,发现主要原因是找不到文件/etc/kubernetes/bootstrap-kubelet.conf.
解决
- 从别的节点拷入
/etc/kubernetes/bootstrap-kubelet.conf到master节点. - 此时kubelet可以启动但是 任然报错
Aug 11 11:18:06 master kubelet[1757]: E0811 11:18:06.553681 1757 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Aug 11 11:18:06 master kubelet[1757]: E0811 11:18:06.631490 1757 kubelet.go:2248] node "master" not found
Aug 11 11:18:06 master kubelet[1757]: E0811 11:18:06.731614 1757 kubelet.go:2248] node "master" not found
Aug 11 11:18:06 master kubelet[1757]: E0811 11:18:06.753619 1757 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Aug 11 11:18:06 master kubelet[1757]: E0811 11:18:06.831748 1757 kubelet.go:2248] node "master" not found
Aug 11 11:18:06 master kubelet[1757]: E0811 11:18:06.931869 1757 kubelet.go:2248] node "master" not found
Aug 11 11:18:06 master kubelet[1757]: E0811 11:18:06.954503 1757 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
- 给匿名用户赋予权限
kubectl create clusterrolebinding test:anonymous --clusterrole=cluster-admin --user=system:anonymous