天际边 發表於 2020-1-21 15:55:00

kubernetes集群证书更新

<p><strong>kubeadm 默认证书为一年,一年过期后,会导致api service不可用,使用过程中会出现:x509: certificate has expired or is not yet valid.</strong></p>
<h3 id="方案一-通过修改kubeadm-调整证书过期时间">方案一 通过修改kubeadm 调整证书过期时间</h3>
<h4 id="修改代码,调整过期时间">修改代码,调整过期时间</h4>
<h3>方案二 通过自动轮换证书默认开启</h3>
<h1>以下方案通过第二种方法模拟集群证书过期</h1>
<h1><strong>准备</strong></h1>
<p><strong>本次集群版本1.15</strong></p>
<p><strong>备份集群证书(略)</strong></p>
<p><strong>cd&nbsp;/etc/kubernetes</strong></p>
<p><strong>tar czvf&nbsp;kubernetes.tar.gz&nbsp;kubernetes</strong></p>
<h1><strong>Master<span style="font-family: 宋体">节点:</span></strong></h1>
<div class="cnblogs_code">
<pre># hwclock --<span style="color: rgba(0, 0, 0, 1)">show

2020年01月21日 星期二 15时16分34秒</span>-<span style="color: rgba(128, 0, 128, 1)">0.856601</span><span style="color: rgba(0, 0, 0, 1)"> 秒

# kubectl get nodes

NAME         STATUS   ROLES    AGE    VERSION

k8s</span>-master   Ready    master   167d   v1.<span style="color: rgba(128, 0, 128, 1)">15.0</span><span style="color: rgba(0, 0, 0, 1)">

k8s</span>-node1    Ready    node   166d   v1.<span style="color: rgba(128, 0, 128, 1)">15.0</span><span style="color: rgba(0, 0, 0, 1)">

# kubeadm alpha certs check-<span style="color: rgba(0, 0, 0, 1)">expiration

CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED

admin.conf               Jan </span><span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

apiserver                  Jan </span><span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

apiserver</span>-etcd-client      Jan <span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

apiserver</span>-kubelet-client   Jan <span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

controller</span>-manager.conf    Jan <span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

etcd</span>-healthcheck-client    Jan <span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

etcd</span>-peer                  Jan <span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

etcd</span>-server                Jan <span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

front</span>-proxy-client         Jan <span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

scheduler.conf             Jan </span><span style="color: rgba(128, 0, 128, 1)">20</span>, <span style="color: rgba(128, 0, 128, 1)">2021</span> <span style="color: rgba(128, 0, 128, 1)">07</span>:<span style="color: rgba(128, 0, 128, 1)">09</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

#</pre>
</div>
<p>&nbsp;</p>
<h1><strong>先生成集群配置文件</strong></h1>
<p>kubeadm config view &gt; /root/kubeadm.yaml<span style="font-family: 宋体"> </span></p>
<p><span style="font-family: 宋体">要提前备份一下集群配置文件</span>,<span style="font-family: 宋体">当集群证书过期后 此命令也不能执行了</span></p>
<p>&nbsp;</p>
<h1><strong>修改时间让集群过期</strong></h1>
<div class="cnblogs_code">
<pre># <span style="color: rgba(0, 0, 255, 1)">date</span> -s <span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(128, 0, 0, 1)">2021-08-08</span><span style="color: rgba(128, 0, 0, 1)">"</span><span style="color: rgba(0, 0, 0, 1)">

2021年 08月 08日 星期日 </span><span style="color: rgba(128, 0, 128, 1)">00</span>:<span style="color: rgba(128, 0, 128, 1)">00</span>:<span style="color: rgba(128, 0, 128, 1)">00</span><span style="color: rgba(0, 0, 0, 1)"> CST

# <span style="color: rgba(0, 0, 255, 1)">date</span><span style="color: rgba(0, 0, 0, 1)">

2021年 08月 08日 星期日 </span><span style="color: rgba(128, 0, 128, 1)">00</span>:<span style="color: rgba(128, 0, 128, 1)">00</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> CST

# kubectl get nodes

Unable to connect to the server: x509: certificate has expired or is not yet valid

#</pre>
</div>
<p>&nbsp;</p>
<p>&nbsp;</p>
<h1><strong>更新证书</strong></h1>
<div class="cnblogs_code">
<pre># kubeadm alpha certs renew all --config=/root/<span style="color: rgba(0, 0, 0, 1)">kubeadm.yaml

certificate embedded </span><span style="color: rgba(0, 0, 255, 1)">in</span> the kubeconfig <span style="color: rgba(0, 0, 255, 1)">file</span> <span style="color: rgba(0, 0, 255, 1)">for</span> the admin to use and <span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> kubeadm itself renewed

certificate </span><span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> serving the Kubernetes API renewed

certificate the apiserver uses to access etcd renewed

certificate </span><span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> the API server to connect to kubelet renewed

certificate embedded </span><span style="color: rgba(0, 0, 255, 1)">in</span> the kubeconfig <span style="color: rgba(0, 0, 255, 1)">file</span> <span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> the controller manager to use renewed

certificate </span><span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> liveness probes to healtcheck etcd renewed

certificate </span><span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> etcd nodes to communicate with each other renewed

certificate </span><span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> serving etcd renewed

certificate </span><span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> the front proxy client renewed

certificate embedded </span><span style="color: rgba(0, 0, 255, 1)">in</span> the kubeconfig <span style="color: rgba(0, 0, 255, 1)">file</span> <span style="color: rgba(0, 0, 255, 1)">for</span><span style="color: rgba(0, 0, 0, 1)"> the scheduler manager to use renewed

# kubeadm alpha certs check-<span style="color: rgba(0, 0, 0, 1)">expiration

CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED

admin.conf               Aug </span><span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

apiserver                  Aug </span><span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

apiserver</span>-etcd-client      Aug <span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

apiserver</span>-kubelet-client   Aug <span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

controller</span>-manager.conf    Aug <span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

etcd</span>-healthcheck-client    Aug <span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

etcd</span>-peer                  Aug <span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

etcd</span>-server                Aug <span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

front</span>-proxy-client         Aug <span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span><span style="color: rgba(0, 0, 0, 1)"> UTC   364d            no      

scheduler.conf             Aug </span><span style="color: rgba(128, 0, 128, 1)">07</span>, <span style="color: rgba(128, 0, 128, 1)">2022</span> <span style="color: rgba(128, 0, 128, 1)">16</span>:<span style="color: rgba(128, 0, 128, 1)">02</span> UTC   364d            no</pre>
</div>
<p>&nbsp;</p>
<p>&nbsp;</p>
<h3><span style="font-family: 宋体">重启</span>master<span style="font-family: 宋体">节点三个容器:</span></h3>
<div class="cnblogs_code">
<pre># docker <span style="color: rgba(0, 0, 255, 1)">ps</span> |<span style="color: rgba(0, 0, 255, 1)">grep</span> -E <span style="color: rgba(128, 0, 0, 1)">'</span><span style="color: rgba(128, 0, 0, 1)">k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd</span><span style="color: rgba(128, 0, 0, 1)">'</span>|<span style="color: rgba(0, 0, 255, 1)">xargs</span><span style="color: rgba(0, 0, 0, 1)"> docker restart

98257170f1fb

k8s_kube</span>-apiserver_kube-apiserver-k8s-master_kube-<span style="color: rgba(0, 0, 0, 1)">system_db9cf46161351d3a7f76537093caa0b8_10

82c07f5d9b6f

k8s_etcd_etcd</span>-k8s-master_kube-<span style="color: rgba(0, 0, 0, 1)">system_2da345f314df09b06ba8257f5457dbed_6

Error response from daemon: No such container: 201c7a840312

Error response from daemon: No such container: kube</span>-apiserver --<span style="color: rgba(0, 0, 0, 1)">ad…

Error response from daemon: No such container: </span><span style="color: rgba(128, 0, 128, 1)">18</span><span style="color: rgba(0, 0, 0, 1)">

Error response from daemon: No such container: months

Error response from daemon: No such container: ago

Error response from daemon: No such container: Up

Error response from daemon: No such container: </span><span style="color: rgba(128, 0, 128, 1)">18</span><span style="color: rgba(0, 0, 0, 1)">

Error response from daemon: No such container: months

Error response from daemon: No such container: 2c4adeb21b4f

Error response from daemon: No such container: etcd </span>--advertise-<span style="color: rgba(0, 0, 0, 1)">cl…

Error response from daemon: No such container: </span><span style="color: rgba(128, 0, 128, 1)">18</span><span style="color: rgba(0, 0, 0, 1)">

Error response from daemon: No such container: months

Error response from daemon: No such container: ago

Error response from daemon: No such container: Up

Error response from daemon: No such container: </span><span style="color: rgba(128, 0, 128, 1)">18</span><span style="color: rgba(0, 0, 0, 1)">

Error response from daemon: No such container: months

# kubectl get nodes

NAME         STATUS   ROLES    AGE    VERSION

k8s</span>-master   Ready    master   2y1d   v1.<span style="color: rgba(128, 0, 128, 1)">15.0</span><span style="color: rgba(0, 0, 0, 1)">

k8s</span>-node1    Ready    node   2y1d   v1.<span style="color: rgba(128, 0, 128, 1)">15.0</span><span style="color: rgba(0, 0, 0, 1)">

# <span style="color: rgba(0, 0, 255, 1)">date</span><span style="color: rgba(0, 0, 0, 1)">

2021年 08月 08日 星期日 </span><span style="color: rgba(128, 0, 128, 1)">00</span>:<span style="color: rgba(128, 0, 128, 1)">04</span>:<span style="color: rgba(128, 0, 128, 1)">33</span><span style="color: rgba(0, 0, 0, 1)"> CST

#</pre>
</div>
<p>&nbsp;</p>
<p>&nbsp;</p>
<h1><strong>注意同步配置文件:</strong></h1>
<p>cp /etc/kubernetes/admin.conf /root/.kube/config</p>
<p><span style="font-family: 宋体">删除</span>.kube<span style="font-family: 宋体">下的缓存目录</span></p>
<p>&nbsp;</p>
<h1><span style="font-family: 宋体">总结</span></h1>
<h2><span style="font-family: 宋体">当集群证书过期时操作步骤:</span></h2>
<p><span style="font-family: 宋体">1.提前备份集群配置文件</span></p>
<p> kubeadm config view &gt; /root/kubeadm.yaml</p>
<p><span style="font-family: 宋体">2.更新集群证书</span></p>
<p>&nbsp; kubeadm alpha certs renew all --config=/root/kubeadm.yaml</p>
<p>3.同步配置文件,清除.kube下缓存</p>
<p>&nbsp; cp /etc/kubernetes/admin.conf /root/.kube/config&nbsp;</p><br><br>
来源:https://www.cnblogs.com/xuliang666/p/12221973.html
頁: [1]
查看完整版本: kubernetes集群证书更新