deepin 安装 microk8s 1.17 踩坑记录
<h1 id="microk8s-117">microk8s 1.17</h1><blockquote>
<p>环境: Debian 系 Deepin 15.11 桌面系统, ubuntu 理论上可参照</p>
<p>安装参考链接: http://www.imooc.com/article/291860</p>
</blockquote>
<h2 id="安装">安装</h2>
<pre><code class="language-sh">sudo apt update
sudo apt install snapd snap
export $PATH=PATH:/snap/bin >> ~/.zshrc && source ~/.zshrc
sudo snap install microk8s --classic
sudo microk8s.status --wait-ready
## status 输出
microk8s is running
addons:
cilium: disabled
dashboard: enabled
dns: enabled
fluentd: disabled
gpu: disabled
helm: disabled
ingress: disabled
istio: disabled
jaeger: disabled
juju: disabled
knative: disabled
kubeflow: disabled
linkerd: disabled
metallb: disabled
metrics-server: disabled
prometheus: disabled
rbac: disabled
registry: disabled
storage: disabled
</code></pre>
<h2 id="监控-pods-状态">监控 pods 状态</h2>
<p><code>watch microk8s.kubectl get all --all-namespaces</code></p>
<blockquote>
<p>这是问题解决后的状态, STATUS 都是 Running</p>
</blockquote>
<pre><code class="language-sh">NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-9b8997588-hlqxz 1/1 Running 54 4h38m
kube-system pod/dashboard-metrics-scraper-687667bb6c-7f79n 0/1 Pending 0 6m50s
kube-system pod/dashboard-metrics-scraper-687667bb6c-r8tgq 0/1 Evicted 0 37m
kube-system pod/heapster-v1.5.2-5c58f64f8b-lj2nf 4/4 Running 0 37m
kube-system pod/kubernetes-dashboard-5c848cc544-47fqk 0/1 Evicted 0 6m53s
kube-system pod/kubernetes-dashboard-5c848cc544-4zdgs 0/1 Evicted 0 6m52s
kube-system pod/kubernetes-dashboard-5c848cc544-7mhmj 0/1 Evicted 0 6m52s
kube-system pod/kubernetes-dashboard-5c848cc544-7xwfw 0/1 Pending 0 6m50s
kube-system pod/kubernetes-dashboard-5c848cc544-c7t4v 0/1 Evicted 0 6m51s
kube-system pod/kubernetes-dashboard-5c848cc544-kfnds 0/1 Evicted 0 6m53s
kube-system pod/kubernetes-dashboard-5c848cc544-l8r6s 0/1 Evicted 0 6m54s
kube-system pod/kubernetes-dashboard-5c848cc544-ms8gg 0/1 Evicted 0 6m54s
kube-system pod/kubernetes-dashboard-5c848cc544-ngvlc 0/1 Evicted 0 6m54s
kube-system pod/kubernetes-dashboard-5c848cc544-p7xqc 0/1 Evicted 0 6m54s
kube-system pod/kubernetes-dashboard-5c848cc544-wlw5m 0/1 Evicted 0 37m
kube-system pod/monitoring-influxdb-grafana-v4-6d599df6bf-nvr62 2/2 Running 0 37m
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 5h34m
kube-system service/dashboard-metrics-scraper ClusterIP 10.152.183.61 <none> 8000/TCP 37m
kube-system service/heapster ClusterIP 10.152.183.168 <none> 80/TCP 37m
kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 4h38m
kube-system service/kubernetes-dashboard ClusterIP 10.152.183.29 <none> 443/TCP 37m
kube-system service/monitoring-grafana ClusterIP 10.152.183.195 <none> 80/TCP 37m
kube-system service/monitoring-influxdb ClusterIP 10.152.183.212 <none> 8083/TCP,8086/TCP 37m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/coredns 1/1 1 1 4h38m
kube-system deployment.apps/dashboard-metrics-scraper 0/1 1 0 37m
kube-system deployment.apps/heapster-v1.5.2 1/1 1 1 37m
kube-system deployment.apps/kubernetes-dashboard 0/1 1 0 37m
kube-system deployment.apps/monitoring-influxdb-grafana-v4 1/1 1 1 37m
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/coredns-9b8997588 1 1 1 4h38m
kube-system replicaset.apps/dashboard-metrics-scraper-687667bb6c 1 1 0 37m
kube-system replicaset.apps/heapster-v1.5.2-5c58f64f8b 1 1 1 37m
kube-system replicaset.apps/kubernetes-dashboard-5c848cc544 1 1 0 37m
kube-system replicaset.apps/monitoring-influxdb-grafana-v4-6d599df6bf 1 1 1 37m
</code></pre>
<h2 id="修改-zshrc">修改 .zshrc</h2>
<p>.bashrc 教程一大把</p>
<p>如果本地没有安装 kubectl 可以使用 alias; 否则请不要全部复制粘贴;</p>
<blockquote>
<p>如果已经安装了 kubectl，可以用下面的命令覆盖配置文件：</p>
<pre><code class="language-sh">microk8s.kubectl config view --raw > $HOME/.kube/config
</code></pre>
<p>-- 来自开篇的安装教程 http://www.imooc.com/article/291860</p>
</blockquote>
<p>以下仅限于 zsh 用户; bash 用户百度下即可</p>
<p><code>vim ~/.zshrc</code></p>
<pre><code class="language-sh">export PATH=$PATH:/usr/local/go/bin:/snap/bin
alias kubectl='microk8s.kubectl'
# 命令补全
if [ $commands ]; then
source <(microk8s.kubectl completion zsh |
sed "s/complete -o default -F __start_kubectl kubectl/complete -o default -F __start_kubectl microk8s.kubectl/g" |
sed "s/complete -o default -o nospace -F __start_kubectl kubectl/complete -o default -o nospace -F __start_kubectl microk8s.kubectl/g");
fi
</code></pre>
<h2 id="添加ctr-proxy">添加ctr proxy</h2>
<blockquote>
<p>microk8s.docker 命令在 1.17 版本被移除; 由 containerd 代替;</p>
<p>之前版本的, 需要修改 dockerd-env 加代理</p>
</blockquote>
<p><code>sudo vim /var/snap/microk8s/current/args/containerd-env</code></p>
<pre><code class="language-sh">
HTTPS_PROXY=http://127.0.0.1:1082
</code></pre>
<p>重启 containerd 服务</p>
<p><code>sudo systemctl restart snap.microk8s.daemon-containerd.service</code></p>
<p><strong>没有proxy 的也可以参照开篇链接教程, 条条大路通罗马, 不是非要proxy才能完成这个事情</strong></p>
<h2 id="修改内存硬盘空间限制">修改内存/硬盘空间限制</h2>
<p><code>sudo vim /var/snap/microk8s/current/args/kubelet</code></p>
<pre><code class="language-sh"># 酌情复制
--eviction-hard="memory.available<1024Mi,nodefs.available<1Gi,imagefs.available<1Gi"
## 意思是: 当本node宿主机的 内存小于 1024Mi / 硬盘存储 小于 1Gi 时, 会将 pod 强制驱逐
</code></pre>
<blockquote>
<p>这里之前有一些理解上的错误, 原本以为是 允许多大 内存/硬盘 使用, 后来发现并不是这样子的; 鉴于这篇文章现在有 7 个阅读, 我对这 7 位读者表示抱歉...<br>
后面针对这样不明确的地方一定查证后再上传</p>
</blockquote>
<h2 id="防火墙-ufw">防火墙 ufw</h2>
<p>关于 <code>CrashLoopBackOff</code> 问题</p>
<pre><code class="language-sh">sudo iptables -P FORWARD ACCEPT
# 1.17版本是 cni0; 之前版本是 cnr0, 参照官网 TroubleShooting
sudo ufw allow in on cni0 && sudo ufw allow out on cni0
sudo ufw default allow routed
</code></pre>
<h2 id="重启-microk8s">重启 microk8s</h2>
<p><code>microk8s.stop && microk8s.start</code></p>
<h2 id="开启-add-on">开启 add-on</h2>
<pre><code class="language-sh">sudo microk8s.enable dns dashboard
# 安装输出备忘部分
# enable dashbord后的输出部分, RBAC 未开启状态下需要依赖 token开头的两个命令 获取 token
If RBAC is not enabled access the dashboard using the default token retrieved with:
token=$(microk8s.kubectl -n kube-system get secret | grep default-token | cut -d " " -f1);microk8s.kubectl -n kube-system describe secret $token
In an RBAC enabled setup (microk8s.enable RBAC) you need to create a user with restricted
permissions as shown in:
https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md
</code></pre>
<h2 id="开启dashboard">开启dashboard</h2>
<p>开启proxy</p>
<p><code>kubectl proxy --address='0.0.0.0'--accept-hosts='^*$'</code></p>
<p>新开命令行, 使用 <code>kubectl get service -n kube-system</code> 查看dashboard的 ip</p>
<pre><code class="language-sh">kubectl get service -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.152.183.169 <none> 8000/TCP 17h
heapster ClusterIP 10.152.183.76 <none> 80/TCP 17h
kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 22h
kubernetes-dashboard ClusterIP 10.152.183.237 <none> 443/TCP 17h
monitoring-grafana ClusterIP 10.152.183.197 <none> 80/TCP 17h
monitoring-influxdb ClusterIP 10.152.183.82 <none> 8083/TCP,8086/TCP 17h
</code></pre>
<p>浏览器访问 https://10.152.183.237 可以到达 k8s-dashboard 界面 *注意 https 一定要加上, 不能去访问 443 端口(10.152.183.237:443 是行不通的)</p>
<p>或者: http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#/login</p>
<h2 id="dashboard-用户">dashboard 用户</h2>
<blockquote>
<p>csdn 大佬路子 https://blog.csdn.net/wucong60/article/details/81911859</p>
<pre><code class="language-sh">####
### dashboard addon 启动之后, microk8s 会自动启动一个 token 认证服务 default-token-b96pr 在 default namespace 里面
### 可以通过命令:
### $ token=$(microk8s.kubectl -n kube-system get secret | grep default-token | cut -d " " -f1);microk8s.kubectl -n kube-system describe secret $token
### 直接获取到 token
####
</code></pre>
<hr>
<pre><code class="language-sh">
# 下面是手动创建secret
# 创建服务
kubectl create serviceaccount cluster-admin-dashboard-sa
# 启动
kubectl create clusterrolebinding cluster-admin-dashboard-sa --clusterrole=cluster-admin --serviceaccount=default:cluster-admin-dashboard-sa
# 获取 pod 名称
kubectl get secret | grep cluster-admin-dashboard-sa
# 获取 token
kubectl describe secrets/cluster-admin-dashboard-sa-token-82dwx
# 查看 token 服务
kubectl get serviceaccount
# 删除手动创建的服务
kubectl delete serviceaccount cluster-admin-dashboard-sa
</code></pre>
</blockquote>
<p>https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md</p>
<p>操作流程:</p>
<h2 id="开启-rbac-认证后的-官方推荐-操作流程">开启 RBAC 认证后的 官方推荐 操作流程</h2>
<h3 id="create-service-account">Create Service Account</h3>
<pre><code class="language-sh">mkdir ~/microk8s && cd ~/microk8s
vim dashboard-adminuser.yaml
# 写入文件
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kube-system
# EOF
# 应用更改
kubectl apply -f ./dashboard-adminuser.yaml
# 输出 serviceaccount/admin-user created
</code></pre>
<h3 id="create-clusterrolebinding">Create ClusterRoleBinding</h3>
<pre><code class="language-sh">touch role-bind.yaml
echo 'apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard' > role-bind.yaml
kubectl apply -f ./role-bind.yaml
# 输出 clusterrolebinding.rbac.authorization.k8s.io/admin-user created
</code></pre>
<h3 id="获取token">获取token</h3>
<pre><code class="language-sh">kubectl -n kube-system describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')
</code></pre>
<h2 id="coredns-不启动---deepin">coredns 不启动 - deepin</h2>
<p>snap 的安装 触发了 apparmor 的启动;</p>
<p>pod错误: CrashLoopBackOff</p>
<p>coredns日志:</p>
<p><code>kubectl logs -f coredns-xxxxxxx-xxxxx -n kube-system</code></p>
<p>:: socket permission denied; listen tcp port failed</p>
<p>暂行解决办法:</p>
<p>option#1. 关闭 apparmor https://wiki.debian.org/AppArmor/HowToUse#Disable_AppArmor</p>
<pre><code>$ sudo mkdir -p /etc/default/grub.d
$ echo 'GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT apparmor=0"' \
| sudo tee /etc/default/grub.d/apparmor.cfg
$ sudo update-grub
$ sudo reboot
</code></pre>
<p>option#2. 参考链接 https://blog.csdn.net/u014062332/article/details/100099196</p>
<h2 id="删除-evicted-pod">删除 Evicted pod</h2>
<p>microk8s 运行一天后, 发现有很多处于 Evicted(被k8s放弃) 的 pod, <s>原因不明,删了去求</s>,原因是 kubectl 配置文件有问题->k8s检测到系统资源达到了阀值, 放弃了pod以释放资源</p>
<p><code>kubectl get pods -n kube-system | grep Evicted | awk '{print $1}' | xargs microk8s.kubectl delete pod -n kube-system</code></p>
<p>参考链接 https://serverfault.com/questions/972120/microk8s-keeps-evicting-pods</p>
<p>参考链接 https://blog.csdn.net/zzq900503/article/details/83788152</p><br><br>
来源：https://www.cnblogs.com/gettolive/p/12091979.html
頁:
[1]