ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群

在家呆 發表於 2019-8-1 21:09:00

ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群

<h1>一、概述</h1>
搭建k8s集群时，需要访问google，下载相关镜像以及安装软件，非常麻烦。
正好阿里云提供了k8s的更新源，国内用户就可以直接使用了。
 
<h1>二、环境介绍</h1>
<table border="0" align="left">
<tbody>
<tr>
<td>操作系统</td>
<td>主机名</td>
<td>IP地址</td>
<td>功能</td>
<td>配置</td>
</tr>
<tr>
<td>ubuntu-16.04.5-server-amd64</td>
<td>k8s-master</td>
<td>192.168.10.130</td>
<td>主节点</td>
<td>2核4G</td>
</tr>
<tr>
<td>ubuntu-16.04.5-server-amd64</td>
<td>k8s-node1</td>
<td>192.168.10.131</td>
<td>从节点</td>
<td>2核4G</td>
</tr>
<tr>
<td>ubuntu-16.04.5-server-amd64</td>
<td>k8s-node2</td>
<td>192.168.10.132</td>
<td>从节点</td>
<td>2核4G</td>
</tr>
</tbody>
</table>
 
 
 
 
 
 注意：请确保CPU至少2核，内存2G
<h1>三、安装前准备</h1>
<h2>主机名</h2>
确保3台主机的 /etc/hostname 已经修改为正确的主机名，修改后，请重启系统。
 
<h2>时间</h2>
务必保证3台服务器的时区是一样的，强制更改时区为上海，执行以下命令
<div class="cnblogs_code">
<pre>ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
bash -c "echo 'Asia/Shanghai' > /etc/timezone"</pre>
</div>
 
安装ntpdate
<div class="cnblogs_code">
<pre>apt-get install -y ntpdate</pre>
</div>
如果出现以下错误
<div class="cnblogs_code">
<pre>E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?</pre>
</div>
执行2个命令解决
<div class="cnblogs_code">
<pre>sudo rm /var/cache/apt/archives/lock
sudo rm /var/lib/dpkg/lock</pre>
</div>
 
使用阿里云的时间服务器更新
<div class="cnblogs_code">
<pre>ntpdate ntp1.aliyun.com</pre>
</div>
3台服务器都执行一下，确保时间一致！
请确保防火墙都关闭了！
 
<h1>四、正式开始</h1>
<h2>禁用swap</h2>
所有主机
<div class="cnblogs_code">
<pre>sudo sed -i '/swap/ s/^/#/' /etc/fstab
sudo swapoff -a</pre>
</div>
 
<h2>安装Docker</h2>
更新apt源,并添加https支持（所有主机）
<div class="cnblogs_code">
<pre>sudo apt-get update && sudo apt-get install apt-transport-https ca-certificates curl software-properties-common -y</pre>
</div>
 
使用utc源添加GPG Key (所有主机)
<div class="cnblogs_code">
<pre>curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo apt-key add </pre>
</div>
 
添加Docker-ce稳定版源地址（所有主机）
<div class="cnblogs_code">
<pre>sudo add-apt-repository "deb https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu $(lsb_release -cs) stable"</pre>
</div>
 
安装docker-ce（所有主机）
安装最新版docker
 
<div class="cnblogs_code">
<pre>sudo apt-get update
sudo apt-get install -y docker-ce=5:19.03.1~3-0~ubuntu-xenial</pre>
</div>
 
 
 
<h2>安装kubelet，kubeadm，kubectl</h2>
添加apt key以及源（所有主机）
<div class="cnblogs_code">
<pre>sudo apt update && sudo apt install -y apt-transport-https curl
curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" >>/etc/apt/sources.list.d/kubernetes.list</pre>
</div>
 
安装（所有主机）
最新版kubelet
<div class="cnblogs_code">
<pre>sudo apt update
sudo apt install -y kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00
sudo apt-mark hold kubelet=1.15.2-00 kubeadm=1.15.2-00 kubectl=1.15.2-00</pre>
</div>
 
安装kubernetes集群（仅master）
<div class="cnblogs_code">
<pre>sudo kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.15.2 --pod-network-cidr=192.169.0.0/16 | tee /etc/kube-server-key</pre>
</div>
参数解释：
--image-repository 指定镜像源，指定为阿里云的源，这样就会避免在拉取镜像超时，如果没问题，过几分钟就能看到成功的日志输入
--kubernetes-version 指定版本
--pod-network-cidr 指定pod网络地址。设置为内网网段！
 
三大内网网络为：
C类：192.168.0.0-192.168.255.255 B类：172.16.0.0-172.31.255.255 A类：10.0.0.0-10.255.255.255
 
输出：
<div class="cnblogs_code"><img id="code_img_closed_327025a7-4298-4230-be6f-b04ab24beb16" class="code_img_closed" src="https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif" alt=""><img id="code_img_opened_327025a7-4298-4230-be6f-b04ab24beb16" class="code_img_opened" style="display: none" src="https://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif" alt="">
<div id="cnblogs_code_open_327025a7-4298-4230-be6f-b04ab24beb16" class="cnblogs_code_hide">
<pre> Using Kubernetes version: v1.15.2
Running pre-flight checks
: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
Pulling images required for setting up a Kubernetes cluster
This might take a minute or two, depending on the speed of your internet connection
You can also perform this action in beforehand using 'kubeadm config images pull'
Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
Activating the kubelet service
Using certificateDir folder "/etc/kubernetes/pki"
Generating "ca" certificate and key
Generating "apiserver" certificate and key
apiserver serving cert is signed for DNS names and IPs [10.96.0.1 192.168.10.130]
Generating "apiserver-kubelet-client" certificate and key
Generating "front-proxy-ca" certificate and key
Generating "front-proxy-client" certificate and key
Generating "etcd/ca" certificate and key
Generating "etcd/peer" certificate and key
etcd/peer serving cert is signed for DNS names and IPs [192.168.10.130 127.0.0.1 ::1]
Generating "etcd/healthcheck-client" certificate and key
Generating "etcd/server" certificate and key
etcd/server serving cert is signed for DNS names and IPs [192.168.10.130 127.0.0.1 ::1]
Generating "apiserver-etcd-client" certificate and key
Generating "sa" key and public key
Using kubeconfig folder "/etc/kubernetes"
Writing "admin.conf" kubeconfig file
Writing "kubelet.conf" kubeconfig file
Writing "controller-manager.conf" kubeconfig file
Writing "scheduler.conf" kubeconfig file
Using manifest folder "/etc/kubernetes/manifests"
Creating static Pod manifest for "kube-apiserver"
Creating static Pod manifest for "kube-controller-manager"
Creating static Pod manifest for "kube-scheduler"
Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
Initial timeout of 40s passed.
All control plane components are healthy after 41.507944 seconds
Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster
Skipping phase. Please see --upload-certs
Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
Marking the node k8s-master as control-plane by adding the taints
Using token: bz16uu.olqxoh5q5bnt50sd
Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
Creating the "cluster-info" ConfigMap in the "kube-public" namespace
Applied essential addon: CoreDNS
Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f .yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.10.130:6443 --token bz16uu.olqxoh5q5bnt50sd \
--discovery-token-ca-cert-hash sha256:9177017ff3016dbb2aadf7484f7823f8b963c989fe9ecdccbe601c9305ce000f</pre>
</div>
View Code</div>
出现WARNING信息，可以忽略。
输出信息，会保存到 /etc/kube-server-key 文件中
 
拷贝kubeconfig文件到家目录的.kube目录 （仅master）
<div class="cnblogs_code">
<pre>mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config</pre>
</div>
 
安装网络插件，让pod之间通信（仅master）
使用最新版的
<div class="cnblogs_code">
<pre>kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml</pre>
</div>
 
查看kube-system命名空间下的pod状态（仅master）
<div class="cnblogs_code">
<pre>kubectl get pod -n kube-system</pre>
</div>
 
等待1分钟，效果如下：
<div class="cnblogs_code">
<pre>NAME READY STATUS RESTARTS AGE
calico-kube-controllers-7bd78b474d-lpfvf 0/1 Running 0 67s
calico-node-vfm28 1/1 Running 0 67s
coredns-bccdc95cf-dm4pb 1/1 Running 0 111s
coredns-bccdc95cf-lvhcg 1/1 Running 0 111s
etcd-k8s-master 1/1 Running 0 69s
kube-apiserver-k8s-master 1/1 Running 0 67s
kube-controller-manager-k8s-master 1/1 Running 0 59s
kube-proxy-jpqsq 1/1 Running 0 111s
kube-scheduler-k8s-master 1/1 Running 0 56s</pre>
</div>
 
查看加入节点命令（仅master）
<div class="cnblogs_code">
<pre>cat /etc/kube-server-key | tail -2</pre>
</div>
输出：
<div class="cnblogs_code">
<pre>kubeadm join 192.168.10.130:6443 --token bz16uu.olqxoh5q5bnt50sd \
--discovery-token-ca-cert-hash sha256:9177017ff3016dbb2aadf7484f7823f8b963c989fe9ecdccbe601c9305ce000f</pre>
</div>
 
加入node节点 (仅node)
在每个node上执行
<div class="cnblogs_code">
<pre>kubeadm join 192.168.10.130:6443 --token bz16uu.olqxoh5q5bnt50sd \
--discovery-token-ca-cert-hash sha256:9177017ff3016dbb2aadf7484f7823f8b963c989fe9ecdccbe601c9305ce000f</pre>
</div>
 
等待5分钟，查看集群状态（仅master）
<div class="cnblogs_code">
<pre>root@k8s-master:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready master 5m54s v1.15.2
k8s-node1 Ready <none> 73s v1.15.2
k8s-node2 Ready <none> 71s v1.15.2</pre>
</div>
 
<h2>命令补全</h2>
（仅master）
<div class="cnblogs_code">
<pre>apt-get install bash-completion

source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc
source~/.bashrc</pre>
</div>
 
<h1>四、部署应用</h1>
（仅master）
这里以flask为例：
<div class="cnblogs_code">
<pre>vim flask.yaml</pre>
</div>
内容如下：
<div class="cnblogs_code">
<pre>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: flaskapp-1
spec:
replicas: 1
template:
metadata:
 labels:
 name: flaskapp-1
spec:
 containers:
 - name: flaskapp-1
 image: jcdemo/flaskapp
 ports:
 - containerPort: 5000

---
apiVersion: v1
kind: Service
metadata:
name: flaskapp-1
labels:
name: flaskapp-1
spec:
type: NodePort
ports:
- port: 5000
name: flaskapp-port
targetPort: 5000
protocol: TCP
nodePort: 30005
selector:
name: flaskapp-1</pre>
</div>
 
<h2>启动应用</h2>
<div class="cnblogs_code">
<pre>kubectl apply -f flask.yaml</pre>
</div>
 
查看应用状态
<div class="cnblogs_code">
<pre>root@k8s-master:~# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
flaskapp-1-59698bc97d-2xnhb 1/1 Running 0 24s 192.168.36.65 k8s-node1 <none> <none></pre>
</div>
以上信息可以说明，这个pod运行在k8s-node1这台主机上面
 
ping pod ip地址，如果可以ping通，说明calico插件运行正常。
<div class="cnblogs_code">
<pre>root@k8s-master:~# ping 192.168.36.65 -c 1
PING 192.168.36.65 (192.168.36.65) 56(84) bytes of data.
64 bytes from 192.168.36.65: icmp_seq=1 ttl=63 time=6.77 ms

--- 192.168.36.65 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 6.777/6.777/6.777/0.000 ms</pre>
</div>
 
测试 pod 是否可以上网
<div class="cnblogs_code">
<pre>root@k8s-master:~# kubectl exec -it flaskapp-1-59698bc97d-2xnhb -- ping www.baidu.com -c 1
PING www.baidu.com (61.135.169.125): 56 data bytes
64 bytes from 61.135.169.125: seq=0 ttl=53 time=27.079 ms

--- www.baidu.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 27.079/27.079/27.079 ms</pre>
</div>
 
使用curl访问pod ip的服务
<div class="cnblogs_code">
<pre>root@k8s-master:~# curl 192.168.36.65:5000
<html><head><title>Docker + Flask Demo</title></head><body><table><tr><td> Start Time </td> <td>2019-Aug-11 04:52:17</td> </tr><tr><td> Hostname </td> <td>flaskapp-1-59698bc97d-2xnhb</td> </tr><tr><td> Local Address </td> <td>192.168.36.65</td> </tr><tr><td> Remote Address </td> <td>192.168.235.192</td> </tr><tr><td> Server Hit </td> <td>1</td> </tr></table></body></html></pre>
</div>
 
查看svc端口
<div class="cnblogs_code">
<pre>root@k8s-master:~# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
flaskapp-1 NodePort 10.107.181.43 <none> 5000:30005/TCP 3m40s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 10m</pre>
</div>
 
直接网页访问k8s-node1的30005端口
<div class="cnblogs_code">
<pre>http://192.168.10.131:30005/</pre>
</div>
 
效果如下：
<img src="https://img2018.cnblogs.com/blog/1341090/201908/1341090-20190811125721617-1361852690.png" alt="">
 
<h1 class="title-article">五、部署dashboard可视化插件</h1>
<h2>概述</h2>
在 Kubernetes Dashboard 中可以查看集群中应用的运行状态，也能够创建和修改各种 Kubernetes 资源，比如 Deployment、Job、DaemonSet 等。用户可以 Scale Up/Down Deployment、执行 Rolling Update、重启某个 Pod 或者通过向导部署新的应用。Dashboard 能显示集群中各种资源的状态以及日志信息。
 
可以说，Kubernetes Dashboard 提供了 kubectl 的绝大部分功能，大家可以根据情况进行选择。
github地址：
https://github.com/kubernetes/dashboard
 
<h2>安装</h2>
Kubernetes 默认没有部署 Dashboard，可通过如下命令安装：
<div class="cnblogs_code">
<pre>kubectl apply -f http://mirror.faasx.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml</pre>
</div>
 
<h2>查看service</h2>
<div class="cnblogs_code">
<pre>root@k8s-master:~# kubectl --namespace=kube-system get deployment kubernetes-dashboard
NAME READY UP-TO-DATE AVAILABLE AGE
kubernetes-dashboard 1/1 1 1 5m23s
root@k8s-master:~# kubectl --namespace=kube-system get service kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-dashboard ClusterIP 10.100.111.103 <none> 443/TCP 5m28s</pre>
</div>
 
<h2>查看pod</h2>
确保状态是Running 
<div class="cnblogs_code">
<pre>root@k8s-master:~# kubectl get pod --namespace=kube-system -o wide | grep dashboard
kubernetes-dashboard-8594bd9565-t78bj 1/1 Running 0 8m41s 192.169.2.7 k8s-node2 <none> <none></pre>
</div>
 
<h2>允许外部访问</h2>
注意：会占用终端
<div class="cnblogs_code">
<pre>kubectl proxy --address='0.0.0.0'--accept-hosts='^*$'</pre>
</div>
 
配置登录权限
Dashboard 支持 Kubeconfig 和 Token 两种认证方式，为了简化配置，我们通过配置文件 dashboard-admin.yaml 为 Dashboard 默认用户赋予 admin 权限。
<div class="cnblogs_code">
<pre>vim dashboard-admin.yml</pre>
</div>
内容如下：
<div class="cnblogs_code">
<pre>apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
labels:
k8s-app: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system</pre>
</div>
 
执行kubectl apply使之生效
<div class="cnblogs_code">
<pre>kubectl apply -f dashboard-admin.yml</pre>
</div>
 
<h2>通过浏览器访问</h2>
注意：192.168.10.130为master ip
<div class="cnblogs_code">
<pre>http://192.168.10.130:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/.</pre>
</div>
 
现在直接点击登录页面的跳过
<img src="https://img2018.cnblogs.com/blog/1341090/201908/1341090-20190807220159100-1202168228.png" alt="">
 
就可以进入 Dashboard 了，效果如下：
<img src="https://img2018.cnblogs.com/blog/1341090/201908/1341090-20190807220211879-1899073531.png" alt="">
 
 
关于dashboard界面结构介绍，请参考链接：
https://www.cnblogs.com/kenken2018/p/10340157.html
 
<h1>六、解决k8s 外网无法访问nodePort问题</h1>
上面flask的例子，无法通过master ip+nodeport访问。
是因为防火墙阻止了
<div class="cnblogs_code">
<pre>root@k8s-master:~# iptables -xnL|grep FORWARD
Chain FORWARD (policy DROP)
cali-FORWARDall--0.0.0.0/0 0.0.0.0/0 /* cali:wUHhoiAYhphO9Mso */
KUBE-FORWARDall--0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
Chain KUBE-FORWARD (1 references)
Chain cali-FORWARD (1 references)</pre>
</div>
 
解决办法：
<div class="cnblogs_code">
<pre>iptables -P FORWARD ACCEPT</pre>
</div>
 
使用master ip+nodeport访问
<div class="cnblogs_code">
<pre>http://192.168.10.130:30005/</pre>
</div>
 
效果如下：
<img src="https://img2018.cnblogs.com/blog/1341090/201908/1341090-20190811132201363-2115968013.png" alt="">
参考链接：
https://blog.csdn.net/a610786189/article/details/80321727
 
 
本文参考链接：
https://www.toutiao.com/i6703112655323791884
https://www.cnblogs.com/busigulang/p/10736040.html
https://www.cnblogs.com/qingfeng2010/p/10540832.html
https://www.cnblogs.com/kenken2018/p/10340157.html
  
来源：https://www.cnblogs.com/xiao987334176/p/11317844.html

頁: [1]

琼殿技术论坛's Archiver

ubuntu 使用阿里云镜像源快速搭建kubernetes 1.15.2集群