Docker学习（五）-Kubernetes 集群搭建 - Spring Boot 应用

有奶油 發表於 2019-11-18 15:01:00

Docker学习（五）-Kubernetes 集群搭建 - Spring Boot 应用

<h2>Docker学习</h2>
Docker学习-VMware Workstation 本地多台虚拟机互通,主机网络互通搭建
Docker学习-Docker搭建Consul集群
Docker学习-简单的私有DockerHub搭建
Docker学习-Spring Boot on Docker
Docker学习-Kubernetes - 集群部署 
Docker学习-Kubernetes - Spring Boot 应用
 
<h2 id="简介">简介</h2>
kubernetes，简称K8s，是用8代替8个字符“ubernete”而成的缩写。是一个开源的，用于管理云平台中多个主机上的容器化的应用，Kubernetes的目标是让部署容器化的应用简单并且高效（powerful）,Kubernetes提供了应用部署，规划，更新，维护的一种机制。
<div class="para">Kubernetes是Google开源的一个容器编排引擎，它支持自动化部署、大规模可伸缩、应用容器化管理。在生产环境中部署一个应用程序时，通常要部署该应用的多个实例以便对应用请求进行负载均衡。</div>
<div class="para">在Kubernetes中，我们可以创建多个容器，每个容器里面运行一个应用实例，然后通过内置的负载均衡策略，实现对这一组应用实例的管理、发现、访问，而这些细节都不需要运维人员去进行复杂的手工配置和处理。</div>
<div class="para"> </div>
<h2 id="基本概念">基本概念</h2>
Kubernetes 中的绝大部分概念都抽象成 Kubernetes 管理的一种资源对象
<ul>
<li>Master：Master 节点是 Kubernetes 集群的控制节点，负责整个集群的管理和控制。Master 节点上包含以下组件：</li>
<li>kube-apiserver：集群控制的入口，提供 HTTP REST 服务</li>
<li>kube-controller-manager：Kubernetes 集群中所有资源对象的自动化控制中心</li>
<li>kube-scheduler：负责 Pod 的调度</li>
<li>
Node：Node 节点是 Kubernetes 集群中的工作节点，Node 上的工作负载由 Master 节点分配，工作负载主要是运行容器应用。Node 节点上包含以下组件：
<ul>
<li>kubelet：负责 Pod 的创建、启动、监控、重启、销毁等工作，同时与 Master 节点协作，实现集群管理的基本功能。</li>
<li>kube-proxy：实现 Kubernetes Service 的通信和负载均衡</li>
<li>运行容器化(Pod)应用</li>
</ul>
</li>
<li>
Pod: Pod 是 Kubernetes 最基本的部署调度单元。每个 Pod 可以由一个或多个业务容器和一个根容器(Pause 容器)组成。一个 Pod 表示某个应用的一个实例
</li>
<li>ReplicaSet：是 Pod 副本的抽象，用于解决 Pod 的扩容和伸缩</li>
<li>Deployment：Deployment 表示部署，在内部使用ReplicaSet 来实现。可以通过 Deployment 来生成相应的 ReplicaSet 完成 Pod 副本的创建</li>
<li>Service：Service 是 Kubernetes 最重要的资源对象。Kubernetes 中的 Service 对象可以对应微服务架构中的微服务。Service 定义了服务的访问入口，服务的调用者通过这个地址访问 Service 后端的 Pod 副本实例。Service 通过 Label Selector 同后端的 Pod 副本建立关系，Deployment 保证后端Pod 副本的数量，也就是保证服务的伸缩性。</li>
</ul>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117092935131-1437208919.png">
Kubernetes 主要由以下几个核心组件组成:
<ul>
<li>etcd 保存了整个集群的状态，就是一个数据库；</li>
<li>apiserver 提供了资源操作的唯一入口，并提供认证、授权、访问控制、API 注册和发现等机制；</li>
<li>controller manager 负责维护集群的状态，比如故障检测、自动扩展、滚动更新等；</li>
<li>scheduler 负责资源的调度，按照预定的调度策略将 Pod 调度到相应的机器上；</li>
<li>kubelet 负责维护容器的生命周期，同时也负责 Volume（CSI）和网络（CNI）的管理；</li>
<li>Container runtime 负责镜像管理以及 Pod 和容器的真正运行（CRI）；</li>
<li>kube-proxy 负责为 Service 提供 cluster 内部的服务发现和负载均衡；</li>
</ul>
当然了除了上面的这些核心组件，还有一些推荐的插件：
<ul>
<li>kube-dns 负责为整个集群提供 DNS 服务</li>
<li>Ingress Controller 为服务提供外网入口</li>
<li>Heapster 提供资源监控</li>
<li>Dashboard 提供 GUI</li>
</ul>
<h2 id="组件通信">组件通信</h2>
Kubernetes 多组件之间的通信原理：
<ul>
<li>apiserver 负责 etcd 存储的所有操作，且只有 apiserver 才直接操作 etcd 集群</li>
<li>
apiserver 对内（集群中的其他组件）和对外（用户）提供统一的 REST API，其他组件均通过 apiserver 进行通信
<ul>
<li>controller manager、scheduler、kube-proxy 和 kubelet 等均通过 apiserver watch API 监测资源变化情况，并对资源作相应的操作</li>
<li>所有需要更新资源状态的操作均通过 apiserver 的 REST API 进行</li>
</ul>
</li>
<li>
apiserver 也会直接调用 kubelet API（如 logs, exec, attach 等），默认不校验 kubelet 证书，但可以通过 <code>--kubelet-certificate-authority</code> 开启（而 GKE 通过 SSH 隧道保护它们之间的通信）
</li>
</ul>
比如最典型的创建 Pod 的流程：
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117093021317-287383941.png">
<ul>
<li>用户通过 REST API 创建一个 Pod</li>
<li>apiserver 将其写入 etcd</li>
<li>scheduluer 检测到未绑定 Node 的 Pod，开始调度并更新 Pod 的 Node 绑定</li>
<li>kubelet 检测到有新的 Pod 调度过来，通过 container runtime 运行该 Pod</li>
<li>kubelet 通过 container runtime 取到 Pod 状态，并更新到 apiserver 中</li>
</ul>
 
<h1>集群部署</h1>
 
<h2>使用kubeadm工具安装</h2>
1. master和node 都用yum 安装kubelet，kubeadm，docker 2. master 上初始化：kubeadm init 3. master 上启动一个flannel的pod 4. node上加入集群:kubeadm join
 
<h2>准备环境</h2>
Centos7  192.168.50.21 k8s-master　　 Centos7  192.168.50.22 k8s-node01 Centos7  192.168.50.23 k8s-node02
修改主机名（3台机器都需要修改）
<div class="cnblogs_code">
<pre>hostnamectl set-hostname k8s-master</pre>
</div>
<div class="cnblogs_code">
<pre>hostnamectl set-hostname k8s-node01</pre>
</div>
<div class="cnblogs_code">
<pre>hostnamectl set-hostname k8s-node02</pre>
</div>
关闭防火墙
<div class="cnblogs_code">
<pre>systemctl stop firewalld.service</pre>
</div>
<h2>配置docker yum源</h2>
<div class="cnblogs_code">
<pre>yum install -y yum-utils device-mapper-persistent-data lvm2 wget
cd /etc/yum.repos.d
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117105446844-1638794250.png">
配置kubernetes yum 源
<div class="cnblogs_code">
<pre>cd /opt/
wget https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
wget https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
rpm --import yum-key.gpg
rpm --import rpm-package-key.gpg
cd /etc/yum.repos.d
vi kubernetes.repo
输入以下内容

name=Kubernetes Repo
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
enabled=1

yum repolist</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117110333551-1325458600.png">
master和node 安装kubelet，kubeadm，docker
<div class="cnblogs_Highlighter">
<pre class="brush:csharp;gutter:true;">yum install docker
yum install kubelet-1.13.1
yum install kubeadm-1.13.1
</pre>
</div>
master 上安装kubectl
<div class="cnblogs_code">
<pre>yum install kubectl-1.13.1</pre>
</div>
<h2>docker的配置</h2>
配置私有仓库和镜像加速地址,私有仓库配置参见 https://www.cnblogs.com/woxpp/p/11871886.html
<div class="cnblogs_code">
<pre>vi /etc/docker/daemon.json</pre>
</div>
 
<div class="cnblogs_code">
<pre>{
"registry-mirror":[
 "http://hub-mirror.c.163.com"
],
"insecure-registries":[
 "192.168.50.24:5000"
]
}</pre>
</div>
 
启动docker
<div class="cnblogs_code">
<pre>systemctl daemon-reload
systemctl start docker docker info</pre>
</div>
master 上初始化：kubeadm init 
<div class="cnblogs_Highlighter">
<pre class="brush:csharp;gutter:true;">vi /etc/sysconfig/kubelet
</pre>
</div>
<div class="cnblogs_code">
<pre>KUBELET_EXTRA_ARGS="--fail-swap-on=false"</pre>
</div>
<div class="cnblogs_code">
<pre>kubeadm init \
--apiserver-advertise-address=192.168.50.21 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.13.1 \
--pod-network-cidr=10.244.0.0/16</pre>
</div>
初始化命令说明：
<pre class="bash"><code class="hljs">--apiserver-advertise-address</code></pre>
指明用 Master 的哪个 interface 与 Cluster 的其他节点通信。如果 Master 有多个 interface，建议明确指定，如果不指定，kubeadm 会自动选择有默认网关的 interface。
<pre class="bash"><code class="hljs">--pod-network-cidr</code></pre>
指定 Pod 网络的范围。Kubernetes 支持多种网络方案，而且不同网络方案对 --pod-network-cidr 有自己的要求，这里设置为 10.244.0.0/16 是因为我们将使用 flannel 网络方案，必须设置成这个 CIDR。
<pre class="bash"><code class="hljs">--image-repository</code></pre>
Kubenetes默认Registries地址是 k8s.gcr.io，在国内并不能访问 gcr.io，在1.13版本中我们可以增加–image-repository参数，默认值是 k8s.gcr.io，将其指定为阿里云镜像地址：registry.aliyuncs.com/google_containers。
<pre><code class="hljs ini">--kubernetes-version=v1.13.1 </code></pre>
关闭版本探测，因为它的默认值是stable-1，会导致从https://dl.k8s.io/release/stable-1.txt下载最新的版本号，我们可以将其指定为固定版本（最新版：v1.13.1）来跳过网络请求。
初始化过程中
<pre> You can also perform this action in beforehand using 'kubeadm config images pull' 是在下载镜像文件，过程比较慢。 </pre>
<pre> Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
All control plane components are healthy after 24.002300 seconds 这个过程也比较慢可以忽略</pre>
<pre> </pre>
<div class="cnblogs_code">
<pre> Using Kubernetes version: v1.13.1
Running pre-flight checks
Pulling images required for setting up a Kubernetes cluster
This might take a minute or two, depending on the speed of your internet connection
You can also perform this action in beforehand using 'kubeadm config images pull'
Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
Activating the kubelet service
Using certificateDir folder "/etc/kubernetes/pki"
Generating "ca" certificate and key
Generating "apiserver-kubelet-client" certificate and key
Generating "apiserver" certificate and key
apiserver serving cert is signed for DNS names and IPs [10.96.0.1 192.168.50.21]
Generating "front-proxy-ca" certificate and key
Generating "front-proxy-client" certificate and key
Generating "etcd/ca" certificate and key
Generating "etcd/healthcheck-client" certificate and key
Generating "apiserver-etcd-client" certificate and key
Generating "etcd/server" certificate and key
etcd/server serving cert is signed for DNS names and IPs [192.168.50.21 127.0.0.1 ::1]
Generating "etcd/peer" certificate and key
etcd/peer serving cert is signed for DNS names and IPs [192.168.50.21 127.0.0.1 ::1]
Generating "sa" key and public key
Using kubeconfig folder "/etc/kubernetes"
Writing "admin.conf" kubeconfig file
Writing "kubelet.conf" kubeconfig file
Writing "controller-manager.conf" kubeconfig file
Writing "scheduler.conf" kubeconfig file
Using manifest folder "/etc/kubernetes/manifests"
Creating static Pod manifest for "kube-apiserver"
Creating static Pod manifest for "kube-controller-manager"
Creating static Pod manifest for "kube-scheduler"
Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
All control plane components are healthy after 24.002300 seconds
storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
Creating a ConfigMap "kubelet-config-1.13" in namespace kube-system with the configuration for the kubelets in the cluster
Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "k8s-master" as an annotation
Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
Marking the node k8s-master as control-plane by adding the taints
Using token: 7ax0k4.nxpjjifrqnbrpojv
Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
creating the "cluster-info" ConfigMap in the "kube-public" namespace
Applied essential addon: CoreDNS
Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f .yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

kubeadm join 192.168.50.21:6443 --token 7ax0k4.nxpjjifrqnbrpojv --discovery-token-ca-cert-hash sha256:95942f10859a71879c316e75498de02a8b627725c37dee33f74cd040e1cd9d6b</pre>
</div>
初始化过程说明：
1) kubeadm 执行初始化前的检查。 2) 生成kubelet的配置文件”/var/lib/kubelet/config.yaml” 3) 生成相关的各种token和证书 4) 生成 KubeConfig 文件，kubelet 需要这个文件与 Master 通信 5) 安装 Master 组件，会从指定的 Registry 下载组件的 Docker 镜像。 6) 生成token记录下来，后边使用kubeadm join往集群中添加节点时会用到 7) 安装附加组件 kube-proxy 和 kube-dns。 8) Kubernetes Master 初始化成功，提示如何配置常规用户使用kubectl访问集群。 9) 提示如何安装 Pod 网络。 10) 提示如何注册其他节点到 Cluster。
 
异常情况：
<div class="cnblogs_code">
<pre> : docker service is not enabled, please run 'systemctl enable docker.service'
 : running with swap on is not supported. Please disable swap
 : hostname "k8s-master" could not be reached
 : hostname "k8s-master": lookup k8s-master on 114.114.114.114:53: no such host
 : kubelet service is not enabled, please run 'systemctl enable kubelet.service'</pre>
</div>
运行
<div class="cnblogs_code">
<pre>systemctl enable docker.service</pre>
</div>
<div class="cnblogs_code">
<pre>systemctl enable kubelet.service</pre>
</div>
会提示以下错误
<div class="cnblogs_code">
<pre>: hostname "k8s-master" could not be reached
 : hostname "k8s-master": lookup k8s-master on 114.114.114.114:53: no such host
error execution phase preflight: Some fatal errors occurred:</pre>
</div>
配置host
<div class="cnblogs_code">
cat >> /etc/hosts << EOF 192.168.50.21 k8s-master 192.168.50.22 k8s-node01 192.168.50.23 k8s-node02 EOF

</div>
再次运行初始化命令会出现
<div class="cnblogs_code">
<pre>: the number of available CPUs 1 is less than the required 2 --设置虚拟机CPU个数大于2
 : /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1</pre>
</div>
<div class="cnblogs_code">
<pre data-spm-anchor-id="a2c4e.11153940.0.i0.1b9e5aa9VoW81r"><code class="shell hljs bash">echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables</code></pre>
</div>
设置好虚拟机CPU个数，重启后再次运行：
<div class="cnblogs_code">
<pre>kubeadm init \
--apiserver-advertise-address=192.168.50.21 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.13.1 \
--pod-network-cidr=10.244.0.0/16</pre>
</div>
 
<div class="cnblogs_code">
<pre> Using Kubernetes version: v1.13.1
Running pre-flight checks
Pulling images required for setting up a Kubernetes cluster
This might take a minute or two, depending on the speed of your internet connection
You can also perform this action in beforehand using 'kubeadm config images pull</pre>
</div>
解决办法：docker.io仓库对google的容器做了镜像，可以通过下列命令下拉取相关镜像
先看下需要用到哪些
<div class="cnblogs_code">
<pre>kubeadm config images list</pre>
</div>
配置yum源
<div class="cnblogs_code">
<pre># vi kubeadm-config.yaml</pre>
</div>
<div class="cnblogs_code">
<pre>apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.13.1
imageRepository: registry.aliyuncs.com/google_containers
apiServer:
certSANs:
- 192.168.50.21
controlPlaneEndpoint: "192.168.50.20:16443"
networking:
# This CIDR is a Calico default. Substitute or remove for your CNI provider.
podSubnet: "172.168.0.0/16"</pre>
</div>
<div class="cnblogs_code">
<pre> kubeadm config images pull --config /opt/kubeadm-config.yaml</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117153439259-891551174.png">
初始化master
<div class="cnblogs_code">
<pre>kubeadm init --config=kubeadm-config.yaml--upload-certs</pre>
</div>
 
<div class="cnblogs_code">
<pre>xecution phase preflight: Some fatal errors occurred:
 : /etc/kubernetes/manifests/kube-apiserver.yaml already exists
 : /etc/kubernetes/manifests/kube-controller-manager.yaml already exists
 : /etc/kubernetes/manifests/kube-scheduler.yaml already exists
 : /etc/kubernetes/manifests/etcd.yaml already exists
 : Port 10250 is in use</pre>
</div>
 kubeadm会自动检查当前环境是否有上次命令执行的“残留”。如果有,必须清理后再行执行init。我们可以通过”kubeadm reset”来清理环境,以备重来。
<div class="cnblogs_code">
<pre> Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
Initial timeout of 40s passed.</pre>
</div>
==原因==
这是因为kubelet没启动
==解决==
systemctl restart kubelet
如果启动不了kubelet
<div class="cnblogs_code">
<pre>kubelet.service - kubelet: The Kubernetes Node Agent</pre>
</div>
 
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117154814657-1743243357.png">
则可能是swap交换分区还开启的原因 -关闭swap
<div class="cnblogs_code">
<pre>swapoff -a</pre>
</div>
-配置kubelet
<div class="cnblogs_code">
<pre>vi /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--fail-swap-on=false"</pre>
</div>
 再次运行
<div class="cnblogs_code">
<pre>kubeadm init \
--apiserver-advertise-address=192.168.50.21 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.13.1 \
--pod-network-cidr=10.244.0.0/16</pre>
</div>
 
 
<h2 id="配置-kubectl">配置 kubectl</h2>
kubectl 是管理 Kubernetes Cluster 的命令行工具，前面我们已经在所有的节点安装了 kubectl。Master 初始化完成后需要做一些配置工作，然后 kubectl 就能使用了。 依照 kubeadm init 输出的最后提示，推荐用 Linux 普通用户执行 kubectl。
<ul>
<li>创建普通用户centos</li>

</ul>
<div class="cnblogs_code">
<pre>#创建普通用户并设置密码123456
useradd centos && echo "centos:123456" | chpasswd centos

#追加sudo权限,并配置sudo免密
sed -i '/^root/a\centosALL=(ALL) NOPASSWD:ALL' /etc/sudoers

#保存集群安全配置文件到当前用户.kube目录
su - centos
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

#启用 kubectl 命令自动补全功能（注销重新登录生效）
echo "source <(kubectl completion bash)" >> ~/.bashrc</pre>
</div>
需要这些配置命令的原因是：Kubernetes 集群默认需要加密方式访问。所以，这几条命令，就是将刚刚部署生成的 Kubernetes 集群的安全配置文件，保存到当前用户的.kube 目录下，kubectl 默认会使用这个目录下的授权信息访问 Kubernetes 集群。 如果不这么做的话，我们每次都需要通过 export KUBECONFIG 环境变量告诉 kubectl 这个安全配置文件的位置。 配置完成后centos用户就可以使用 kubectl 命令管理集群了。
查看集群状态：
kubectl get cs
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117170436395-764170430.png">
 
 
 
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117170602677-571190357.png">
 
 
 部署网络插件
<div class="cnblogs_code">
<pre>kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117170722717-1727191576.png">
  
kubectl get 重新检查 Pod 的状态
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191117170748717-1773195017.png">
 
 
<h2 id="部署worker节点">部署worker节点</h2>
 在master机器保存生成号的镜像文件
<div class="cnblogs_code">
<pre>docker save -o master.tar registry.aliyuncs.com/google_containers/kube-proxy:v1.13.1 registry.aliyuncs.com/google_containers/kube-apiserver:v1.13.1 registry.aliyuncs.com/google_containers/kube-controller-manager:v1.13.1 registry.aliyuncs.com/google_containers/kube-scheduler:v1.13.1registry.aliyuncs.com/google_containers/coredns:1.2.6registry.aliyuncs.com/google_containers/etcd:3.2.24 registry.aliyuncs.com/google_containers/pause:3.1</pre>
</div>
注意对应的版本号
将master上保存的镜像同步到节点上
<div class="cnblogs_code">
<pre>scp master.tar node01:/root/
scp master.tar node02:/root/</pre>
</div>
将镜像导入本地，node01,node02
<div class="cnblogs_code">
<pre> docker load< master.tar</pre>
</div>
配置host，node01,node02
<div class="cnblogs_code">
cat >> /etc/hosts << EOF 192.168.50.21 k8s-master 192.168.50.22 k8s-node01 192.168.50.23 k8s-node02 EOF

</div>
配置iptables，node01,node02
<div class="cnblogs_code">
<pre>echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables</pre>
</div>
-关闭swap，node01,node02
<div class="cnblogs_code">
<pre>swapoff -a</pre>
</div>
-配置kubelet，node01,node02
<div class="cnblogs_code">
<pre>vi /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--fail-swap-on=false" </pre>
</div>
<div class="cnblogs_code">
<pre>systemctl enable docker.service</pre>
</div>
<div class="cnblogs_code">
<pre>systemctl enable kubelet.service</pre>
</div>
启动docker，node01,node02
<div class="cnblogs_code">
<pre>service docker strat</pre>
</div>
部署网络插件，node01,node02
<div class="cnblogs_code">
<pre>kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml</pre>
</div>
获取join指令，master
<div class="cnblogs_code">
<pre>kubeadm token create --print-join-command</pre>
</div>
<div class="cnblogs_code">
<pre>kubeadm token create --print-join-command
kubeadm join 192.168.50.21:6443 --token n9g4nq.kf8ppgpgb3biz0n5 --discovery-token-ca-cert-hash sha256:95942f10859a71879c316e75498de02a8b627725c37dee33f74cd040e1cd9d6b</pre>
</div>
 
在子节点运行指令，node01,node02
<div class="cnblogs_code">
<pre>kubeadm join 192.168.50.21:6443 --token n9g4nq.kf8ppgpgb3biz0n5 --discovery-token-ca-cert-hash sha256:95942f10859a71879c316e75498de02a8b627725c37dee33f74cd040e1cd9d6b
Running pre-flight checks
Trying to connect to API Server "192.168.50.21:6443"
Created cluster-info discovery client, requesting info from "https://192.168.50.21:6443"
Requesting info from "https://192.168.50.21:6443" again to validate TLS against the pinned public key
Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.50.21:6443"
Successfully established connection with API Server "192.168.50.21:6443"
Reading configuration from the cluster...
FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
WARNING: unable to stop the kubelet service momentarily:
Downloading configuration for the kubelet from the "kubelet-config-1.13" ConfigMap in the kube-system namespace
Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
Activating the kubelet service
Waiting for the kubelet to perform the TLS Bootstrap...
Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "k8s-node01" as an annotation

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the master to see this node join the cluster.</pre>
</div>
在master上查看节点状态
<div class="cnblogs_code">
<pre>kubectl get nodes</pre>
</div>
 
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118094401937-281646431.png">
 这种状态是错误的，只有一台联机正确
查看node01，和node02发现 node01有些进程没有完全启动
删除node01所有运行的容器，node01
<div class="cnblogs_code">
<pre>docker stop $(docker ps -q) & docker rm $(docker ps -aq)</pre>
</div>
重置 kubeadm ，node01
<div class="cnblogs_code">
<pre>kubeadm reset</pre>
</div>
获取join指令，master
<div class="cnblogs_code">
<pre>kubeadm token create --print-join-command</pre>
</div>
再次在node01上运行join
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118100842382-1377946367.png">
 
 
 
查看node01镜像运行状态
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118101005720-1338173325.png">
  
查看master状态
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118101026300-1476887468.png">
  
nodes状态全部为ready，由于每个节点都需要启动若干组件，如果node节点的状态是 NotReady，可以查看所有节点pod状态，确保所有pod成功拉取到镜像并处于running状态：
<div class="cnblogs_code">
<pre>kubectl get pod --all-namespaces -o wide</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118101251489-1025836717.png">
  
<h2>配置kubernetes UI图形化界面</h2>
创建kubernetes-dashboard.yaml
<div class="cnblogs_code">
<pre># Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# ------------------- Dashboard Secret ------------------- #

apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-certs
namespace: kube-system
type: Opaque

---
# ------------------- Dashboard Service Account ------------------- #

apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system

---
# ------------------- Dashboard Role & Role Binding ------------------- #

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
rules:
# Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
# Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create"]
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system

---
# ------------------- Dashboard Deployment ------------------- #

kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
 k8s-app: kubernetes-dashboard
template:
metadata:
 labels:
 k8s-app: kubernetes-dashboard
spec:
 containers:
 - name: kubernetes-dashboard
 image: registry.cn-hangzhou.aliyuncs.com/rsqlh/kubernetes-dashboard:v1.10.1
 imagePullPolicy: IfNotPresent
 ports:
 - containerPort: 8443
 protocol: TCP
 args:
 - --auto-generate-certificates
 # Uncomment the following line to manually specify Kubernetes API server Host
 # If not specified, Dashboard will attempt to auto discover the API server and connect
 # to it. Uncomment only if the default does not work.
 # - --apiserver-host=http://my-address:port
 volumeMounts:
 - name: kubernetes-dashboard-certs
 mountPath: /certs
 # Create on-disk volume to store exec logs
 - mountPath: /tmp
 name: tmp-volume
 livenessProbe:
 httpGet:
 scheme: HTTPS
 path: /
 port: 8443
 initialDelaySeconds: 30
 timeoutSeconds: 30
 volumes:
 - name: kubernetes-dashboard-certs
 secret:
 secretName: kubernetes-dashboard-certs
 - name: tmp-volume
 emptyDir: {}
 serviceAccountName: kubernetes-dashboard
 # Comment the following tolerations if Dashboard must not be deployed on master
 tolerations:
 - key: node-role.kubernetes.io/master
 effect: NoSchedule

---
# ------------------- Dashboard Service ------------------- #

kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
type: NodePort
ports:
- port: 443
targetPort: 8443
nodePort: 30000
selector:
k8s-app: kubernetes-dashboard</pre>
</div>
 
执行以下命令创建kubernetes-dashboard：
<div class="cnblogs_code">
<pre>kubectl create -f kubernetes-dashboard.yaml</pre>
</div>
如果出现
<div class="cnblogs_code">
<pre>Error from server (AlreadyExists): error when creating "kubernetes-dashboard.yaml": secrets "kubernetes-dashboard-certs" already exists
Error from server (AlreadyExists): error when creating "kubernetes-dashboard.yaml": serviceaccounts "kubernetes-dashboard" already exists
Error from server (AlreadyExists): error when creating "kubernetes-dashboard.yaml": roles.rbac.authorization.k8s.io "kubernetes-dashboard-minimal" already exists
Error from server (AlreadyExists): error when creating "kubernetes-dashboard.yaml": rolebindings.rbac.authorization.k8s.io "kubernetes-dashboard-minimal" already exists
Error from server (AlreadyExists): error when creating "kubernetes-dashboard.yaml": deployments.apps "kubernetes-dashboard" already exists</pre>
</div>
运行delete清理
<div class="cnblogs_code">
<pre>kubectl delete -f kubernetes-dashboard.yaml</pre>
</div>
查看组件运行状态
<div class="cnblogs_code">
<pre>kubectl get pods --all-namespaces</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118104901969-230500779.png">
 
 
 ErrImagePull 拉取镜像失败
手动拉取并重置tag
<div class="cnblogs_code">
<pre>docker pull registry.cn-hangzhou.aliyuncs.com/rsqlh/kubernetes-dashboard:v1.10.1
docker tag registry.cn-hangzhou.aliyuncs.com/rsqlh/kubernetes-dashboard:v1.10.1 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1</pre>
</div>
重新创建
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118105307627-1338684217.png">
  
 
ImagePullBackOff
默认情况是会根据配置文件中的镜像地址去拉取镜像，如果设置为IfNotPresent 和Never就会使用本地镜像。
IfNotPresent ：如果本地存在镜像就优先使用本地镜像。 Never：直接不再去拉取镜像了，使用本地的；如果本地不存在就报异常了。
<div class="cnblogs_code">
<pre> spec:
 containers:
 - name: kubernetes-dashboard
 image: registry.cn-hangzhou.aliyuncs.com/rsqlh/kubernetes-dashboard:v1.10.1
 imagePullPolicy: IfNotPresent</pre>
</div>
 
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118110910185-388305445.png">
查看映射状态 
<div class="cnblogs_code">
<pre> kubectl get service-n kube-system</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118142116354-1249744125.png">
 
 
 <img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118142017885-2030922.png">
 
创建能够访问 Dashboard 的用户
 新建文件 <code>account.yaml</code> ，内容如下：
<div class="cnblogs_code">
<pre># Create Service Account
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kube-system
---
# Create ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kube-system</pre>
</div>
<div class="cnblogs_code">
<pre>kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118142354720-1245828815.png">
 
 复制token登陆
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118142548439-1861171209.png">
 
 
<div class="cnblogs_code">
<pre>configmaps is forbidden: User "system:serviceaccount:kube-system:admin-user" cannot list resource "configmaps" in API group "" in the namespace "default" </pre>
</div>
 授权用户
<div class="cnblogs_code">
<pre>kubectl create clusterrolebinding test:admin-user --clusterrole=cluster-admin --serviceaccount=kube-system:admin-user</pre>
</div>
<img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118143021255-385635182.png">
 <img src="https://img2018.cnblogs.com/blog/333725/201911/333725-20191118171622410-372526831.png">
 
 <code>NodePort</code>方式，可以到任意一个节点的<code>XXXX</code>端口查看
 Docker学习-Kubernetes - Spring Boot 应用
 
本文参考：
https://www.cnblogs.com/tylerzhou/p/10971336.html
https://www.cnblogs.com/zoujiaojiao/p/10986320.html
 

</div>
<div id="MySignature" role="contentinfo">
<style>#yangshicopy { background: url("http://images.cnitblog.com/blog/435188/201408/122329534672560.png") no-repeat scroll 1% 50% #fffefe; border: 1px solid #e5e5e5; padding: 10px 10px 10px 60px; font-size: 12px; margin-top: 10px }
#yangshicopy1 { margin-top: 10px; border: 1px solid #e5e5e5; padding: 10px 10px 10px 20px; font-size: 12px }
#yangshicopy1 img { width: 60px; height: 60px; margin-left: 10px }
#yangshicopy1 a:link { text-decoration: none }</style>
<div id="yangshicopy">
 <div style="margin-left: 13px">
 作者：释迦苦僧 
 出处：http://www.cnblogs.com/woxpp 
 本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接。</div>
</div>
<div id="yangshicopy1">
 <div style="margin-left: 13px">
生活不易,五行缺金，求打点
 
<img style="width: 150px; height: 150px" src="http://files.cnblogs.com/files/woxpp/QQ%E6%88%AA%E5%9B%BE20161229110254.bmp">
</div>
</div> 
来源：https://www.cnblogs.com/woxpp/p/11875547.html

頁: [1]

琼殿技术论坛's Archiver

Docker学习（五）-Kubernetes 集群搭建 - Spring Boot 应用