Kubernetes部署高可用MongoDB集群

大西瓜说电影 發表於 2019-11-26 16:46:00

<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">在Kubernetes中，部署MongoDB主要用到的是<code style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 12.6px; padding: 2px 4px; color: rgba(199, 37, 78, 1); background-color: rgba(249, 242, 244, 1); border-radius: 4px">mongo-db-sidecar</code>。 Docker Hub
<h3 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 20px">1. 架构</h3>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">Mongodb的集群搭建方式主要有三种，主从模式，Replica set模式，sharding模式, 三种模式各有优劣，适用于不同的场合，属Replica set应用最为广泛，主从模式现在用的较少，sharding模式最为完备，但配置维护较为复杂。 <code style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 12.6px; padding: 2px 4px; color: rgba(199, 37, 78, 1); background-color: rgba(249, 242, 244, 1); border-radius: 4px">mongo-db-sidecar</code>使用的是Replica set模式，Mongodb的Replica Set即副本集方式主要有两个目的，一个是数据冗余做故障恢复使用，当发生硬件故障或者其它原因造成的宕机时，可以使用副本进行恢复。另一个是做读写分离，读的请求分流到副本上，减轻主（Primary）的读压力。 二进制部署MongoDB集群无需其他服务，直接在主节点执行类似以下的命令即可创建集群:
<pre><code class="hljs javascript" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">cfg={ _id:"testdb", members:[ {_id:0,host:'192.168.255.141:27017',priority:2}, {_id:1,host:'192.168.255.142:27017',priority:1}, {_id:2,host:'192.168.255.142:27019',arbiterOnly:true}] };
rs.initiate(cfg)</code></pre>
<hr style="box-sizing: content-box; height: 0; margin: 1.5em auto; border-top: 2px dotted rgba(238, 238, 238, 1); border-right: 0; border-bottom: 0; border-left: 0; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">
<h3 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 20px">2. 部署</h3>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">本文是部署Mongodb的实践，因为此服务需要用到<code style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 12.6px; padding: 2px 4px; color: rgba(199, 37, 78, 1); background-color: rgba(249, 242, 244, 1); border-radius: 4px">namespace</code>下的<code style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 12.6px; padding: 2px 4px; color: rgba(199, 37, 78, 1); background-color: rgba(249, 242, 244, 1); border-radius: 4px">pods</code>的<code style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 12.6px; padding: 2px 4px; color: rgba(199, 37, 78, 1); background-color: rgba(249, 242, 244, 1); border-radius: 4px">list</code>权限进行集群操作，所以如果在实际部署时，请记得先进行2.5的RBAC操作，然后再进行2.4的Statefulset部署。
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">2.1 Namespace</h4>
<pre><code class="hljs sql" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">kubectl create ns mongo</code></pre>
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">2.2 StorageClass</h4>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">这里需要提前部署好NFS或者其他可提供SC的存储集群。 Kubernetes使用NFS做持久化存储
<pre><code class="hljs shell" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px"># mongo-clutser-sc.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: mongodb-data
provisioner: fuseim.pri/ifs

# create
kubectl create -f mongo-clutser-sc.yaml</code></pre>
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">2.3 Headless Service</h4>
<pre><code class="hljs python" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">apiVersion: v1
kind: Service
metadata:
name: mongo
namespace: mongo
labels:
name: mongo
spec:
ports:
- port: 27017
targetPort: 27017
clusterIP: None
selector:
role: mongo</code></pre>
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">2.4 Statefulset</h4>
<pre><code class="hljs cpp" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: mongo
namespace: mongo
spec:
serviceName: "mongo"
replicas: 3
template:
metadata:
 labels:
 role: mongo
 environment: prod
spec:
 terminationGracePeriodSeconds: 10
 containers:
 - name: mongo
 image: harbor.s.com/redis/mongo:3.4.22
 command:
 - mongod
 - "--replSet"
 - rs0
 - "--bind_ip"
 - 0.0.0.0
 - "--smallfiles"
 - "--noprealloc"
 ports:
 - containerPort: 27017
 volumeMounts:
 - name: mongo-persistent-storage
 mountPath: /data/db
 - name: mongo-sidecar
 image: harbor.s.com/redis/mongo-k8s-sidecar
 env:
 - name: MONGO_SIDECAR_POD_LABELS
 value: "role=mongo,environment=prod"
volumeClaimTemplates:
- metadata:
 name: mongo-persistent-storage
spec:
 accessModes: ["ReadWriteOnce"]
 storageClassName: mongodb-data
 resources:
 requests:
 storage: 10Gi</code></pre>
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">2.5 RBAC</h4>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">这时候查看集群状态，发现是不可用的。
<pre><code class="hljs sql" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">kubectl exec -it mongo-0 -n mongo -- mongo
Defaulting container name to mongo.
Use 'kubectl describe pod/mongo-0 -n mongo' to see all of the containers in this pod.
MongoDB shell version v3.4.22
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.22
Server has startup warnings:
2019-08-24T09:23:57.039+0000 I CONTROL
2019-08-24T09:23:57.039+0000 I CONTROL ** WARNING: Access control is not enabled for the database.
2019-08-24T09:23:57.039+0000 I CONTROL ** Read and write access to data and configuration is unrestricted.
2019-08-24T09:23:57.039+0000 I CONTROL ** WARNING: You are running this process as the root user, which is not recommended.
2019-08-24T09:23:57.039+0000 I CONTROL
2019-08-24T09:23:57.040+0000 I CONTROL
2019-08-24T09:23:57.040+0000 I CONTROL ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2019-08-24T09:23:57.040+0000 I CONTROL ** We suggest setting it to 'never'
2019-08-24T09:23:57.040+0000 I CONTROL
2019-08-24T09:23:57.040+0000 I CONTROL ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2019-08-24T09:23:57.040+0000 I CONTROL ** We suggest setting it to 'never'
2019-08-24T09:23:57.040+0000 I CONTROL
> rs.status()
{
 "info" : "run rs.initiate(...) if not yet done for the set",
 "ok" : 0,
 "errmsg" : "no replset config has been received",
 "code" : 94,
 "codeName" : "NotYetInitialized"
}
> </code></pre>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">应该是<code style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 12.6px; padding: 2px 4px; color: rgba(199, 37, 78, 1); background-color: rgba(249, 242, 244, 1); border-radius: 4px">mongo k8s sidecar</code>没有正确的配置，查看其日志：
<pre><code class="hljs javascript" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">kubectl logs mongo-0 mongo-sidecar -n mongo

···
Error in workloop { [Error: ]
message:
{ kind: 'Status',
apiVersion: 'v1',
metadata: {},
status: 'Failure',
message:
 'pods is forbidden: User "system:serviceaccount:mongo:default" cannot list resource "pods" in API group "" at the cluster scope',
reason: 'Forbidden',
details: { kind: 'pods' },
code: 403 },
statusCode: 403 }</code></pre>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">信息显示默认分配的sa账号没有list此namespace下pods的权限，搜索了下这个问题早在很久之前在github上就有人提出，作者也给出了对应的解决方案，需要给默认的sa账号提权，增加list pods的权限，但是实际测试发现虽然给<code style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: 12.6px; padding: 2px 4px; color: rgba(199, 37, 78, 1); background-color: rgba(249, 242, 244, 1); border-radius: 4px">system:serviceaccount:mongo:dafault</code>赋予pods的list权限，仍然会报错，以下是rbac配置： mongo-k8s-sidecar/role.yaml at 2640ed1c2971b1279c2961efd257cde9fbe39574 · cvallance/mongo-k8s-sidecar · GitHub
<pre><code class="hljs cpp" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px"># 使用后仍然无用的配置
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: mongo
name: mongo-pod-read
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: mongo-pod-read
namespace: mongo
subjects:
- kind: ServiceAccount
name: default
namespace: mongo
roleRef:
kind: Role
name: mongo-pod-read
apiGroup: rbac.authorization.k8s.io</code></pre>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">所以我们需要重新想办法，给此sa更大的权限，这里使用默认的clusterrole view权限进行赋权，我们可以使用clusterrole对sa进行界定namespace的赋权，相当于clusterrole是一个可以进行clusterrole与role进行binding的模板： GCE - K8s 1.8 - pods is forbidden - Cannot list pods - Unknown user "system:serviceaccount:default:default" · Issue #75 · cvallance/mongo-k8s-sidecar · GitHub
<pre><code class="hljs cs" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px"># 正确的rbac配置
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: mongo-default-view
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: default
namespace: mongo</code></pre>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">但是pod在创建后，是无法更换sa账号与sa权限的，所以需要重建pod:
<pre><code class="hljs shell" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px"># 查看statefulset
kubectl get statefulset -n mongo
NAME READY AGE
mongo 3/3 23h

# scale
kubectl scale statefulset mongo -n mongo --replicas=0
statefulset.apps/mongo scaled

# 过会重新配置副本数为3
kubectl scale statefulset mongo -n mongo --replicas=3
statefulset.apps/mongo scaled

# 查看已经建立完毕
kubectl get all -n mongo
NAME READY STATUS RESTARTS AGE
pod/mongo-0 2/2 Running 0 21s
pod/mongo-1 2/2 Running 0 17s
pod/mongo-2 2/2 Running 0 12s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/mongo ClusterIP None <none> 27017/TCP 23h

NAME READY AGE
statefulset.apps/mongo 3/3 23h
</code></pre>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">再次查看集群状态，发现状态已经正常，集群创建成功：
<pre><code class="hljs bash" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">kubectl exec -it mongo-0 -n mongo -- mongo
rs0:PRIMARY> rs.status()
{
 "set" : "rs0",
 "date" : ISODate("2019-08-25T08:58:12.550Z"),
 "myState" : 1,
 "term" : NumberLong(2),
 "syncingTo" : "",
 "syncSourceHost" : "",
 "syncSourceId" : -1,
 "heartbeatIntervalMillis" : NumberLong(2000),
 "optimes" : {
 "lastCommittedOpTime" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 },
 "appliedOpTime" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 },
 "durableOpTime" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 }
 },
 "members" : [
 {
 "_id" : 0,
 "name" : "10.244.4.87:27017",
 "health" : 1,
 "state" : 2,
 "stateStr" : "SECONDARY",
 "uptime" : 19,
 "optime" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 },
 "optimeDurable" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 },
 "optimeDate" : ISODate("2019-08-25T08:58:05Z"),
 "optimeDurableDate" : ISODate("2019-08-25T08:58:05Z"),
 "lastHeartbeat" : ISODate("2019-08-25T08:58:11.877Z"),
 "lastHeartbeatRecv" : ISODate("2019-08-25T08:58:11.192Z"),
 "pingMs" : NumberLong(0),
 "lastHeartbeatMessage" : "",
 "syncingTo" : "10.244.3.65:27017",
 "syncSourceHost" : "10.244.3.65:27017",
 "syncSourceId" : 3,
 "infoMessage" : "",
 "configVersion" : 171757
 },
 {
 "_id" : 1,
 "name" : "10.244.5.9:27017",
 "health" : 1,
 "state" : 2,
 "stateStr" : "SECONDARY",
 "uptime" : 19,
 "optime" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 },
 "optimeDurable" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 },
 "optimeDate" : ISODate("2019-08-25T08:58:05Z"),
 "optimeDurableDate" : ISODate("2019-08-25T08:58:05Z"),
 "lastHeartbeat" : ISODate("2019-08-25T08:58:11.875Z"),
 "lastHeartbeatRecv" : ISODate("2019-08-25T08:58:11.478Z"),
 "pingMs" : NumberLong(0),
 "lastHeartbeatMessage" : "",
 "syncingTo" : "10.244.4.87:27017",
 "syncSourceHost" : "10.244.4.87:27017",
 "syncSourceId" : 0,
 "infoMessage" : "",
 "configVersion" : 171757
 },
 {
 "_id" : 3,
 "name" : "10.244.3.65:27017",
 "health" : 1,
 "state" : 1,
 "stateStr" : "PRIMARY",
 "uptime" : 80,
 "optime" : {
 "ts" : Timestamp(1566723485, 1),
 "t" : NumberLong(2)
 },
 "optimeDate" : ISODate("2019-08-25T08:58:05Z"),
 "syncingTo" : "",
 "syncSourceHost" : "",
 "syncSourceId" : -1,
 "infoMessage" : "could not find member to sync from",
 "electionTime" : Timestamp(1566723473, 1),
 "electionDate" : ISODate("2019-08-25T08:57:53Z"),
 "configVersion" : 171757,
 "self" : true,
 "lastHeartbeatMessage" : ""
 }
 ],
 "ok" : 1
}
rs0:PRIMARY> </code></pre>
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">2.6 扩容</h4>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">如果需要对mongo扩容，只需要调整statefulset的replicas即可：
<pre><code class="hljs sql" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">kubectl scale statefulset mongo --replicas=4 -n mongo</code></pre>
<hr style="box-sizing: content-box; height: 0; margin: 1.5em auto; border-top: 2px dotted rgba(238, 238, 238, 1); border-right: 0; border-bottom: 0; border-left: 0; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">
<h3 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 20px">3. 使用/访问</h3>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">mongo cluster访问默认连接为：
<pre><code class="hljs cpp" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">mongodb://mongo1,mongo2,mongo3:27017/dbname_?</code></pre>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">在kubernetes中最常用的FQDN连接服务的连接为：
<pre><code class="hljs shell" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">#appName.$HeadlessServiceName.$Namespace.svc.cluster.local</code></pre>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">因为我们采用statefulset部署的pod，所以命名均有规则，所以实际上如果连接4副本的mongodb cluster，上面的默认连接该为（默认为namespace之外）：
<pre><code class="hljs cpp" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px">mongodb://mongo-0.mongo.mongo.svc.cluster.local:27017,mongo-1.mongo.mongo.svc.cluster.local:27017,mongo-2.mongo.mongo.svc.cluster.local:27017,mongo-3.mongo.mongo.svc.cluster.local:27017/?replicaSet=rs0</code></pre>
<hr style="box-sizing: content-box; height: 0; margin: 1.5em auto; border-top: 2px dotted rgba(238, 238, 238, 1); border-right: 0; border-bottom: 0; border-left: 0; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">
<h3 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 20px">4. 监控</h3>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">使用helm chart prometheus-mongodb-exporter进行监控。
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">4.1 部署exporter</h4>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">注意，这里的uri后如果是集群，必须使用“”，不然会各种告警，我在这踩了无数的坑 看起来uri是固定的，而不是自动发现，所以如果需要对集群的副本进行增加或删除，则需要helm修改uri，更新配置后重建pod。 image是为了内网容易部署，将默认image下载后放入harbor，并未做任何其他修改，可忽略。
<pre><code class="hljs python" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px"># vi values.yaml 编辑定制参数
mongodb:
uri: "mongodb://mongo-0.mongo.mongo.svc.cluster.local:27017,mongo-1.mongo.mongo.svc.cluster.local:27017,mongo-2.mongo.mongo.svc.cluster.local:27017,mongo-3.mongo.mongo.svc.cluster.local:27017/?replicaSet=rs0"
image:
repository: harbor.s.com/mongo/mongodb-exporter
tag: 0.7.0</code></pre>
<pre><code class="hljs nginx" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px"># 部署
helm upgrade --install mongo-exporter stable/prometheus-mongodb-exporter -f values.yaml --namespace mongo --force</code></pre>
<pre><code class="hljs nginx" style="box-sizing: border-box; font-family: Menlo, Monaco, Consolas, "Courier New", monospace; font-size: inherit; padding: 0; color: rgba(0, 0, 0, 1); background-color: rgba(0, 0, 0, 0); border-radius: 0; overflow-wrap: normal; display: block; overflow-x: auto; position: relative; margin: 10px 10px 1px 50px"># 查看结果
kubectl port-forward service/mongo-exporter-prometheus-mongodb-exporter 9216
curl http://127.0.0.1:9216/metrics</code></pre>
<h4 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 15px">4.2 配置prometheus operator</h4>
<hr style="box-sizing: content-box; height: 0; margin: 1.5em auto; border-top: 2px dotted rgba(238, 238, 238, 1); border-right: 0; border-bottom: 0; border-left: 0; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">
<h3 style="box-sizing: border-box; font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace; line-height: 1.4; color: rgba(102, 102, 102, 1); margin: 30px 0 10px; font-size: 20px">感谢：</h3>
<p style="box-sizing: border-box; margin: 0 0 20px; word-break: break-all; color: rgba(88, 102, 110, 1); font-family: "Source Sans Pro", "Hiragino Sans GB", "Microsoft Yahei", SimSun, Helvetica, Arial, sans-serif, monospace">1.Kubernetes RBAC 详解-www.qikqiak.com|阳明的博客|Kubernetes|Docker|istio|Python|Golang|Cloud Native 2.使用 StatefulSet 搭建 MongoDB 集群 - Kubernetes - Wiki.Shileizcc.com 3.GitHub - cvallance/mongo-k8s-sidecar: Kubernetes sidecar for Mongo 4.MongoDB集群搭建及使用 - SuperMap技术控 - CSDN博客 
来源：https://www.cnblogs.com/evenchen/p/11936706.html

頁: [1]

琼殿技术论坛's Archiver

Kubernetes部署高可用MongoDB集群