6.prometheus监控-监控redis/rabbitmq/mongodb

油桐树 發表於 2024-4-24 18:44:00

6.prometheus监控--监控redis/rabbitmq/mongodb

<h3>1.监控redis</h3>
<h4>1.1 redis_exporter安装方式</h4>
<h5>1.1.1 二进制源码安装方式</h5>
参考nginx二进制安装方法
<div class="cnblogs_code">
<pre>redis_exporter下载地址：https://github.com/oliver006/redis_exporter/releases

系统服务：
cat > /etc/systemd/system/redis_exporter.service <<"EOF"

Description=Prometheus Redis Exporter
After=network.target


Type=simple
User=prometheus
Group=prometheus
Restart=always
ExecStart=/opt/prometheus/redis_exporter/redis_exporter \
-redis.addr localhost:6379 \
-redis.password 123456

WantedBy=multi-user.target
EOF</pre>
</div>
<h5>1.1.2 docker安装</h5>
<div class="cnblogs_code">
<pre># docker直接运行
docker run -d --restart=always --name redis_exporter -p 9121:9121 oliver006/redis_exporter --redis.addr redis://192.168.10.100:6379 --redis.password '123456'

# docker-compose方式
cat >docker-compose.yaml <<EOF
version: '3.3'
services:
redis_exporter:
image: oliver006/redis_exporter
container_name: redis_exporter
restart: always
environment:
 REDIS_ADDR: "192.168.10.100:6379"
 REDIS_PASSWORD: 123456
ports:
 - "9121:9121"
EOF

# 启动
docker-compose up -d

# metrics地址
http://192.168.10.100:9121/metrics</pre>
</div>
<h4>1.2 Prometheus配置</h4>
<div class="cnblogs_code">
<pre># 配置prometheus去采集（拉取）redis_exporter的监控样本数据

cd /data/docker-prometheus

#在scrape_configs(搜刮配置):下面增加如下配置：

cat >> prometheus/prometheus.yml << "EOF"
- job_name: 'redis_exporter'
static_configs:
- targets: ['192.168.10.100:9121']
 labels:
 instance: test服务器
EOF

# 重载
curl -X POST http://localhost:9090/-/reload</pre>
</div>
<img src="https://img2024.cnblogs.com/blog/1523753/202404/1523753-20240424175611847-941202725.png">
<h4> 1.3 granfa展示</h4>
https://grafana.com/grafana/dashboards/11835-redis-dashboard-for-prometheus-redis-exporter-helm-stable-redis-ha/
<img src="https://img2024.cnblogs.com/blog/1523753/202404/1523753-20240424175822455-1927963402.png">
<h4> 1.5 常用监控指标</h4>
<div class="cnblogs_code">
<pre>redis_up # 服务器是否在线
redis_uptime_in_seconds # 运行时长，单位 s
rate(redis_cpu_sys_seconds_total) + rate(redis_cpu_user_seconds_total) # 占用 CPU 核数
redis_memory_used_bytes # 占用内存量
redis_memory_max_bytes # 限制的最大内存，如果没限制则为 0
delta(redis_net_input_bytes_total) # 网络接收的 bytes
delta(redis_net_output_bytes_total) # 网络发送的 bytes

redis_connected_clients # 客户端连接数
redis_connected_clients / redis_config_maxclients # 连接数使用率
redis_rejected_connections_total # 拒绝的客户端连接数
redis_connected_slaves # slave 连接数</pre>
</div>
<h4>1.6 触发器配置</h4>
将触发器根据服务不同，进行分开，避免规则列表过长
<div class="cnblogs_code">
<pre>cd /data/docker-prometheus
mkdir prometheus/rules
vim prometheus/prometheus.yml
# 报警(触发器)配置
rule_files:
- "alert.yml"
- "rules/*.yml" </pre>
</div>
<div class="lake-content">
redis触发器（告警规则）
<div class="cnblogs_code"><img src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_3849f796-3f0a-458a-8996-268df7e2cdc0" class="code_img_opened lazyload" style="display: none" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_3849f796-3f0a-458a-8996-268df7e2cdc0" class="cnblogs_code_hide">
<pre>cat >> prometheus/rules/redis.yml <<"EOF"
groups:
- name: redis
rules:
- alert: RedisDown
expr: redis_up == 0
for: 0m
labels:
 severity: critical
annotations:
 summary: 'Redis Down,实例:{{ $labels.instance }}'
 description: "Redis实例 is down"
- alert: RedisMissingBackup
expr: time() - redis_rdb_last_save_timestamp_seconds > 60 * 60 * 24
for: 0m
labels:
 severity: critical
annotations:
 summary: "Redis备份丢失,实例:{{ $labels.instance }}"
 description: "Redis 24小时未备份"

- alert: RedisOutOfConfiguredMaxmemory
expr: redis_memory_used_bytes / redis_memory_max_bytes * 100 > 90
for: 2m
labels:
 severity: warning
annotations:
 summary: "Redis超出配置的最大内存,实例:{{ $labels.instance }}"
 description: "Redis内存使用超过配置最大内存的90%"
- alert: RedisTooManyConnections
expr: redis_connected_clients > 100
for: 2m
labels:
 severity: warning
annotations:
 summary: "Redis连接数过多,实例:{{ $labels.instance }}"
 description: "Redis当前连接数为： {{ $value }}"
- alert: RedisNotEnoughConnections
expr: redis_connected_clients < 1
for: 2m
labels:
 severity: warning
annotations:
 summary: "Redis没有足够的连接,实例:{{ $labels.instance }}"
 description: "Redis当前连接数为： {{ $value }}"
- alert: RedisRejectedConnections
expr: increase(redis_rejected_connections_total) > 0
for: 0m
labels:
 severity: critical
annotations:
 summary: "Redis有拒绝连接,实例:{{ $labels.instance }}"
 description: "与Redis 的某些连接被拒绝{{ $value }}"
EOF</pre>
</div>
redis告警规则</div>
检查配置，重新加载
docker exec -it prometheus promtool check config /etc/prometheus/prometheus.yml
curl -X POST http://localhost:9090/-/reload
检查：
http://192.168.10.14:9090/rules#redis
http://192.168.10.14:9090/alerts?search=
<img alt="" loading="lazy" src="https://img2024.cnblogs.com/blog/1523753/202404/1523753-20240424181643895-1013069436.png">
<h3> 2.监控rabbitmq</h3>
<h4> 2.1 安装方式</h4>
rabbitmq_exporter二进制安装
<div class="lake-content">
<div class="cnblogs_code">
<pre>rabbit_exporter下载地址: https://github.com/kbudde/rabbitmq_exporter/releases

系统服务：
cat > /etc/systemd/system/rabbitmq_exporter.service <<"EOF"

Description=prometheus rabbitmq exporter
After=network.target


Environment=RABBIT_USER=guest
Environment=RABBIT_PASSWORD=guest
Environment=RABBIT_URL=http://localhost:15672
OUTPUT_FORMAT=JSON
Type=simple
User=prometheus
Group=prometheus
Restart=always
ExecStart=/opt/prometheus/rabbitmq_exporter/rabbitmq_exporter


WantedBy=multi-user.target
EOF</pre>
</div>
</div>
docker-compose安装rabbitmq
<div class="cnblogs_code">
<pre>cd /data/rabbitmq
cat >>docker-compose.yaml<<"EOF"

version: '3'
services:
rabbitmq:
image: rabbitmq:3.7.15-management
container_name: rabbitmq
restart: always
volumes:
 - /data/rabbitmq/data:/var/lib/rabbitmq
 - /data/rabbitmq/log:/var/log/rabbitmq
ports:
 - 5672:5672
 - 15672:15672
EOF

# 启动
docker-compose up -d</pre>
</div>
<h4>2.2 安装 rabbitmq_exporter方式</h4>
docker安装rabbitmq_exporter
</div>
<div class="cnblogs_code">
<pre># docker直接安装
docker run -d --restart=always -p 9419:9419--name rabbitmq_exporter -e RABBIT_URL=http://192.168.10.100:15672 -e RABBIT_USER=guest -e RABBIT_PASSWORD=guest kbudde/rabbitmq-exporter

# docker-compose安装
cat >docker-compose.yaml <<EOF
version: '3.3'
services:
rabbitmq_exporter:
image: kbudde/rabbitmq-exporter
container_name: rabbitmq_exporter
restart: always
environment:
 RABBIT_URL: "http://192.168.10.100:15672"
 RABBIT_USER: "guest"
 RABBIT_PASSWORD: "guest"
 PUBLISH_PORT: "9419"
 OUTPUT_FORMAT: "JSON"
ports:
 - "9419:9419"
EOF

# 启动
docker-compose up -d # 参数解释</pre>
</div>
<div class="lake-content">
<table class="ne-table" style="height: 283px; width: 764px" align="left">
<tbody>
<tr>
<td width="250">
Environment variable
</td>
<td width="250">
default
</td>
<td width="250">
description
</td>
</tr>
<tr>
<td width="250">
RABBIT_URL
</td>
<td width="250">
http://127.0.0.1:15672
</td>
<td width="250">
rabbitMQ管理插件的url（必须以http(s)://开头）
</td>
</tr>
<tr>
<td width="250">
RABBIT_USER
</td>
<td width="250">
guest
</td>
<td width="250">
rabbitMQ 管理插件的用户名。
</td>
</tr>
<tr>
<td width="250">
RABBIT_PASSWORD
</td>
<td width="250">
guest
</td>
<td width="250">
rabbitMQ 管理插件的密码。
</td>
</tr>
<tr>
<td width="250">
OUTPUT_FORMAT
</td>
<td width="250">
JSON
</td>
<td width="250">
输出格式
</td>
</tr>
<tr>
<td width="250">
PUBLISH_PORT
</td>
<td width="250">
9419
</td>
<td width="250">
运行端口(监听端口)
</td>
</tr>
</tbody>
</table>
</div>
 
 
 
 
 
 
 
 
 
 
metrics地址：http://192.168.10.100:9419/metrics
<h4>2.3 Prometheus配置</h4>
<div class="lake-content">
配置prometheus去采集（拉取）rabbitmq_exporter的监控样本数据
<div class="cnblogs_code">
<pre>cd /data/docker-prometheus

#在scrape_configs(搜刮配置):下面增加如下配置：

cat >> prometheus/prometheus.yml << "EOF"
- job_name: 'rabbitmq_exporter'
static_configs:
- targets: ['192.168.10.100:9419']
 labels:
 instance: test服务器
EOF

# 重新加载配置
curl -X POST http://localhost:9090/-/reload</pre>
</div>
<img src="https://img2024.cnblogs.com/blog/1523753/202404/1523753-20240424183247882-359276597.png">
<h4> 2.4 常用监控指标</h4>
<div class="cnblogs_code">
<pre>rabbitmq_queue_messages_unacknowledged_global 队列中有未确认的消息总数（未被消费的消息）

rabbitmq_node_disk_free_limit使用磁盘大小
rabbitmq_node_disk_free 磁盘总大小

rabbitmq_node_mem_used 使用内存大小
rabbitmq_node_mem_limit 内存总大小

rabbitmq_sockets_used 使用sockets的数量
rabbitmq_sockets_available 可用的sockets总数

rabbitmq_fd_used 使用文件描述符的数量
rabbitmq_fd_available 可用的文件描述符总数</pre>
</div>
<h4>2.5 rabbitmq触发器告警规则</h4>
<div class="cnblogs_code"><img src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_4c5c46e6-9406-44f0-9c9f-7a5ffb432c3d" class="code_img_opened lazyload" style="display: none" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_4c5c46e6-9406-44f0-9c9f-7a5ffb432c3d" class="cnblogs_code_hide">
<pre>cat > prometheus/rules/rabbitmq.yml <<"EOF"
groups:
- name: Rabbitmq
rules:
- alert: RabbitMQDown
expr: rabbitmq_up != 1
labels:
 severity: High
annotations:
 summary: "Rabbitmq Down,实例:{{ $labels.instance }}"
 description: "Rabbitmq_exporter连不上RabbitMQ! ! !"
- alert: RabbitMQ有未确认消息
expr: rabbitmq_queue_messages_unacknowledged_global> 0
for: 1m
labels:
 severity: critical
annotations:
 summary: "RabbitMQ有未确认消息,实例:{{ $labels.instance }}"
 description: 'RabbitMQ未确认消息>0,当前值为：{{ $value }}'
- alert: RabbitMQ可用磁盘空间不足告警
expr: rabbitmq_node_disk_free_alarm != 0
#expr: rabbitmq_node_disk_free_limit / rabbitmq_node_disk_free *100 > 90
for: 0m
labels:
 severity: critical
annotations:
 summary: "RabbitMQ可用磁盘空间不足,实例:{{ $labels.instance }}"
 description: "RabbitMQ可用磁盘空间不足，请检查"
- alert: RabbitMQ可用内存不足告警
expr: rabbitmq_node_mem_alarm != 0
#expr: rabbitmq_node_mem_used / rabbitmq_node_mem_limit * 100 > 90
for: 0m
labels:
 severity: critical
annotations:
 summary: "RabbitMQ可用内存不足,实例:{{ $labels.instance }}"
 description: "RabbitMQ可用内存不足，请检查"
- alert: RabbitMQ_socket连接数使用过高告警
expr: rabbitmq_sockets_used / rabbitmq_sockets_available * 100 > 60
for: 0m
labels:
 severity: critical
annotations:
 summary: "RabbitMQ_socket连接数使用过高,实例:{{ $labels.instance }}"
 description: 'RabbitMQ_sockets使用>60%,当前值为：{{ $value }}'
- alert: RabbitMQ文件描述符使用过高告警
expr: rabbitmq_fd_used / rabbitmq_fd_available * 100 > 60
for: 0m
labels:
 severity: critical
annotations:
 summary: "RabbitMQ文件描述符使用过高,实例:{{ $labels.instance }}"
 description: 'RabbitMQ文件描述符使用>60%,当前值为：{{ $value }}'
EOF</pre>
</div>
rabbitmq报警规则</div>
<div class="cnblogs_code">
<pre># 检查配置
docker exec -it prometheus promtool check config /etc/prometheus/prometheus.yml

#重新加载配置
curl -X POST http://localhost:9090/-/reload

# 查看
http://192.168.10.14:9090/rules
http://192.168.10.14:9090/alerts?search=</pre>
</div>
<h4>2.6 grafana dashboard展示</h4>
<div class="lake-content">
grafana展示prometheus从rabbitmq_exporter收集到的的数据
id:4279
</div>
</div>
<img alt="" loading="lazy" src="https://img2024.cnblogs.com/blog/1523753/202404/1523753-20240424184339903-1584281263.png">
<h3> 3.监控mongodb</h3>
安装方式自己定
<h4>3.1 docker-compose安装mongo</h4>
<div class="cnblogs_code">
<pre>cd /data/mongodb

cat >>docker-compose.yaml<<"EOF"
version: '3'
services:
mongo:
image: mongo:4.2.5
container_name: mongo
restart: always
volumes:
 - /data/mongo/db:/data/db
ports:
 - 27017:27017
command: [--auth]
environment:
 MONGO_INITDB_ROOT_USERNAME: root
 MONGO_INITDB_ROOT_PASSWORD: 123456
EOF

docker-compose up -d</pre>
</div>
<h4>3.2  监控mongodb</h4>
<h5>3.2.1 创建监控用户</h5>
<div class="lake-content">
登陆mongodb创建监控用户，权限为“readAnyDatabase”，如果是cluster环境，需要有权限“clusterMonitor”
docker exec -it mongo mongo admin
创建监控用户
<div class="cnblogs_code">
<pre>> db.auth('root','123456')
1
> db.createUser({ user:'exporter',pwd:'password',roles:[ { role:'readAnyDatabase', db: 'admin'},{ role: "clusterMonitor", db: "admin" }]});
#测试使用上面创建的用户信息进行连接。
> db.auth('exporter', 'password')
1
#表示成功
> exit</pre>
</div>
<h5>3.2.2 安装mongodb_exporter</h5>
<h6>3.2.2.1 mongodb_exporter安装方式自选</h6>
二进制安装：
<div class="lake-content">
mongodb_exporter地址:https://github.com/percona/mongodb_exporter/releases
或：https://github.com/prometheus/mysqld_exporter/releases
systemd服务
<div class="cnblogs_code">
<pre>cat <<EOF >/usr/lib/systemd/system/mongodb_exporter.service

Description=mongodb_exporter
Documentation=https://github.com/percona/mongodb_exporter
After=network.target

Type=simple
User=prometheus
Environment="MONGODB_URI=mongodb://exporter:password@localhost:27017/admin"
ExecStart=/opt/prometheus/mongodb_exporter/mongodb_exporter --log.level=error --collect-all --compatible-mode
Restart=on-failure

WantedBy=multi-user.target
EOF</pre>
</div>
docker 安装方式：
<div class="cnblogs_code">
<pre>docker run -d --restart=always -p 9216:9216 -p 17001:17001 --restart=always --name=mongodb-exporterbitnami/mongodb-exporter:latest --collect-all --compatible-mode --mongodb.uri=mongodb://exporter:password@192.168.10.100:27017/admin?ssl=false</pre>
</div>
docker-compose安装方式
<div class="cnblogs_code">
<pre>cat >docker-compose.yaml <<EOF
version: '3.3'
services:
mongodb_exporter:
image: bitnami/mongodb-exporter:latest
container_name: mongodb_exporter
restart: always
environment:
 MONGODB_URI: "mongodb://exporter:password@192.168.11.62:27017/admin?ssl=false"
command:
 - '--collect-all'
 - '--compatible-mode'
ports:
 - "9216:9216"
EOF</pre>
</div>
参数解释
<div class="lake-content">
<table id="c1ed3f8c" class="ne-table">
<tbody>
<tr>
<td width="250">
Flag
</td>
<td width="250">
含义
</td>
<td width="250">
案例
</td>
</tr>
<tr>
<td width="250">
-h, --help
</td>
<td width="250">
显示上下文相关的帮助
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--compatible-mode
</td>
<td width="250">
启用旧的 mongodb-exporter 兼容指标
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--discovering-mode
</td>
<td width="250">
启用自动发现集合
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--mongodb.collstats-colls
</td>
<td width="250">
逗号分隔的 databases.collections 列表以获取 $collStats
</td>
<td width="250">
--mongodb.collstats-colls=db1,db2.col2
</td>
</tr>
<tr>
<td width="250">
--mongodb.indexstats-colls
</td>
<td width="250">
逗号分隔的 databases.collections 列表以获取 $indexStats
</td>
<td width="250">
--mongodb.indexstats-colls=db1.col1,db2.col2
</td>
</tr>
<tr>
<td width="250">
--mongodb.direct-connect
</td>
<td width="250">
是否应该进行直接连接。如果指定了多个主机或使用了 SRV URI，则直接连接无效
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--mongodb.global-conn-pool
</td>
<td width="250">
使用全局连接池而不是为每个 http 请求创建新池
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--mongodb.uri
</td>
<td width="250">
MongoDB 连接 URI ($MONGODB_URI)
</td>
<td width="250">
--mongodb.uri=mongodb://user:pass@127.0.0.1:27017/admin?ssl=true
</td>
</tr>
<tr>
<td width="250">
--web.listen-address
</td>
<td width="250">
用于侦听 Web 界面和遥测的地址
</td>
<td width="250">
--web.listen-address=":9216"
</td>
</tr>
<tr>
<td width="250">
--web.telemetry-path
</td>
<td width="250">
指标公开路径
</td>
<td width="250">
--web.telemetry-path="/metrics"
</td>
</tr>
<tr>
<td width="250">
--web.config
</td>
<td width="250">
具有用于基本身份验证的 Prometheus TLS 配置的文件的路径
</td>
<td width="250">
--web.config=STRING
</td>
</tr>
<tr>
<td width="250">
--log.level
</td>
<td width="250">
仅记录具有给定严重性或更高严重性的消息。有效级别：[调试、信息、警告、错误、致命]
</td>
<td width="250">
--log.level="error"
</td>
</tr>
<tr>
<td width="250">
--collector.diagnosticdata
</td>
<td width="250">
启用从 getDiagnosticData 收集指标
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--collector.replicasetstatus
</td>
<td width="250">
启用从 replSetGetStatus 收集指标
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--collector.dbstats
</td>
<td width="250">
启用从 dbStats 收集指标
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--collector.topmetrics
</td>
<td width="250">
启用从 top admin command 收集指标
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--collector.indexstats
</td>
<td width="250">
启用从 $indexStats 收集指标
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--collector.collstats
</td>
<td width="250">
启用从 $collStats 收集指标
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--collect-all
</td>
<td width="250">
启用所有收集器。与指定所有 --collector. 相同
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--collector.collstats-limit=0
</td>
<td width="250">
如果有超过个集合，请禁用 collstats、dbstats、topmetrics 和 indexstats 收集器。0=无限制
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--metrics.overridedescendingindex
</td>
<td width="250">
启用降序索引名称覆盖以将 -1 替换为 _DESC
</td>
<td width="250"> </td>
</tr>
<tr>
<td width="250">
--version
</td>
<td width="250">
显示版本并退出
</td>
<td width="250"> </td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
metrics地址：http://192.168.10.100:9216/metrics
<h4>3.3 Prometheus配置</h4>
<div class="lake-content">
配置prometheus去采集（拉取）mongodb_exporter的监控样本数据
<div class="cnblogs_code">
<pre>cd /data/docker-prometheus

#在scrape_configs(搜刮配置):下面增加如下配置：

cat >> prometheus/prometheus.yml << "EOF"
- job_name: 'mongodb_exporter'
static_configs:
- targets: ['192.168.10.100:9216']
 labels:
 instance: test服务器
EOF</pre>
</div>
检查：
<img src="https://img2024.cnblogs.com/blog/1523753/202404/1523753-20240424185859877-1667283778.png">
<h4> 3.4 常用的监控指标</h4>
<div class="cnblogs_code">
<pre>mongodb_ss_connections{conn_type="available"} 可用的连接总数

mongodb_ss_mem_virtual
mongodb_ss_mem_resident

# 关于 server status
mongodb_up # 服务器是否在线
mongodb_ss_ok{cl_id="", cl_role="mongod", rs_state="0"} # 服务器是否正常运行，取值为 1、0 。标签中记录了 Cluster、ReplicaSet 的信息
mongodb_ss_uptime # 服务器的运行时长，单位为秒
mongodb_ss_connections{conn_type="current"} # 客户端连接数

# 关于主机
mongodb_sys_cpu_num_cpus # 主机的 CPU 核数

# 关于 collection
mongodb_collstats_storageStats_count{database="xx", collection="xx"}# collection 全部文档的数量
mongodb_collstats_storageStats_size # collection 全部文档的体积，单位 bytes
mongodb_collstats_storageStats_storageSize # collection 全部文档占用的磁盘空间，默认会压缩
delta(mongodb_collstats_latencyStats_reads_ops) # collection 读操作的数量（每分钟）
delta(mongodb_collstats_latencyStats_reads_latency) # collection 读操作的延迟（每分钟），单位为微秒
mongodb_collstats_latencyStats_write_ops
mongodb_collstats_latencyStats_write_latency

# 关于 index
mongodb_collstats_storageStats_nindexes # collection 的 index 数量
mongodb_collstats_storageStats_totalIndexSize # collection 的 index 占用的磁盘空间
delta(mongodb_indexstats_accesses_ops) # index 被访问次数

# 关于操作
delta(mongodb_ss_opcounters) # 执行各种操作的数量
delta(mongodb_ss_opLatencies_latency) # 执行各种操作的延迟，单位为微秒
delta(mongodb_ss_metrics_document) # 各种文档的变化数量

# 关于锁
delta(mongodb_ss_locks_acquireCount{lock_mode="w"}) # 新加锁的数量。R 表示共享锁，W 表示独占锁，r 表示意向共享锁，w 表示意向独占锁
mongodb_ss_globalLock_currentQueue{count_type="total"}# 被锁阻塞的操作数</pre>
</div>
<h4>3.5 mongodb触发器告警规则配置</h4>
<div class="cnblogs_code"><img src="http://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif"><img id="code_img_opened_a914b98a-da35-4336-bdcd-f4d4b5fb5d99" class="code_img_opened lazyload" style="display: none" data-src="http://images.cnblogs.com/OutliningIndicators/ExpandedBlockStart.gif">
<div id="cnblogs_code_open_a914b98a-da35-4336-bdcd-f4d4b5fb5d99" class="cnblogs_code_hide">
<pre>cat >> prometheus/rules/mongodb.yml <<"EOF"
groups:
- name: PerconaMongodbExporter
rules:
- alert: MongodbDown
 expr: 'mongodb_up == 0'
 for: 0m
 labels:
 severity: critical
 annotations:
 summary: "MongoDB Down 容器: $labels.instance"
 description: "MongoDB 容器 is down, 当前值:{{ $value }}"

- alert: MongodbNumberCursorsOpen
 expr: 'mongodb_ss_metrics_cursor_open{csr_type="total"} > 10 * 1000'
 for: 2m
 labels:
 severity: warning
 annotations:
 summary: "MongoDB 数字有标打开告警容器: $labels.instance"
 description: "MongoDB 为客户端打开的游标过多 > 10k, 当前值:{{ $value }}"

- alert: MongodbCursorsTimeouts
 expr: 'increase(mongodb_ss_metrics_cursor_timedOut) > 100'
 for: 2m
 labels:
 severity: warning
 annotations:
 summary: "MongoDB 游标超时容器: $labels.instance"
 description: "太多游标超时, 当前值:{{ $value }}"

- alert: MongodbTooManyConnections
 expr: 'avg by(instance) (rate(mongodb_ss_connections{conn_type="current"})) / avg by(instance) (sum (mongodb_ss_connections) by (instance)) * 100 > 80'
 for: 2m
 labels:
 severity: warning
 annotations:
 summary: "MongoDB 太多连接容器: $labels.instance"
 description: "MongoDB 连接数 > 80%, 当前值:{{ $value }}"

- alert: MongodbVirtualMemoryUsage
 expr: '(sum(mongodb_ss_mem_virtual) BY (instance) / sum(mongodb_ss_mem_resident) BY (instance)) > 3'
 for: 2m
 labels:
 severity: warning
 annotations:
 summary: "MongoDB虚拟内存使用告警容器: $labels.instance"
 description: "虚拟内存使用过高, 当前值:{{ $value }}"
EOF</pre>
</div>
mongodb告警规则</div>
检查并重新加载配置
<div class="cnblogs_code">
<pre>docker exec -it prometheus promtool check config /etc/prometheus/prometheus.yml

curl -X POST http://localhost:9090/-/reload</pre>
</div>
页面检查：
http://192.168.10.14:9090/alerts?search=
http://192.168.10.14:9090/rules
<h4>3.6 grafana dashboard展示</h4>
<div class="lake-content">
grafana展示prometheus从mongodb_exporter收集到的的数据
https://github.com/percona/grafana-dashboards/tree/main/dashboards/MongoDB
<img alt="" loading="lazy" src="https://img2024.cnblogs.com/blog/1523753/202404/1523753-20240424190421388-1820930961.png">
 
</div>
 
 
</div>
  
来源：https://www.cnblogs.com/yangmeichong/p/18156069

頁: [1]

琼殿技术论坛's Archiver

6.prometheus监控--监控redis/rabbitmq/mongodb