监控平台
监控说明
在以往的企业项目部署实施中，我们发现绝大客户拥有自己独立的监控平台，所以我们并没有把监控平台集成到我们的环境中。但也有部分用户缺少这方面的环境，如果您 有监控需求，我们为您提供了一套简单可视化的监控平台，您可以参考以下方法安装
监控指标
硬件监控
| 名称 | 指标 | 正常 | 警告 | 异常 |
|---|---|---|---|---|
| CPU | 使用率 | <80% | 80%-90% | >90% |
| 内存 | 使用率 | <80% | 80%-90% | >90% |
| 磁盘 | 使用率 | <70% | 70%-80% | >80% |
| 网络 | 带宽(KB) | >512 | >=100 | <50 |
| 时间误差 | 秒 | <=15 | <=30 | >30 |
中间件监控
| 名称 | 指标 | 正常 | 警告 | 异常 |
|---|---|---|---|---|
| mysql | 节点 | mysql 集群正常 | mysql 异常剩余2节点 | mysql 集群故障 |
| redis | 节点 | redis 集群正常 | redis 异常剩余2个节点 | redis 集群故障 |
| rabbitmq | 节点 | rabbitmq 集群正常 | rabbitmq 异常剩余2个节点 | rabbitmq 集群故障 |
| minio | 节点 | minio 集群正常 | minio 异常剩余2节点 | minio 集群故障 |
| elasticsearch | 节点 | Elasticsearch 集群正常 | Elasticsearch 异常剩余2节点 | Elasticsearch 集群故障 |
k8s节点
| 名称 | 指标 | 正常 | 警告 | 异常 |
|---|---|---|---|---|
| 节点 | 故障节点数量 | 0% | 33% | 66% |
| pod | pod状态 | 正常 | 重启 | 非Running |
监控安装
tip
因监控文件存放在模块部署包中，所以您需要在当时执行部署操作的机器上安装监控服务
Input: /usr/local/python-3.7.6/bin/laipvt deploy --monitor
Output:
[INFO] 2022-05-05 14:29:38,056 util.py [line:345] modify_umask: 主机: 172.18.242.18, 执行命令: sed -i '$aumask 0022 #laiye' /etc/bashrc
[INFO] 2022-05-05 14:29:38,465 util.py [line:287] wrapper: 即将运行middleware下的deploy_monitor方法
[DEBUG] 2022-05-05 14:29:38,467 template.py [line:19] _fill: 渲染/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/config/dashboards.yaml文件到/tmp/dashboards.yaml
[DEBUG] 2022-05-05 14:29:38,469 template.py [line:19] _fill: 渲染/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/config/datasources.yaml文件到/tmp/datasources.yaml
[DEBUG] 2022-05-05 14:29:38,470 template.py [line:19] _fill: 渲染/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/config/prometheus.yaml文件到/tmp/prometheus.yaml
[DEBUG] 2022-05-05 14:29:38,476 template.py [line:19] _fill: 渲染/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/config/mysql_datasources.yaml文件到/tmp/mysql_datasources.yaml
[DEBUG] 2022-05-05 14:29:38,477 template.py [line:19] _fill: 渲染/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/config/k8s_datasources.yaml文件到/tmp/k8s_datasources.yaml
[DEBUG] 2022-05-05 14:29:38,479 template.py [line:19] _fill: 渲染/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/prometheus_istio.yaml文件到/tmp/prometheus_istio.yaml
[INFO] 2022-05-05 14:29:38,483 middlewareinterface.py [line:82] send_config_file: 分发文件/tmp/dashboards.yaml到172.18.242.18:/home/rpa/monitor/conf/dashboards.yaml
[INFO] 2022-05-05 14:29:39,132 middlewareinterface.py [line:82] send_config_file: 分发文件/tmp/prometheus.yaml到172.18.242.18:/home/rpa/monitor/conf/prometheus.yaml
[INFO] 2022-05-05 14:29:39,618 middlewareinterface.py [line:82] send_config_file: 分发文件/tmp/datasources.yaml到172.18.242.18:/home/rpa/monitor/conf/datasources.yaml
[INFO] 2022-05-05 14:29:40,118 middlewareinterface.py [line:82] send_config_file: 分发文件/tmp/k8s_datasources.yaml到172.18.242.18:/home/rpa/monitor/conf/k8s_datasources.yaml
[INFO] 2022-05-05 14:29:40,567 middlewareinterface.py [line:82] send_config_file: 分发文件/tmp/mysql_datasources.yaml到172.18.242.18:/home/rpa/monitor/conf/mysql_datasources.yaml
[INFO] 2022-05-05 14:29:41,064 middlewareinterface.py [line:82] send_config_file: 分发文件/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/json到172.18.242.18:/home/rpa/monitor/json
[INFO] 2022-05-05 14:29:41,635 middlewareinterface.py [line:82] send_config_file: 分发文件/tmp/prometheus_istio.yaml到172.18.242.18:/home/rpa/monitor/prometheus_istio.yaml
[DEBUG] 2022-05-05 14:29:42,132 template.py [line:19] _fill: 渲染/root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/monitor.tmpl文件到/tmp/docker-compose-monitor.yaml
[INFO] 2022-05-05 14:29:42,139 middlewareinterface.py [line:71] send_docker_compose_file_hosts: 分发文件/tmp/docker-compose-monitor.yaml到172.18.242.18:/home/rpa/monitor/docker-compose.yml
[INFO] 2022-05-05 14:29:42,637 middlewareinterface.py [line:111] start_all_node: 启动172.18.242.18 monitor服务
[INFO] 2022-05-05 14:29:54,114 monitor.py [line:71] _install_prometheus: ['kubectl apply -f /home/rpa/monitor/prometheus_istio.yaml', 'helm --host=localhost:44134 install --name=k8s-metrics --set replicaCount=1 --set image.repository=172.18.242.18:8889/middleware/kube-state-metrics --set image.tag=latest /root/checker-XerbzmNpb/XerbzmNpb/middleware/monitor/kube-state-metrics']
安装完成后，会默认启用3000端口作为监控页面登录地址，默认用户名密码为admin/admin
单节点部署：关注页面中的单节点全局监控面板
多节点部署：关注页面中的高可用全局监控面板
如果您不是技术人员，您只需要关注页面中的颜色即可
| 绿色 | 正常 |
|---|---|
| 黄色 | 警告 |
| 红色 | 异常 |
