写在前面

在k8s中，强烈推荐使用helm来部署服务，运维产品业务

安装prometheus+grafana+alertmanager

选择kube-prometheus-stack

我是全新的环境，自然是prometheus、grafana、alertmanager都要安装，所以很自然的选择了这些都集成且完备的监控方案：kube-prometheus-stack是一套完整的监控方案，使用选择了这套安装包

创建命名空间

每个项目最好有独立的名称空间

 kubectl create namespace monitoring

设置helm Chart仓库

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

正式安装

helm install prometheus prometheus-community/kube-prometheus-stack --namespace monitoring

执行后大概几秒钟，出现以下输出就表示成功了：

NAME: prometheus
LAST DEPLOYED: Fri Feb  2 13:15:21 2024
NAMESPACE: monitoring
STATUS: deployed
REVISION: 1
NOTES:
kube-prometheus-stack has been installed. Check its status by running:
  kubectl --namespace monitoring get pods -l "release=prometheus"

检查pod

可以看到以下有了prometheus，grafana，alertmanager等pod，且都是running，表示成功了。

root@# kubectl get pods -A
NAMESPACE     NAME                                                     READY   STATUS      RESTARTS   AGE
kube-system   coredns-6799fbcd5-hbnx2                                  1/1     Running     0          16h
kube-system   local-path-provisioner-84db5d44d9-rrxf8                  1/1     Running     0          16h
kube-system   helm-install-traefik-crd-x8bxs                           0/1     Completed   0          16h
kube-system   helm-install-traefik-ptknv                               0/1     Completed   1          16h
kube-system   svclb-traefik-c2a297d7-cmwjk                             2/2     Running     0          16h
kube-system   traefik-f4564c4f4-5f7jf                                  1/1     Running     0          16h
kube-system   metrics-server-67c658944b-spr96                          1/1     Running     0          16h
kube-system   svclb-traefik-c2a297d7-gwfwx                             2/2     Running     0          15h
kube-system   svclb-traefik-c2a297d7-jvfsr                             2/2     Running     0          4h47m
monitoring    prometheus-prometheus-node-exporter-4tnnz                1/1     Running     0          40s
monitoring    prometheus-kube-prometheus-operator-7f49cfc85d-srsjh     1/1     Running     0          40s
monitoring    prometheus-prometheus-node-exporter-5wz96                1/1     Running     0          40s
monitoring    prometheus-kube-state-metrics-6db866c85b-wzd5b           1/1     Running     0          40s
monitoring    alertmanager-prometheus-kube-prometheus-alertmanager-0   2/2     Running     0          31s
monitoring    prometheus-grafana-9c98f646b-lnpwd                       3/3     Running     0          40s
monitoring    prometheus-prometheus-kube-prometheus-prometheus-0       2/2     Running     0          30s
monitoring    prometheus-prometheus-node-exporter-b4jgx                1/1     Running     0          40s

配置grafana外网访问

一般来说，我们希望可以通过外网访问grafana来达到查看整个集群的监控数据的效果，所以就需要配置grafana的外部访问，使用ingress可以完美实现通过域名访问grafana的效果

查看grafana的service名称

kubectl get services -A | grep grafana

我这里是prometheus-grafana在monitoring的名称空间中，端口为80，拿到这些信息就可以写ingress的yaml了 创建grafana-ingress.yaml写入以下内容

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: monitoring
spec:
  rules:
    - host: grafana.at9.net
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: prometheus-grafana
                port:
                  number: 80

查看grafana的密码

kubectl get secret prometheus-grafana -n monitoring -o jsonpath="{.data.admin-password}" | base64 --decode

我这里输出的密码为：prom-operator 拿到密码，访问域名，可以登录。

配置alertmanager告警

待续ing…