Prometheus Alerting: Alertmanager Routing

Q: Silence vs Inhibit difference?

Silence temporarily suppresses an alert (like a maintenance window). Inhibit suppresses when another alert is firing (dependency suppression).

Q: How to send Prometheus alerts to WeChat?

Use webhook to a relay service; WeChat bot receives.

Routing Tree Structure

route:
  receiver: default
  group_by: [alertname, instance]
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  routes:
    - match:
        severity: critical
      receiver: pagerduty
      repeat_interval: 30m
    - match:
        severity: warning
      receiver: wechat
      repeat_interval: 24h

Common Alert Rules

groups:
  - name: host
    rules:
      - alert: HighCpu
        expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "CPU > 80% for 5m"

Note: Placeholder version.