Prometheus, Grafana & cAdvisor

URL Prometheus

http://localhost:9090/

URL Grafana

http://localhost:3300/

URL cAdvisor

http://localhost:8080/

Production

hetzner:/opt/prometheus/

Docker-compose-yml anlegen:

version: '3.7'

services:
  prometheus:
    image: prom/prometheus
    container_name: prometheus
    user: root
    volumes:
      - ./prometheus:/etc/prometheus
      - ./prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--web.enable-lifecycle'
    ports:
      - "9090:9090"
    networks:
      - internal
    restart: unless-stopped

  grafana:
    image: grafana/grafana
    container_name: grafana
    depends_on:
      - prometheus
    ports:
      - "3000:3000"
    volumes:
      - ./grafana_data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=${GF_PASS}
    networks:
      - internal
    restart: unless-stopped

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    ports:
      - "8080:8080"
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    networks:
      - internal
    restart: unless-stopped
    
networks:
   internal:
      external: false

In Unterordner prometheus/prometheus.yml anlegen:

global:
  scrape_interval: 15s # By default, scrape targets every 15 seconds.
  evaluation_interval: 15s # Evaluate rules every 15 seconds.
  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
    monitor: 'Hetzner'

# Alertmanager configuration goes here (if used)
alerting:
  alertmanagers:
  - static_configs:
    - targets:
       # - alertmanager:9093

scrape_configs:
  - job_name: prometheus
    honor_labels: true
    honor_timestamps: true
    scheme: http
    scrape_interval: 60s
    scrape_timeout: 55s
    metrics_path: /metrics
    static_configs:
      - targets: ['prometheus:9090']
      - 
  - job_name: cadvisor
    scrape_interval: 5s
    static_configs:
      - targets: ['cadvisor:8080']

  - job_name: 'dwb@smns'
    static_configs:
      - targets: ['smns.diversityworkbench.de:9175']

Prometheus (9090), Grafana (3000) & cAdvisor (8080) laufen nur lokal auf Hetzner, daher ist es notwendig einen SSH Tunnel zu öffnen, um die Seiten anzuschauen.
Voraussetzung ist natürlich, dass man seinen SSH Key auf dem Server hinterlegt hat.

SSH Connectstring mit mehrfachem Tunnel:

ssh username@144.76.68.113 -p 666 -L 3000:144.76.68.113:3000 -L 8080:144.76.68.113:8080 -L 9090:144.76.68.113:9090

Die Seiten sind dann über http://localhost:9090/, http://localhost:3000/ bzw. http://localhost:8080/ erreichbar.

~/.ssh/config:

Host Hetzner
  User xyz
  HostName 144.76.68.113
  Port 666
  IdentityFile ~/.ssh/id_rsa.pub
  localforward 3000 144.76.68.113:3000
  localforward 8080 144.76.68.113:8080
  localforward 9090 144.76.68.113:9090