A homelab SRE project by Matthew Harwood ·github ·linkedin ·email

Oberon SRE

Live cluster health · Refreshes every 30s · Resource graph · Design rationale · Puck trading dashboard

What this is

This is a live operational view of my homelab Kubernetes cluster, which hosts an automated SPX options trading bot called Puck. Every box on this page reflects the actual state of real hardware, services, and application invariants, queried from Prometheus running on the cluster itself.

The data model is declarative. Resources carry explicit Utilisation, Saturation and Errors thresholds, following Brendan Gregg's USE method. Edges have criticality, hard or soft. Conditions have multi-window observations, so one predicate drives both a per-resource signature (instant) and a service-level SLO (30-day window) from a single definition. Services are derived views over the graph. State propagates upstream to downstream with criticality weighting, so a green service degraded by a red dependency shows up as OK, broken by upstream, which reads distinctly from a service failing on its own metrics.

The dashboard is the visible artefact of a homelab SRE methodology project. Source and design rationale are on the about page, citing the SRE Book chapters 4 through 22, DDIA chapters 1 and 8, and Brendan Gregg's Systems Performance work on USE and off-CPU analysis. Scope is homelab: single rack, one trading service, one public UI, one node with a known hardware fault. The thinking is production-shaped, applied at homelab size.

Known anti-pattern: Prometheus and Alertmanager currently run inside the same Kubernetes cluster they monitor. If the cluster goes down, so does the dashboard that would tell you it's down. The plan is to move the observability stack to a separate node or small cluster once more hardware arrives, so this dashboard can survive a failure of the workload cluster. Documented openly. Pretending it's fine is the more interesting failure mode.

How to read it. Click any service tile to see its members, SLOs, and currently-firing signatures. Click an active signature for its runbook. Hosts at the bottom show per-host USE with both their own state and effective state (which propagates from upstream).

Active signatures

0 firing · 1 suppressed · 0 paging

Each firing condition links to its runbook. A page badge means the same predicate also emits an AlertManager rule — everything else is dashboard-only diagnostic (no pages on per-resource thresholds).

No active signatures. All conditions within tolerance.

Suppressed (1)
  • node-unreachablehost/green

Services

Hosts (USE)

HostRoleOwn / EffectiveUtilSatErrNotes
host/redcontrol-planeOK0.0790.1050
host/blueworkerOK0.0670.1420
host/greenworkerFailing000offline: Hardware fault — pending physical inspection / part replacement
host/yellowworkerSelf OKUpstream0.1150.1680