A homelab SRE project by Matthew Harwood ·github ·linkedin ·email
Oberon SRE
Live cluster health · Refreshes every 30s · Resource graph · Design rationale · Puck trading dashboard
What this is
This is a live operational view of my homelab Kubernetes cluster, which hosts an automated SPX options trading bot called Puck. Every box on this page reflects the actual state of real hardware, services, and application invariants, queried from Prometheus running on the cluster itself.
The data model is declarative. Resources carry explicit Utilisation, Saturation and Errors thresholds, following Brendan Gregg's USE method. Edges have criticality, hard or soft. Conditions have multi-window observations, so one predicate drives both a per-resource signature (instant) and a service-level SLO (30-day window) from a single definition. Services are derived views over the graph. State propagates upstream to downstream with criticality weighting, so a green service degraded by a red dependency shows up as OK, broken by upstream, which reads distinctly from a service failing on its own metrics.
The dashboard is the visible artefact of a homelab SRE methodology project. Source and design rationale are on the about page, citing the SRE Book chapters 4 through 22, DDIA chapters 1 and 8, and Brendan Gregg's Systems Performance work on USE and off-CPU analysis. Scope is homelab: single rack, one trading service, one public UI, one node with a known hardware fault. The thinking is production-shaped, applied at homelab size.
Known anti-pattern: Prometheus and Alertmanager currently run inside the same Kubernetes cluster they monitor. If the cluster goes down, so does the dashboard that would tell you it's down. The plan is to move the observability stack to a separate node or small cluster once more hardware arrives, so this dashboard can survive a failure of the workload cluster. Documented openly. Pretending it's fine is the more interesting failure mode.
How to read it. Click any service tile to see its members, SLOs, and currently-firing signatures. Click an active signature for its runbook. Hosts at the bottom show per-host USE with both their own state and effective state (which propagates from upstream).
Active signatures
0 firing · 1 suppressed · 0 pagingEach firing condition links to its runbook. A page badge means the same predicate also emits an AlertManager rule — everything else is dashboard-only diagnostic (no pages on per-resource thresholds).
No active signatures. All conditions within tolerance.
Suppressed (1)
- node-unreachable → host/green
Services
Puck (Trading Bot)
OKPython execution bot trading SPX put-spreads via IBKR. State-machine driven (6-state asyncio FSM).
SLOs (5/5 compliant)
23 resources in scope
Oberon UI
DegradedNext.js public dashboard rendering Puck state and (now) SRE dashboard.
SLOs (2/2 compliant)
31 resources in scope
Network (MikroTik gateway)
OKSingle-router gateway between LAN and consumer ISP. Single point of failure for all egress.
SLOs (3/3 compliant)
9 resources in scope
Hosts (USE)
| Host | Role | Own / Effective | Util | Sat | Err | Notes |
|---|---|---|---|---|---|---|
| host/red | control-plane | OK | 0.079 | 0.105 | 0 | |
| host/blue | worker | OK | 0.067 | 0.142 | 0 | |
| host/green | worker | Failing | 0 | 0 | 0 | offline: Hardware fault — pending physical inspection / part replacement |
| host/yellow | worker | Self OK↑Upstream | 0.115 | 0.168 | 0 |