ErsatzGPU

Why this project

GPU scarcity is the biggest barrier to actually learning this stuff.

You can read papers about NVLink topology, KV-cache growth, or disaggregated prefill/decode serving — but without real GPUs you can't feel any of it. ErsatzGPU closes that gap: every number is computed from the same formulas that govern real hardware, the K8s layer schedules real pods against real (fake) GPU resources, and the observability stack is a real Grafana/Prometheus/Langfuse deployment. Only the GPUs themselves are ersatz.

🧮

Real formulas, not lookups

VRAM breakdown, KV cache, cost, and power all respond correctly to input changes — bigger model, different GPU, more nodes — the same way real hardware would.

☸️

Real Kubernetes scheduling

Pods actually get scheduled against fake GPU resources — device-plugins, MIG partitioning, a 100+ node KWOK-simulated fleet — not a mockup.

📈

Real observability

A genuine Grafana + Prometheus + Langfuse deployment, fed by simulated workloads — practice dashboards and LLM tracing without a live training job.

Use case

What people actually use it for

Learning GPU internals

VRAM breakdown, architecture variants (GQA, MoE, MLA), NVLink/InfiniBand topology, training cost and power — all interactive, in a browser.

Learning inference serving

Prefill vs. decode, prefix-cache routing, vLLM PagedAttention, speculative decoding, and llm-d-style disaggregated serving tradeoffs.

Practicing K8s GPU scheduling

Watch pods get scheduled against fake GPU resources — including MIG partitioning and a 100+ node simulated fleet — without hardware.

Prototyping observability

A real Grafana + Prometheus + Langfuse stack fed by simulated GPU workloads, for practicing dashboards or LLM tracing on demand.

CI/CD for GPU workloads

A reusable GitHub Actions workflow that deploys the whole stack to a real k3d cluster and asserts GPU scheduling actually works.

Teaching and demos

No hardware bill, no cloud GPU quota request — spin up a full mock datacenter on a laptop for a class or a talk.

How it works

One set of formulas, four layers deep

engine/ has zero web or Kubernetes dependencies — it's the single source of truth the API, the K3s trainer Job, and the real inference server all call into, so the website's numbers and a real scheduled pod's simulated behavior never drift apart.

Diagram: engine/ (pure-Python VRAM / KV-cache / topology / cost / inference formulas) feeds api/ (FastAPI service — wraps engine/, streams inference over SSE, optional Grafana/Langfuse push, launches real k8s Jobs), which feeds web/ (Next.js site — model/GPU pickers, live training panel, prompt playgrounds, GPU-monitoring sparklines), which feeds k3s/ (fake-GPU K8s layer — device-plugin or fake-gpu-operator backend, Helm chart, sample trainer Job, real inference server)

Observability and WebUI

A real stack, not a screenshot

Runnable via docker-compose (no cluster needed) or in-cluster, layered on kube-prometheus-stack and Langfuse's official Helm chart rather than reimplementing either.

🖥️ WebUI

Training tab — model/GPU pickers, VRAM breakdown, topology diagram, cost calculator, a live training panel
Inference tab — prefill/decode explainer, colocated-vs-disaggregated throughput, a live streaming prompt playground
Live sparkline charts for power draw, SM utilization, and memory-bus-busy %

📊 Observability stack

Grafana pre-provisioned with the project's own dashboard and the vendored community NVIDIA DCGM dashboard
Prometheus scraping real DCGM_FI_*-shaped metrics
Langfuse tracing prefill/decode spans for every playground request and real workloads against the mock cluster

docker compose --profile observability up docker compose --profile langfuse up

Ecosystem integrations

What each dependency actually unlocks

Project	What it enables
llm-d	The disaggregated prefill/decode serving model the Inference tab is built on
fake-gpu-operator	Opt-in richer GPU backend — MIG, DRA, its own Prometheus metrics, KWOK-simulated fleets
KWOK	Backs the 100+ node simulated fleet with no real kubelet required
Prometheus	Scrapes the real exposition-format metrics, including DCGM series
Grafana	Pre-provisioned dashboards, locally or via kube-prometheus-stack's sidecar
Langfuse	Per-request LLM tracing for simulated and real workloads against the mock cluster
Kubernetes / Helm	The whole k3s/ layer — device-plugin, trainer, inference-server, and the simgpu chart
GitHub Actions	A reusable e2e workflow other repos can call directly against a real k3d-deployed mock cluster

Roadmap

Built, and what's ahead

Shipped

✓Calculation engine + FastAPI service

✓Next.js site with Training / Inference tabs

✓K3s fake-GPU device-plugin layer + Helm chart

✓llm-d-style disaggregated inference simulator

✓Architecture-aware formulas (GQA / MoE / MLA)

✓Tensor/pipeline parallelism + power modeling

✓Speculative decoding + Prometheus metrics bridge

✓fake-gpu-operator backend + KWOK-simulated fleet

✓Grafana + Prometheus + Langfuse observability stack

✓Real per-request inference streaming and tracing

✓Reusable CI/CD e2e workflow

Ahead

→Broader accelerator-vendor coverage in the topology model

→Deeper MIG-aware capacity planning

→Expanding the CI template library beyond the single reusable e2e workflow

Live status board: ErsatzGPU Roadmap · Contributions welcome — see CONTRIBUTING.md

Learn how a real GPU
datacenter actually works.
With zero GPUs.