HPC & AI/GPU Cluster Services — Slurm, Kubernetes and Inference
Your GPUs are expensive and mostly idle. We fix the part that wastes them: the scheduler, the fabric, the storage feeding them, and the way jobs queue.
What this usually looks like
You bought a rack of GPUs. Now you need them scheduled, fed, cooled, monitored and shared between teams that all believe they need the whole rack.
- Utilisation is unknown, or known to be poor, and nobody can prove where it went.
- Multi-node training fails intermittently and everyone blames the network.
- Research wants Slurm, platform engineering wants Kubernetes, and both are right.
- Inference endpoints are hand-started scripts with no capacity model.
Three ways to buy this
Audit if you want to know how bad it is. Build if you have the team and want the capability. Managed if you don't want to own the pager. Most engagements start with the audit and decide afterwards.
Audit
A fixed-price health check — the cheapest way in
- GPU utilisation measurement: per card, per user, per job, over a real period
- Interconnect health check: NVLink, InfiniBand/RoCE, NCCL all-reduce bandwidth
- Scheduler configuration and queue-time analysis
- Storage throughput test at the rate training actually demands
Build
One-off delivery: design, code, hardening, handover
- Slurm cluster with GPU GRES, partitions, QoS and accounting (or Slinky to bring Slurm into Kubernetes)
- Kubernetes GPU stack: NVIDIA GPU Operator, DRA, MIG partitioning, Kueue/Volcano/KAI gang scheduling
- InfiniBand / RoCE fabric with NCCL tuning and fabric-manager validation
- Parallel filesystem for training data (BeeGFS, Lustre, or CephFS) plus NVMe local caching
- Inference serving: vLLM, Triton, KServe, Gateway API inference extension
- Ray / KubeRay and Kubeflow for distributed workloads
Managed
Build plus 7×24 on-call, SLA and quarterly drills
- Utilisation reporting per team with a before/after baseline
- Driver, CUDA and fabric-manager patch cycles with rollback
- Queue-time and fair-share tuning as demand shifts
- Thermal and power telemetry review
What we actually deploy
Specific tooling, not categories. If a version matters to your decision, ask — we will tell you which one we would pick and why, including when the answer is the older one.
Schedulers
- Slurm (SchedMD / NVIDIA builds)
- Slinky (Slurm in Kubernetes)
- OpenPBS / PBS Pro
- IBM Spectrum LSF
- Kueue
- Volcano
- KAI Scheduler
- Nomad (GPU)
Kubernetes GPU stack
- NVIDIA GPU Operator
- DRA (K8s 1.34 GA)
- MIG / MPS / vGPU
- Node Feature Discovery
- Topology Manager
- KubeRay
Fabric & collectives
- InfiniBand (HDR/NDR)
- RoCE v2
- NCCL
- NVLink / NVSwitch
- GPUDirect RDMA
- SHARP
- UFM telemetry
Toolchains
- CUDA 12.x
- ROCm
- oneAPI
- OpenMPI / MPICH
- UCX
- OpenHPC
- Spack
- Apptainer / Singularity
- Enroot + Pyxis
Inference serving
- vLLM
- LMDeploy
- NVIDIA Triton
- KServe
- Seldon
- Gateway API Inference Extension
- Ray Serve
Training frameworks
- PyTorch DDP / FSDP
- DeepSpeed
- Megatron-LM
- JAX
- Kubeflow Pipelines
Facilities & storage
- liquid cooling telemetry
- GPU power capping
- BeeGFS / Lustre / WekaFS
- NVMe cache tiers
- DCGM
What we will be measured on
These are the numbers we put in the delivery report. If we cannot measure it, we do not promise it.
- A GPU utilisation report showing measured before and after — the number AI customers actually care about.
- Multi-node all-reduce bandwidth baseline captured, with NCCL tuning documented.
- Queue-time and fair-share policy reported per team, reviewed quarterly.
- Inference endpoints with autoscaling, capacity model, and token-level rate limits.
- Job preemption and gang scheduling behaving predictably, demonstrated under contention.
What we would build, and where the weak point is
A single Slurm controller with no failover stops the whole cluster when it hiccups — and slurmctld hiccups more than people expect. It gets an HA pair or it gets documented as an accepted risk.
marks a single point of failure we would raise with you rather than hide. Every architecture has one; the question is whether you know where it is.
What we don't do
Every vendor says yes to everything. Here is where our answer is no, or where someone else needs to be in the room. If this list looks wrong for your situation, tell us — we would rather argue about scope before the contract than after.
- Out of scope
Model development, fine-tuning strategy, prompt engineering
We build the cluster that trains and serves. What you train on it is your research.
- Requires partner
GPU hardware procurement and allocation quotas between teams
We specify requirements and implement your fair-share policy. The policy itself is a management decision.
- Requires partner
Datacentre power, cooling and rack capacity provisioning
We provide the requirement figures (kW, airflow, weight) and monitor the resulting telemetry.
- Out of scope
Guaranteeing a utilisation target without scheduling policy changes
Utilisation is a function of workflow design as much as configuration. We will show you the data and name the honest ceiling.
What it looked like last time
A research cluster with unmeasured GPU usage: instrumentation revealed 31% average utilisation dominated by idle-but-allocated cards. After scheduler policy changes and preemption tuning, measured utilisation reached 78% with no additional hardware purchased.
Anonymised under NDA. No client names, no logos, no invented metrics. Full case library on the work page.
Talk to the hpc / ai infrastructure engineer about your hpc & ai / gpu clusters
Send us the architecture, or just describe what broke most recently. You'll get an engineer's answer, not a discovery questionnaire.
Talk to a hpc / ai infrastructure engineer: botonwa@gmail.com · +86 130 5181 3099