AI
Ai2 says it replaced GPU priority fights with budgets, fair share and time slicing
Ai2 describes replacing a priority-based GPU scheduler with time budgets, hierarchical fair share and minimum-runtime contracts for heavily oversubscribed research clusters. The system turned resource claims into explicit organizational budgets while preserving burst capacity and allowing preemption after meaningful progress. In a 30-day rollout, teams received 98% of owed GPU hours, occupancy stayed at 98%, debug-job wait times fell sharply and repair work requiring human intervention dropped 74%. The account also reports steeper learning costs, disrupted interactive sessions and a possible fragmentation problem still under investigation.
Editorial summary · Publisher article
What matters
- Ai2 manages thousands of GPUs for about 150 researchers while submitted demand exceeds available capacity by roughly two to three times.
- Budgeted workloads receive protected minimum runtimes, while unallocated capacity remains usable but preemptible.
- The reported rollout delivered at least 95% of allocated GPU hours to 13 of 15 teams, with the lowest result at 90%.
- Ai2 says it is developing restorable sessions and investigating whether time slicing could lengthen waits for the largest jobs.
Context & caveats
- The performance figures come from Ai2’s own rollout and are not an independent benchmark.
- Some debug-workload comparisons use small samples, and the largest-workload fragmentation risk remains unresolved.
Why this made the edition
A detailed first-party engineering account presents a concrete GPU-allocation redesign, measured production results, tradeoffs and unresolved risks. Its claims are clearly attributed to Ai2’s own experience.
Source material