CollectiveLab is an event-driven benchmark harness for studying how ring and tree allreduce behave under topology, message size, and congestion constraints.
Distributed training discussions often compare collective algorithms in the abstract, but infra teams need reproducible crossover points under specific topology and oversubscription assumptions.
Gap statement:
Existing distributed training benchmarks compare collectives at coarse aggregate throughput, but fail to expose when ring or tree collapses under topology asymmetry, oversubscription, and straggler-heavy traffic.
- xAI: collective efficiency under Ethernet/RDMA scaling pressure
- NVIDIA: communication substrate benchmarking and topology sensitivity
- Apple: rigorous performance modeling under tightly bounded systems assumptions
make setup
make bench
make test.venv/bin/collectivelab benchmark --topology traces/sample_topology.json --format json- Finds ring vs tree crossover points by message size
- Quantifies topology penalty under oversubscription
- Emits deterministic JSON for repeatable comparison runs