Homa: The End of TCP for AI Clusters — John Ousterhout, Stanford

AI workloads are shifting from massive, throughput-dependent data transfers to granular, latency-sensitive exchanges, particularly within inference and agentic applications. Legacy transport protocols like TCP and RDMA struggle in this environment, as they lack message-boundary awareness and suffer from high tail latency caused by congestion and head-of-line blocking. To resolve these bottlenecks, Homa—a clean-slate transport protocol—utilizes receiver-based congestion control and shortest-remaining-processing-time scheduling to prioritize short messages. By managing congestion at the receiver and leveraging switch priority queues, Homa reduces tail latency by over an order of magnitude compared to traditional protocols. This approach ensures that small metadata and synchronization exchanges do not stall GPU computation, ultimately improving overall system throughput for modern distributed AI applications.