Stories, tutorials, and deep dives spanning across diverse worlds.
Deep-dive architectural guide covering speculative decoding in vllm: tripling llm inference throughput with eagle & medusa with battle-tested production examples.
Comprehensive guide covering SGLang vs vLLM: High-Throughput LLM Inference, RadixAttention & Structured Decoding with production-grade architecture and code examples.
Comprehensive guide covering vLLM PagedAttention Deep Dive: KV Cache Memory Fragmentation, Chunked Prefill & Prefix Caching with production-grade architecture and code examples.
Cut vLLM TTFT by up to 2.5× using Medusa multi-head speculation and EAGLE draft models. Includes GPU benchmarks, config flags, and when speculative decoding hurts throughput instead of helping.