Serving a large language model is a constant negotiation between accuracy and cost. Nearly every lever that makes inference cheap — low-bit quantization, KV cache eviction, sparse or linear attention, speculative drafting — assumes that activations and attention mass are reasonably well behaved. They are not. Trained transformers reliably develop two extreme phenomena: attention sinks, where a small number of positions, often the very first token, absorb a disproportionate share of the softmax mass while contributing almost nothing semantically; and activation outliers, where a few hidden dimensions carry magnitudes orders of magnitude above the rest. These are not rare pathologies but a stable property of how the architecture is trained, and they are precisely the structures that break efficiency methods. A quantizer sized for the typical channel saturates on the outlier one; a cache policy that evicts low-utility tokens deletes the sink and the model collapses; a sparsity pattern derived from attention scores concentrates on positions that encode no information.
The field's standard response is to treat each symptom locally, and existing work falls into three broad classes. Architectural methods change the attention or normalization formulation so the extremes never form, by giving the model an explicit no-op channel rather than forcing it to improvise one out of a real token. Transformation-based methods leave the trained model intact and redistribute the offending mass before it reaches the kernel, through per-channel scaling, rotation, or other basis changes that make the activation distribution quantizable. Runtime methods accept the extremes as given and engineer around them — pinning sink tokens in the cache, carrying outlier channels at higher precision, or designing eviction and attention-sparsity policies that know where the extremes live. Each class is effective within its own stage of the pipeline, but each also treats sinks and outliers as separate defects with separate patches, so a fix applied at one stage routinely invalidates an assumption made at another.
We argue that attention sinks and activation outliers are two readouts of a single underlying quantity, and that an efficiency stack should be built on that quantity rather than on its symptoms. Our prior work established the components of this view: outlier-efficient associative memory layers that suppress the extremes at the source during training, their transfer to domains where the extremes are especially damaging such as genomic foundation models, filtering methods that exploit attention sparsity to separate load-bearing mass from the no-op mass that sinks absorb, and an analysis of the softmax normalization itself as the mechanism that manufactures sinks under long context. SinkOut unifies these into one framework: measure the extrema, attribute them to the mechanism that produces them, and expose that measurement to the inference stack so that quantization, cache management, and sparse attention make consistent decisions from a shared signal instead of independently patching the same phenomenon three times.