Blog
Research notes, papers, and releases.
DoSA: Dots Sparse Attention
Sparse attention reads few keys, but choosing them still means scoring every key with every indexer head. DoSA proves most keys cannot win and skips them, with exactly the same Top-K.