Long Context Language Models

Technology

• Researchers introduce HeadWiseKV to compress hybrid language model caches: Researchers have introduced HeadWiseKV, a training-free framework designed to compress the residual global key-value (KV) caches of hybrid language models. It assigns each physical KV head a static, multilevel history window to make cache demand predictable before serving.

  LLM • arXiv

  Permalink