On-Policy Attention Linearization (OPAL) is introduced in which the hybrid attention student samples its own long-context trajectories and receives dense supervision from the frozen full-attention teacher.
Arian Raje, Anupam Nayak, Anthony Fei et al.· arXiv.org· 0 citations
For every odd integer $t\ge17$, we prove that an explicit circulant graph on $5t-10$ vertices is doubly saturated $R(3,t)$-good. The graph is triangle-free and has independence number $t-1$. Adding any nonedge creates a triangle, whereas deleting any edge creates an independent set of order $t$. This settles Conjecture...
Abhishek Saigal, Akaash R. Parthasarathy· 0 citations
A lightweight latency estimator is designed that simulates autoregressive token batch processing in the serving framework and estimates the time-to-first-token (TTFT) of queries and is incorporated into a latency-aware router that jointly optimizes latency, accuracy, and cost when assigning queries to model instances.
Shivam Patel, Akaash R. Parthasarathy, Ankur Mallick et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.