Flow-based spectral kernel attention (FSKA), a framework for learning expressive nonstationary attention kernels, is proposed and it is shown that expressive kernel learning enables a simplified attention design with removed query--key projections and fixed orthogonal value projections, resulting in a more parameter-ef...
Zi-Ye Yang, Ying Li, A. Lancho et al.· 0 citations
Transformers provide a state-of-the-art modeling framework, yet poor calibration limits their reliability in safety-critical applications. A promising direction addresses this issue by interpreting attention as a Gaussian process (GP) posterior, which enables principled uncertainty calibration but incurs cubic complexi...
Amir Mohammad Mahfoozi, Zi-Ye Yang, Ying Li et al.· 0 citations
This work analytically marginalize the sparse GP posterior to derive a tractable drift-matching objective that accounts for both the posterior mean and uncertainty of the unknown drift, and introduces an irregular-time-aware variational state posterior that incorporates the actual observation times during both encoding...
Zhi-Di Lin, Yu-Hao Liu, Ying Li et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.