It is shown that robustness to post-training quantization (PTQ) is a transferable direction in weight space, and rigorously that quantization vectors are well-defined and do not suffer from reparameterization symmetries, and provide a local geometric account of their effect.
Daniele Solombrino, Antonio Andrea Gargiulo, A. R. Minut et al.· arXiv.org· 2 citations
Large Language Models (LLMs) have made it exceptionally convenient for users to send their private data to model providers, which process them in plaintext. Fully Homomorphic Encryption (FHE) promises to secure remote inference by computing directly on encrypted data, yet it has remained prohibitively slow. The effort...
Alessandro Zirilli, Davide Marincione, E. Kornaropoulos et al.· 0 citations
We show that a quantized model that keeps its classification accuracy still changes $14$ to $46\%$ of its top-1 retrieval results, and that aggregate ranking metrics reveal only part of this damage. We tie this failure to the gap between the two highest model scores and use that gap to decide when to trust a quantized...
This work introduces Homomorphic Encryption-Aware Training (HEAT), a fine-tuning method that makes the per-nonlinearity iteration counts learnable, enabling them and the model weights to co-adapt during training.
Alessandro Zirilli, Davide Marincione, E. Kornaropoulos et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.