Generative large language models (LLMs) have achieved state-of-the-art performance on many real-world tasks such as code generation and question answering. These models predominantly rely on an autoregressive decoding strategy that generates output tokens sequentially. However, their pervasive deployment raises serious privacy concerns, motivating private inference frameworks based on fully homomorphic encryption (FHE). A major limitation of existing FHE frameworks is their inefficiency in evaluating nonlinear operations, which incur substantial overhead and dominate the decode stage. In this paper, we propose ROSETTA, a hybrid CKKS/TFHE framework that overcomes this limitation. We first observe that nonlinear operations in the decode stage exhibit heterogeneous workload patterns, which can be handled effectively via a hybrid approach. We then realize this with two key contributions: 1) an adaptive segmented lookup-table protocol based on TFHE that enables efficient and accurate evaluation of nonlinear operations; and 2) a scheme-aware operator-selection framework that automatically assigns each nonlinear operator to CKKS or TFHE to minimize end-to-end decoding latency. We demonstrate that ROSETTA achieves up to $4.8\times$ Softmax speedup and $1.5$--$2.1\times$ end-to-end speedup over the SOTA framework CacheMir.
Jiang-Rui Yu, Bao-Sheng Zhang, Liang Kong et al.· 0 citations
Fully homomorphic encryption (FHE) enables computation on encrypted data, but its long ciphertext dataflow and high-dimensional modular arithmetic make it vulnerable to silent data corruption caused by transient hardware faults. Existing protection methods either target dedicated accelerators or impose substantial execution, modular-arithmetic, and memory-access overheads on CPUs. This work presents an efficient fault-tolerance scheme for CPU-based CKKS computation. It checks the input-output consistency of polynomial operators while reducing protection overhead at three levels. First, modulus-aware bucket checksum exploits wide CPU accumulators to reduce expensive modular reductions. Second, dataflow-fused in-operator checking embeds checksum accumulation into operator dataflows, avoiding separate scans of long ciphertext polynomials. Third, cross-operator check fusion eliminates redundant checksum computations between adjacent operators while preserving end-to-end checking invariants. We implement the scheme in OpenFHE and evaluate it on representative encrypted applications and ciphertext primitives under random single-bit transient faults. It achieves a 100 percent empirical detection rate across 150,000 non-crashing corrupted-result cases and maintains application accuracy close to the fault-free baseline over a wide range of fault rates. The scheme incurs only 6.0 percent to 8.4 percent runtime overhead, averaging 6.8 percent, and reduces average protection overhead by 4.9 times compared with direct checksum-based protection.
Jianan Mu, Ge Yu, Tenghui Hua et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
