Jul 2026· IEEE International Symposium on High-Performance Parallel Distributed Computing· pp. 58-70· 2 citations· 57 references
Computer Science
TL;DR
The OPAL universal framework is the OPAL universal framework, an adaptive architecture that endows generic lossy compressors with on-demand retrieval capabilities, enabling flexible and progressive access across multiple spatial regions, resolutions, and precisions.
Abstract
Scientific applications generate massive amounts of data, necessitating effective data reduction techniques like error-bounded lossy compression. While state-of-the-art error-bounded lossy compressors effectively reduce storage footprint, they typically treat the compressed data as monolithic blobs, forcing analysts to decompress large volumes just to access specific data modalities. Conversely, existing storage solutions designed for flexible access lack the efficiency of modern compression. To bridge this gap between efficient compression and flexible data retrieval, we propose OPAL, a unified system that harmonizes high-performance compression with flexible, on-demand data access. Our first contribution is the OPAL universal framework, an adaptive architecture that endows generic lossy compressors with on-demand retrieval capabilities, enabling flexible and progressive access across multiple spatial regions, resolutions, and precisions. Our second contribution is OPALI, a customized compression algorithm designed natively with flexible data access in mind. By leveraging novel mechanisms such as shadow anchors, hybrid storage, and grouped bit-plane coding, OPALI eliminates the overhead inherent in generic wrappers. Extensive evaluations on real-world scientific datasets demonstrate that OPAL transforms standard compressors into flexible retrieval engines, while OPALI achieves superior compression ratios and retrieval performance, significantly accelerating scientific discovery.
Exascale simulations generate data far faster than it can be stored or analyzed, making efficient data reduction essential. Error-controlled lossy compression offers high compression ratios under user-specified error bounds, but the target tolerance must be fixed at compression time. Progressive compression relaxes this restriction, yet existing methods still rely on fixed refactoring strategies and do not fully exploit correlations among decomposed coefficients, limiting the efficiency of progressive retrieval. In this work, we present an adaptive progressive compression framework that improves retrieval efficiency for two common targets, namely error-bound and peak Signal-to-Noise ratios. Our contributions are fourfold. (1) We propose to leverage two complementary interpolation schemes for adaptive progressive compression toward different targets, and we optimize them to achieve high efficiency. (2) We propose coefficient decomposition, a novel method that exploits the commonly overlooked spatial correlations among decorrelated data, which further improves the efficiency. (3) We develop the adaptive progressive compression workflow with automatic selection of the best-fit refactoring pipeline and tailored optimizations. (4) We evaluate the proposed framework on five real-world scientific datasets against three state-of-the-art progressive compressors. Experimental results demonstrate that the proposed framework improves the compression ratio by up to $42.3\%$ under the same requested error tolerance and up to $92.5\%$ at the same PSNR, compared with the best-performing existing methods. When transferring $512$ GB of scientific data to remote sites, the framework delivers up to $1.26\times$ speedup in the end-to-end data transfer performance. Furthermore, our method achieves the highest visualization quality while retrieving the least amount of data from storage.
Cortado is compared with state-of-the-art approaches and it is found that it is the only progressive compressor that guarantees the evaluated error bounds on all tested inputs while achieving the highest throughputs, comparable reconstruction quality, and on-par compression ratios.
It is argued that replacing autoregressive LLMs with DLMs within the same compression framework could overcome the throughput bottleneck caused by their one-symbol-per-step limitation, and the newly proposed DLM-based framework advances the state of the art in lossless text compression.
This first comprehensive study of data heterogeneity and lossless general-purpose compression performance for representative datasets from the PETRA III synchrotron radiation source provides a quantitative basis for future archival and storage decisions at PETRA III, its future successor, PETRA IV, and other large-scale scientific facilities.
M. Buschmann, Yannis Schumann, Christian Voss et al.· 0 citations
The growing scale of scientific data makes lossy compression essential for reducing data volume under controllable error. Transformation-based compressors using multilevel decomposition, such as MGARD, achieve strong compression ratios but map poorly to GPU architectures. We propose BlockMGARD, an adaptive, Region-of-Interest (ROI)-supported GPU lossy compressor, with four contributions: (1) an In-cache Block decomposition leveraging GPU on-chip memory and constant lookup tables to accelerate decomposition; (2) a hybrid hierarchy combining In-cache Block and global decomposition to balance speed and compression ratio; (3) an end-to-end pipeline with fine-grained ROI error control for feature preservation; and (4) an evaluation against state-of-the-art methods on five real-world datasets. Compared to MGARD-X, BlockMGARD achieves up to 4.2x and 9.1x higher compression and decompression throughput, and up to 8.63x higher compression ratio than uniform-tolerance baselines under ROI-aware error control. Across four GPUs, BlockMGARD achieves near-ideal linear scaling and up to 1.58x I/O cost reduction over MGARD-X.
Yan-Liang Li, Qian Gong, Qing Liu et al.· 0 citations
Error-bounded lossy compression is essential for storing and transferring the vector-field data produced by large-scale scientific simulations. Although it enforces a user-specified error bound to limit numerical distortion, it does not preserve the field's topology: small admissible perturbations can create or eliminate critical points on which downstream feature analysis depends. Existing GPU compressors achieve high throughput but are topology-agnostic, whereas the only compressor with provable critical-point preservation (cpSZ) runs on the CPU at throughput far below the data-generation rates of modern GPU-based systems. We observe that, although preserving critical points is inherently a coupled and sequential constraint, it can be reformulated into independent parallel tasks, either on a per-block basis or, speculatively, on a per-point basis. We present FaCTz, the first GPU-based error-bounded lossy compressor that guarantees critical-point preservation. FaCTz provides a block-wise mode optimized for throughput and a speculative per-point mode optimized for compression ratio. Across three vector-field datasets, FaCTz preserves every critical point while achieving throughput of up to 60 GB/s, approximately two orders of magnitude (up to approximately 640x) faster than the multithreaded CPU implementation of cpSZ. Its speculative mode further improves the compression ratio by approximately a factor of two over the throughput-oriented mode.
Mingze Xia, Yuxiao Li, Sheng Di et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
