MPFX is presented, a correctly-rounded, multi-precision number library that exploits correct double rounding to efficiently simulate many number formats and introduces a format inference algorithm that bounds each expression by a number format guaranteed to contain its result.
Abstract
Number libraries, which implement number formats and their arithmetic in software, are used to simulate hardware, making hardware behavior reproducible and thus verifiable and easier to design. However, modern machine learning accelerators are difficult to simulate, as they increasingly use specialized number formats that existing number libraries do not support. It is costly to extend these libraries using traditional methods: each combination of operation, format, and rounding mode typically requires a bespoke implementation. An easier way is to double round - compute at higher precision, then re-round - so that one high-precision kernel serves many formats, but double rounding is only known to be correct in select cases. We give a precise, efficiently checkable characterization of when double rounding is correct across many formats and rounding modes. Our result rests on a novel abstract number format that unifies fixed- and floating-point representations, reducing the correctness of double rounding to format containment: whether every value of one format is representable in another. We mechanized these results in Lean 4. In addition, we introduce a format inference algorithm that, given a sequence of operations, bounds each expression by a number format guaranteed to contain its result. We apply these insights in two ways. First, we present MPFX, a correctly-rounded, multi-precision number library that exploits correct double rounding to efficiently simulate many number formats. MPFX's correctly-rounded operations are up to 11x (mean: 6.05x) faster than MPFR's, and competitive with SoftFloat's - between 0.46x and 1.63x as fast (mean: 0.94x) - at the same precision. Second, in a case study, we implement a software simulation of a hardware specification using both SoftFloat and MPFX's primitives; the MPFX version is up to 13.5x faster.
Motivated by accelerating machine learning (ML), many computer vendors and chip manufacturers are building accelerators for matrix multiplication, which save time and energy by operating in the lower precisions needed for ML. This has in turn motivated many efforts to use these accelerators to provide faster matrix mul...
James Demmel, G. Henry, I. Kozachenko et al.· 2 citations
We present FloatLib, a verified arbitrary-precision floating-point arithmetic library in Lean 4 that combines broad format coverage, machine-checked correctness, and efficient certified execution. To our knowledge, FloatLib is the first Lean library to unify IEEE binary and decimal arithmetic, arbitrary-width posits, P...
Robert Joseph George, W. Adkisson, Anima Anandkumar· 0 citations
Mixed-precision algorithms exploit reduced-precision hardware, however computing with the same matrix at distinct precisions typically requires duplicate storage, repeated conversions, or operation-emulation. We introduce TOMOS, a composite-precision matrix format that provides implicit 32-bit and 16-bit access from a...
Interval arithmetic provides guaranteed error bounds for finite-precision numerical computations, but obtaining these error bounds is much slower than the original computation. Even on highly memory bound workloads like small 1D stencils, the existing approaches fail to hide the overhead of changing from scalar to inte...
Xing-Jian Qi, R. Strzodka· Proceedings of the Internati...· 0 citations
MatLAB-based software models of the matrix multipliers are developed for each architecture and validated for bit-level reproducibility against hardware using a randomized test suite consisting of 10 million sets of random input vectors.
Faizan A. Khattak, M. Mikaitis, Carlo Graziani· 0 citations
A descriptor of GEMM reduction order is introduced, and the first black-box reconstruction of a closed-source library's arithmetic for bit-level correctness is performed, including the first black-box reconstruction of a closed-source library's arithmetic for bit-level correctness.
Zi-Teng Yang, Nicholas J. Riasanovsky, Warren Deng et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.