Skip to content

LOBERT: Generative AI Foundation Model for Limit Order Book Messages

Nov 2025 · arXiv.org · Vol abs/2511.12563 · 3 citations · 21 references
Computer Science

TL;DR

This work introduces LOBERT, a general-purpose encoder-only foundation model for LOB data suitable for downstream fine-tuning and achieves leading performance in tasks such as predicting mid-price movements and next messages, while reducing the required context length compared to previous methods.

Abstract

Modeling the dynamics of financial Limit Order Books (LOB) at the message level is challenging due to irregular event timing, rapid regime shifts, and the reactions of high-frequency traders to visible order flow. Previous LOB models require cumbersome data representations and lack adaptability outside their original tasks, leading us to introduce LOBERT, a general-purpose encoder-only foundation model for LOB data suitable for downstream fine-tuning. LOBERT adapts the original BERT architecture for LOB data by using a novel tokenization scheme that treats complete multi-dimensional messages as single tokens while retaining continuous representations of price, volume, and time. With these methods, LOBERT achieves leading performance in tasks such as predicting mid-price movements and next messages, while reducing the required context length compared to previous methods.

View source

Similar papers

Preprint Oct 2026

Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books

Autoregressive foundation models offer state-of-the-art capabilities to simulate the market by generating the message flow of limit order books (LOBs). Yet, without understanding their inner workings, research directions are hard to narrow and practical use in real-world trading environments remains uncertain. This stu...

Valentin Mohl, Alexandre Bismuth, Aramis Marti-Shahandeh et al. · 0 citations
#artificial intelligence Preprint Sep 2026

UQ-LOB: Uncertainty-Aware Limit Order Book Mid-Price Forecasting

Forecasting short-horizon mid-price movements from limit order book (LOB) data is central to algorithmic trading, yet most deep LOB forecasters are point predictors: they output a direction or a displacement, but never indicate which of their forecasts can be trusted. We introduce UQ-LOB, a lightweight, encoder-agnosti...

D. Manoharan, Eljas Linna, K. Baltakys et al. · 0 citations

Probabilistic Limit Order Book Forecasting with Conditional Diffusion Models

Modeling limit order book (LOB) dynamics is central to market forecasting and execution-strategy evaluation. While diffusion models achieve strong empirical performance, existing LOB snapshot models typically lack a learned recurrent encoder and formal statistical guarantees. We introduce LOBGrad, a conditional DDPM fr...

Jun-Yu Chen, Franklin Liu, Mo-Shu Xu et al. · 0 citations
#machine learning Preprint Sep 2026

Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

As diffusion large language models (dLLMs) become more capable, they are moving from research settings to real-world \textit{serving}, where request management (such as scheduling and resource allocation) relies on accurate estimation of per-request inference cost. However, common cost proxies fall short for dLLMs: out...

Hao-Yu Zheng, Fang-Cheng Fu, Bin-Hang Yuan et al. · 0 citations
#machine learning Preprint Oct 2026

On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction

This work proposes using coupled-noise match verification for diffusion blocks, following shared-Gumbel-noise coupling approaches, and introduces Speculating Positions Order Trajectory (SPOT), which learns to predict this order.

Eliezer Shlomi, Eliron Rahimi, A. Mendelson et al. · 2 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.