Joint-embedding predictive architectures learn abstract states by predicting target embeddings from context embeddings from context embeddings, but their transition models are typically opaque neural maps, so SJEPA is introduced, a reconstruction-free JEPA framework that learns predictive representations whose induced dynamics admit compact symbolic descriptions.
Abstract
Joint-embedding predictive architectures learn abstract states by predicting target embeddings from context embeddings, but their transition models are typically opaque neural maps. We introduce SJEPA, a reconstruction-free JEPA framework that learns predictive representations whose induced dynamics admit compact symbolic descriptions. Its hybrid transition combines a symbolic law with a regularised neural correction for dynamics outside the selected grammar. The central principle is to learn the simplest adequate dynamics: representation constraints preserve informative, non-collapsed predictive coordinates, while operator compression favours low-complexity symbolic-neural transitions that remain predictively adequate. We formalise this principle through induced-dynamics complexity, analyse predictive-coordinate non-identifiability, and show that unconstrained operator compression creates a direct shortcut to representation collapse. The framework supports both alternating representation-equation learning and symbolic dynamics fitted to fixed representations. In controlled pendulum experiments, joint learning discovers substantially simpler symbolic dynamics with lower long-horizon rollout error and divergence than post-hoc fitting, while an unconstrained one-step diagnostic realises the predicted collapse shortcut. Under grammar misspecification, correction regularisation preserves the representable symbolic mechanism and directs the neural component towards residual dynamics. The results expose a controllable trade-off among predictive fidelity, representation quality, symbolic parsimony, and symbolic-neural allocation.
Optimal Transport Dropout (OTD) is introduced, which offers a way to learn structured predictive uncertainty without explicit posterior inference or ensembles of independently trained predictors.
Giacomo Lorenzon, Francesco Regazzoni· arXiv.org· 0 citations
Two machine-learned representations are compared, DeepTICA and SPIB-VAE, with linear TICA for recovering steady-state observables from flawed distributions and a kinetic score computed from a coarse MSM at a resolution comparable to that used for RiteWeight random clustering is provided.
This work introduces LRC-JEPA, a lightweight end-to-end world model that routes information into a compact predictive latent and learned-query residual-context embeddings and shows that the resulting representation is sufficient, minimal, nuisance-invariant, and disentangled.
Lu-Zhe Huang, Lei Chu, Jing-Yi Liang et al.· arXiv.org· 2 citations
This work introduces LpWorldModel, a JEPA model regularized with Rectified Distribution Matching Regularization to match encoder features to a Rectified Generalized Gaussian distribution, yielding non-negative sparse codes, and finds that the learned sparse representations are mode-factored.
Yilun Kuang, Yash Dagade, Quentin Le Lidec et al.· 5 citations
This work proposes a JEPA-style world model in which representations with bilinear-parameterized dynamics can be learned directly from high-dimensional observations, reducing planning time by nearly three orders of magnitude while retaining or even improving control accuracy.
Antonio Pariente, Ignacio Boero, Nikolai Matni et al.· 0 citations
PRISM-M, a simple recurrent mathematical implementation of the Principle of Representation Integration for Stable Models, treats extraction, compression, integration, stabilization, and prediction/action as five interacting operations and tests whether they can produce internal states that persist over time while remai...
E. Masliah· bioRxiv· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.