Existing patching and multi-scale methods advance multivariate time series forecasting but treat learned representations as transient byproducts of prediction, lacking explicit mechanisms that enforce structural consistency across temporal scales. We propose M2Patch, a CNN-based architecture that organizes channel-independent observations into a structured latent space via two complementary differentiable penalties. Multi-scale patching decomposes the input into overlapping temporal granularities, depthwise separable CNN blocks with progressively growing dilation extracts scale-specific features at linear complexity, and per-scale learned projections compress these features into a compact latent representation. An intra-scale smoothness penalty enforces temporal continuity between adjacent patches, while an inter-scale alignment penalty restores cross-granularity interaction through learnable cross-scale mappings, so that all scales encode mutually consistent representations of the underlying dynamics. Extensive experiments on ten real-world benchmark datasets demonstrate that M2Patch significantly outperforms state-of-the-art baselines. Further analyses establish M2Patch as a structure-aware recognizer: it recovers channel functional groupings and remains robust under patch-level input corruption, confirming that the structured latent space captures the data's intrinsic dynamics.
This work proposes RouteCost, a production-inspired multi-stage framework that decomposes the problem into time-aware demand forecasting, fee-card-informed baseline pricing, Stage 2 residual correction, and proxy-based box-consolidation inference inference.
Xianling Zeng, Zihan Yu, Sichen Zhao et al.· 0 citations
A pipeline that is fully automatic and cross lingual, using large language models, GPT 5, GPT 4o and Claude Sonnate 4, to independently generate, evaluate, and refine neural network architectures for handwritten optical character recognition, opening the door to rapid and adaptable deployment of OCR technology across many languages and domains.
Results demonstrate that hierarchical modeling of 3D facial geometry enables interpretable, ontology-linked phenotype classification, though performance on rare leaf terms remains limited.
F. Hellmann, Alexander Hustinx, Benjamin D. Solomon et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Mask2Real-WM is presented, a two-stage action-conditioned world model for dexterous manipulation that decouples pixel prediction into a dynamics model and a rendering model that shows that mask conditioning and simulation pretraining are both required for per-DoF action controllability across all 23 degrees of freedom.
Riccardo Feingold, Davide Liconti, Chenyu Yang et al.· 1 citation
As AI labs approach a data ceiling where compute capacity outpaces the rate of new high-quality text generation, language model pretraining is shifting toward a data-constrained, compute-abundant regime that demands productive multi-epoch training on fixed corpora. Standard autoregressive (AR) pretraining overfits severely in this setting, reaching its optimum early and then continuously deteriorating. We investigate training-time data augmentation as a regularizer to mitigate this overfitting and enable productive training for hundreds of epochs on the same data. We introduce three orthogonal categories of augmentation for AR pretraining: token-level noise (masking, random replacement), sequence permutations (right-to-left prediction, Fill-in-the-Middle), and target offset prediction ($x_{t+i}$ for $i > 1$). Through systematic ablations, we find that individual augmentations delay overfitting and lower validation loss relative to the baseline, with random token replacement achieving the best minimum loss among individual methods. Combining augmentation categories further lowers the minimum validation loss. Our experiments demonstrate that data augmentations mitigate AR pretraining's data inefficiency and offer a promising solution to the data-constrained regime~\footnote{All code and data are available at https://github.com/ michaelchen-lab/ data-augmentations-for-pretraining.
Michael K. Chen, Xikun Zhang, Fan Bai et al.· 0 citations
Semi-structured information extraction (IE) from OCR-derived clinical reports is crucial for efficiently reconstructing patients' longitudinal medical histories. In practice, this scenario commonly involves three tasks: (i) field-header (key) discovery, (ii) key-conditioned question answering (QA), and (iii) end-to-end key-value pair extraction. However, existing evaluations often under-model two factors: heterogeneous and incompletely known key representations, and OCR-induced noise. This makes it difficult to assess model robustness in real-world settings.
We present MedStruct-S, a benchmark specifically designed to evaluate these tasks under unknown keys and OCR noise. MedStruct-S contains 3,582 annotated real-world clinical report pages. Using MedStruct-S, we benchmark two representative paradigms: encoder-only sequence labeling with post-processing and decoder-only structured generation, covering four encoder-only and five decoder-only models spanning 0.11B to 103B parameters. Our results show that encoder-only models achieve the best performance for non-null-value key-conditioned QA despite being substantially smaller than decoder-only models. When comparing models of similar order of magnitude, encoder-only models still perform better overall. Without controlling for model scale, fine-tuned decoder-only models deliver the strongest overall results. These findings show that the benchmark provides a reliable and practical basis for selecting and comparing models across different semi-structured IE settings.
This work presents AutoOR, a scalable synthetic data generation and reinforcement learning pipeline that trains LLMs to autoformulate optimization problems specified in natural language across linear, mixed-integer, and non-linear categories and introduces a curriculum RL strategy that bootstraps from limited initial training data to make this class tractable for post-training.
S. Motwani, Chuan Du, A. Petrov et al.· 2 citations
Findings show that preserving the temporal extent of recurrent history is important for efficient whole-piece modeling, and that memory cost can instead be reduced through KV representation compression.
Yungang Yi, Weihua Li, Matthew Kuo et al.· 0 citations
FiLoRA is introduced, an instruction-conditioned, parameter-efficient adaptation framework that enables controllable modulation of feature reliance while keeping the task and predictive objective fixed and suggests that instruction-conditioned parameter adaptation can serve as a practical mechanism for intervening on internal model behavior.
Hyunsuk Chung, Caren Han, Yerin Choi et al.· arXiv.org· 1 citation
This work offers the first random generator of synthetic causal benchmarks with coverage guarantees and transparent assumptions operating on the three levels of causal reasoning: observation, intervention, and counterfactual, and demonstrates its utility by evaluating several state-of-the-art methods under diverse conditions and assumptions.
Panayiotis N. Panayiotou, Audrey Poinsot, A. Leite et al.· arXiv.org· 0 citations
NINJA (short for Needle-in-haystack jailbreak attack), a method that jailbreaks aligned LMs by appending benign, model-generated content to harmful user goals to reveal fundamental vulnerabilities in modern LMs.
R. Shah, C. Wu, Shashwat Saxena et al.· arXiv.org· 4 citations
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
