Language models have recently solved open Erd\H{o}s problems and achieved gold-medal level performance on International Mathematical Olympiad problems. But their solutions, written in natural language, cannot yet be verified by proof assistants. An erroneous reasoning step can invalidate every step that comes after, so...
Kanghyeon Kim, Johannes F. Lutzeyer, Yanzhu Guo· 0 citations
There is growing interest in applying Parameter-Efficient Fine-Tuning (PEFT) to Continual Learning, as it enables faster training, mitigates catastrophic forgetting, and facilitates better adaptation to new tasks. Prior approaches have explored a range of strategies, including extending new LoRA branches for each incom...
P. Singh, Mohammadi Zaki, Akash Saha et al.· 0 citations
Information-theoretic and geometric compression have been widely studied as possible explanations for generalization. Yet their temporal dynamics in recurrent networks remain largely unexplored. Using discretized LSTMs, whose finite embedding space enables direct estimates of information quantities, we observe pronounc...
Nikolaus Kopp, Franz Pernkopf, Bernhard Geiger· 0 citations
Flow matching models are typically trained to transport a Gaussian source to the data distribution, with source and target samples paired independently. We replace the source with a fitted Gaussian mixture and couple each data point to a mixture component through an entropic optimal transport problem. We show that the...
Stanislas Ducotterd, Bassam El Rawas, Michael Unser· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Counting is a simple task on which transformers can nevertheless fail at long sequence lengths. We study the cost of counting in a single attention layer, up to a maximum length $L$, focusing on how softmax attention and layer normalization shape the representation from which the count is recovered. We mainly focus on...
A. Shaffy, Manoj Dhakal, Safal Shrestha et al.· 0 citations
We introduce IND-Eval, a public benchmark that organizes industrial time series across domains into source-specific forecasting tasks. Relative to established benchmarks such as GIFT-Eval, it emphasizes finer sampling resolutions and a greater prevalence of multivariate forecasting. Evaluating Chronos-2, TiRex-2, and T...
Vebj{\o}rn Haugland (Cognite), Pavel Gulin (Cognite), David Foss (Cognite) et al.· 0 citations
Learning a response law for continuous decisions from observational time series is central to many real-world decision systems, yet temporal non-stationarity poses a fundamental challenge: as outcomes drift over time, naively pooled models can conflate action effects with shifting baselines and even reverse action rank...
Changjian Liu, Tianyu Wang, Yuwei Xu et al.· 0 citations
Natural-language self-consistency and executable programs provide complementary test-time evidence for mathematical reasoning, yet either channel may support the same semantic error. Thus we propose ACED, a reference-anchored framework that treats the self-consistency answer as the default and executable outputs as sup...
DR-APO (Decoupled-Representation Advantage-Guided Policy Optimization) is introduced, which decouples policy improvement from policy representation through advantage-guided target improvement and representation-compatible direct action fitting, providing a shared optimization principle for Gaussian, diffusion, and flow...
Expressive flow policies can represent multiple high-value actions, but sampling them with few integration steps while retaining tractable action densities remains challenging. We introduce DEFlow, a deep equilibrium flow policy that replaces explicit Euler sampling with implicit Euler integration and solves all interm...
Ruslan Ageev, Huaze Tang, Runqing Miao et al.· 0 citations
Electroencephalography (EEG) representation learning is challenged by noisy signals, inter-subject variability, and heterogeneous electrode montages in different datasets and recording settings. Recent self-supervised EEG pretraining methods improve transferability by converting heterogeneous recordings into unified to...
Generating accurate 3D molecules requires jointly determining a variable number of atoms, their identities, and geometry across spatial scales: global shape emerges coarsely, whereas chemical bonds demand fine local precision. Existing atom-based diffusion and flow models operate largely at a single scale, while dense...
A machine-learning approach called CheMeleon provides a practical way to develop new drugs or other molecules with much less detailed, expensive experimentation.
Research Manager Mike Walker reflects on a career spanning hacker culture, cybersecurity, and AI, sharing lessons about trust, teamwork, leadership, and why solving difficult problems requires curiosity, judgment, agility—and an understanding of the rules. The post Learn the rules: The hacker mindset appeared first on Microsoft Research.
Learn how Google Antigravity and our Cheminformatics Skills accelerate drug discovery workflows, search compounds, and propose novel rigidified macrocycles.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.