Adaptive self-supervised learning for real-time problem solving in autonomous systems.

Forecasting in Offline Reinforcement Learning for Non-stationary Environments

Data-driven optimization: efficient adaptive learning for self-driving laboratories

Self-driving laboratories promise to compress materials-discovery timelines from years to weeks by replacing trial-and-error experimentation with closed-loop, algorithm-guided campaigns. Yet, despite the rapid proliferation of robotic and automation hardware, today's autonomous labs rely almost exclusively on Bayesian optimization (BO) to decide what experiment to run next. BO is a sensible approach to low-dimensional optimization problems with smooth response surfaces, but it struggles in precisely the regimes that matter most for real materials campaigns: tight experimental budgets, dozens of process parameters, mixed-integer choices, hard physical constraints, and noisy expensive measurements. In this talk, I will show how moving from BO to partitioning-based algorithms can substantially improve data efficiency, scale gracefully to dozens of process variables, and handle the constraints and noise that characterize realistic experimental campaigns. I will summarize a recently completed large-scale black-box optimization (BBO) benchmark in which we compared 42 solvers across 502 problems ranging from one to 300 dimensions and from smooth and convex to nonsmooth and nonconvex. The results overturn several community assumptions: BO solves only about 9% of problems within a 2,500-evaluationbudget, while a new branch-and-model (BAM) algorithm reaches an 81% success rate, with GLCCLUSTER, MULTIMIN, MCS, and SNOBFIT also performing strongly. A minimal, irreducible set of eight complementary solvers attains 88% solvability on the full suite. I will then move from in-silico benchmarks to the wet lab, presenting a recent algorithmguided experimental campaign on high-performance perovskite solar cells in which a non- BO solver was used to co-optimize six process variables spanning the perovskite, electrontransport, and hole-transport layers. Time permitting, I will also share early results from applying ensembles of BBO algorithms to digital twins of self-driving labs across additional materials systems. I will close with a forward-looking research vision: accelerating autonomous labs by developing, benchmarking, and experimentally validating data-efficient adaptive algorithms across batteries, semiconductors, catalysts, polymeric membranes, and biomolecules. The benchmarking software will be released as open source, with BAM and most BBO software available free to academic users, so that experimental groups can deploy these tools on their own self-driving platforms. Bio: Nick Sahinidis is the Butler Family Chair and Professor in the H. Milton Stewart School of Industrial and Systems Engineering and the School of Chemical and Biomolecular Engineering at Georgia Tech. His current research activities are at the interface between computer science and operations research, with applications in various engineering and scientific areas, including: global optimization of mixed-integer nonlinear programs: theory, algorithms, and software; informatics problems in chemistry and biology; process and energy systems engineering. Professor Sahinidis teaches mathematical optimization, process systems engineering, and scientific computing. He has developed a bioinformatics M.S. program and has taught courses ranging from thermodynamics and metabolic engineering to approximation algorithms and GPU computing. Sahinidis has served on the editorial boards of many leading journals and in various positions within AIChE (American Institute of Chemical Engineers). He received an NSF CAREER award, the INFORMS Computing Society Prize, the MOS Beale-Orchard-Hays Prize, the Computing in Chemical Engineering Award, the Constantin Carathéodory Prize, and the National Award and Gold Medal from the Hellenic Operational Research Society. Sahinidis is a member of the U.S. National Academy of Engineering and a fellow of AIChE and INFORMS.

Engineering Self-Adaptive and Autonomous Systems: A Critical Survey and an Adaptation–Assurance Framework

: Self-adaptive systems (SAS) autonomously reconfigure in response to uncertain environments and evolving requirements. Two decades of research have produced a rich but fragmented body of work spanning architecture, machine learning, uncertainty management, and verification, each strand typically surveyed in isolation. This paper makes three contributions. First, it provides a critical survey, rather than a catalogue, of foundational and contemporary SAS methodologies, organized using the reporting principles of established systematic-review guidelines. Second, it introduces an original synthesizing artifact, the Adaptation–Assurance Gap Map, which positions each class of adaptation mechanism (rule-based, control-theoretic, search-based, model-based, reinforcement learning, deep predictive, federated, and LLM-assisted) against the maturity of assurance evidence routinely available for it, exposing systematically under-assured combinations. The map is governed by an explicit rating rubric, complemented by a technique-maturity model that includes an assurance-cost dimen-sion. Third, it positions the framework against prior SAS taxonomies and converts the findings into a gap-to-research traceability matrix with concrete, falsifiable directions. Using exemplars (DeltaIoT, SWIM, and an ML-component retraining case), we show that the field’s principal weakness is not a shortage of adaptation mechanisms but a shortage of assurance that scales with them in scope, tempo, and cost.

DynBudget: Adaptive Risk Calibration and Dynamic Budgeting for Safe Reinforcement Learning Under Distribution Drift

Ensuring safety constraint satisfaction in non-stationary Reinforcement Learning (RL) environments remains a major challenge, as distribution drift can quickly invalidate previously calibrated risk estimates. To address this issue, we propose DynBudget, a closed-loop Safe RL framework integrating a learned safety critic, temperature-calibrated risk estimation, and a dynamic safety budget. By jointly adjusting the calibration temperature and the intervention threshold based on real-time violation feedback, the safety shield enables rapid adaptation to environmental drifts. We carried out experiments in two stages to fully evaluate the adaptation mechanism as well as the practical robustness of DynBudget. First, we studied the framework in a number of discrete MiniGrid contexts, and then we tested it on various continuous robotic tasks in Safety-Gymnasium. Our empirical results indicate that our approach is able to effectively reduce safety violations, shorten recovery time after a drift, and preserve competitive task performance when faced with abrupt changes in hazards or physical perturbations. Finally, we show that shielding with dynamic budgets is an interpretable and viable approach to Safe RL in autonomous systems.

Learning from the Test: Self-Referential Differential Testing for Deep RL Agents

Deep Reinforcement Learning (DRL) has achieved significant success in complex decision-making problems. As DRL systems are increasingly deployed in real-world applications, ensuring their quality and reliability is paramount. Current works primarily focus on detecting safety-critical failures, often neglecting policy optimality, which can lead to reduced efficiency, user distrust, and economic losses. This oversight, compounded by the inherent"testing oracle problem"for optimality, leaves a significant gap in comprehensively evaluating DRL systems. To address this gap, we propose Delta (Differential Testing for DRL Agents), a novel and comprehensive framework that automatically identifies both safety-critical and optimality bugs in DRL agents. Delta employs a two-phase approach: (1) Safety Testing, where the Agent Under Test (AUT) is evaluated for catastrophic failures while collecting data from its decision-making policy, and (2) Optimality Testing, where this collected data from the prior phase is used to train a challenger agent via Offline Reinforcement Learning. Differential testing is then performed by comparing the challenger agent against the AUT; instances where the challenger achieves higher cumulative rewards indicate optimality issues in the AUT. We demonstrate Delta's effectiveness across five environments. We investigate the effectiveness of three offline RL algorithms (BC, BCQ, and CQL) in generating challenger agents. Experimental results demonstrate that safety testing datasets are valuable for training competent DRL agents. Challenger agents trained with BCQ proved most effective for identifying optimality issues within the framework of Delta. Across the five environments, Delta uncovered an average of 2,518 optimality issues, outperforming the baseline methods by 50.2%.

Learning Adaptive Multi-Task Guidance, Navigation, and Control via Hypernetworks

Autonomous free-flying robots in orbital environments require controllers that are both versatile and resource-efficient, yet maintaining a separate, task-specific policy for each mission profile is architecturally brittle and limits operational flexibility as requirements evolve. We introduce HYPER-GNC, a multi-task reinforcement learning framework in which a hypernetwork maps physics-informed task embeddings to the weights of a shared actor-critic policy, enabling a single compact controller to master four distinct GNC tasks: velocity tracking, docking, inspection, and navigation with obstacle avoidance. The continuous embedding space allows the controller to generalize to novel mission configurations at deployment time without any retraining. Extensive experiments demonstrate that HYPER-GNC achieves sample efficiency comparable to single-task specialists while maintaining stability under significant inertial perturbations and external body wrenches. We further validate the framework on a physical satellite emulator, successfully bridging the simulation-to-reality gap across all mission profiles. Code, trained models, and deployment scripts are made publicly available to support reproducibility.