Sample-Efficient Model-Based Reinforcement Learning for Autonomous Droplet Navigation via Controller Initialization

Abstract

Model-based reinforcement learning (MBRL) is well suited to controlling physical systems that resist analytical modeling, but its reliance on real-world interaction makes data collection the dominant cost. This paper quantifies the effect of initialization choice on this cost, on a newly adapted robotic platform that guides a liquid droplet on a tilt-actuated Labyrinth using MBRL. Unlike rigid microrobots, liquid droplets exhibit deformability and pinning, rendering their response to actuation nonlinear and history-dependent. A closed-loop gain-tuned PID controller, individually optimized per geometry, succeeds in at most one of five trials. The control policy is learned directly on the physical system, with no simulator, and two schemes for seeding the world model are compared, namely random exploration and rollouts from the same tuned PID controller. PID initialization reaches the same terminal success as random initialization while reducing the number of physical training episodes by 30-50%. This repurposes a weak classical controller as a sample-efficient prior for hard-to-model systems on costly hardware.