Post-Placement Optimization with Pin-Level Graph Isomorphism Network and Recurrent Q-Learning
TL;DR
A deep recurrent Q-network (DRQN)-based framework for post-placement circuit optimization that jointly leverages all three operations to enable integrated optimization of timing and power and design a reward function that applies symmetric logarithmic compression to each metric at a fixed scale, allowing controllable trade-offs across the design.
Abstract
Post-placement optimization has been actively studied using techniques such as gate sizing, threshold-voltage (Vth) swapping, and buffer insertion. Recently, many approaches have adopted machine learning, particularly reinforcement learning (RL), to automate these optimizations. However, existing RL-based methods fall short of comprehensive modeling of global circuit characteristics, failing to jointly optimize timing and power while covering only narrow combinations of optimization actions. In this work, we propose a deep recurrent Q-network (DRQN)-based framework for post-placement circuit optimization that jointly leverages all three operations. The circuit is sequentially clustered based on timing criticality, and each cluster is modeled using a pin-level Graph Isomorphism Network with Edge features (GINE) combined with a long short-term memory (LSTM) to capture sequential dependencies across gates. Based on this representation, the RL agent simultaneously determines gate sizing and Vth selection, and buffer insertion for each gate and its driving net. To enable integrated optimization of timing and power, we design a reward function that applies symmetric logarithmic compression to each metric at a fixed scale, allowing controllable trade-offs across the design. Experimental results demonstrate that the proposed framework reduces TNS by 96% and total power by 22% on average, compared to 69% and 5% by a timing-only RL baseline, and ablation studies validate the contributions of the GINE model and recurrent architecture.