Skip to content
Open access

Deep excavation wall design using reinforcement learning

Jun 2026 · Archives of Civil Engineering · Vol 72, pp. 93-107 · 0 citations · 15 references

TL;DR

The results demonstrate the potential of RL-based methods to explore complex design spaces efficiently while respecting physical constraints, highlighting their suitability for supporting automated or decision-assisted design of diaphragm walls.

Abstract

This study investigates the application of reinforcement learning (RL) for obtaining near-optimal designs of diaphragm walls in geotechnical engineering. A physics-based numerical environment is developed to simulate soil–structure interaction, relying on a Winkler spring formulation with pressure-dependent soil springs to approximate the nonlinear response of the ground. This modelling framework allows the agent to evaluate candidate designs through physically meaningful structural responses rather than surrogate performance indicators. To reflect realistic engineering practice, a specifically designed action space and reward function are formulated, incorporating both discrete design decisions and continuous geometric parameters. During training, the agent iteratively proposes a design configuration, observes the response computed by the physical simulator, and updates its policy based on the resulting reward signal. Several common RL algorithms are investigated, including Proximal Policy Optimization (PPO), REINFORCE, and the Parameterized Deep Q-Network (P-DQN), enabling a comparative assessment of policy-based and hybrid value based approaches for this task. The algorithms are evaluated in terms of learning stability, convergence behaviour, and the quality of the resulting design solutions. The results demonstrate the potential of RL-based methods to explore complex design spaces efficiently while respecting physical constraints, highlighting their suitability for supporting automated or decision-assisted design of diaphragm walls.

Read PDF

Similar papers

#reinforcement learning Open access Oct 2026

Seismic Control of a Smart Base-Isolated Building with Nonlinear Behavior Using Deep Reinforcement Learning

DRL is highlighted as a promising data-driven strategy for robust and adaptive control of nonlinear structural systems under partial observability by addressing a critical limitation of passive systems and accelerates the decay of residual vibrations.

Takehiko Asai · 0 citations
Conference Open access Jun 2026

A hybrid modelling method for preliminary design of civil aircraft engine nacelles

Accurately modelling the nonlinear drag map within the design parameter space is a key challenge in preliminary nacelle design. However, due to sharp gradient variations in the drag distribution and the limited size of the available dataset, traditional surrogate models often suffer from insufficient predictive accuracy and poor generalization. To address this challenge, this study systematically evaluates the modelling performance of representative reduced-order models and deep learning–based approaches, and proposes a hybrid modelling framework (PMG) that integrates Proper Orthogonal Decomposition (POD), Multilayer Perceptron (MLP), and Gaussian Process Regression (GPR). The performance of various methods is evaluated and validated using high-resolution numerical simulations across the nacelle design parameter space. The results show that the PMG model reduces the required number of samples by 80% while accurately capturing the characteristics of complex drag distributions. Under small-sample conditions, the PMG model demonstrates superior predictive accuracy compared to traditional reduced-order models and deep learning-based approaches. This framework provides a promising approach for the rapid evaluation of preliminary nacelle designs.

Hao Liu, Chenxing Hu, Xiaochuan Yuan · 0 citations
Open access 2026

Machine-Learning-Based Multi-Surrogate-Assisted Joint Optimization for Hydraulic Fracturing Design and Production Control

: The distribution of hydraulic fractures and production control strategies have significant influences on the fluid flow and production performance of low-permeability waterflooding reservoirs. Traditional approaches typically focus solely on fracture parameters while overlooking production control. To address this limitation, this work proposes a joint optimization framework that simultaneously integrates hydraulic fracturing design and production control. However, the joint optimization of hydraulic fracture distribution and production control requires a large amount of reservoir numerical simulation. To solve this problem, a novel adaptive multi-surrogate-assisted differential evolution (AMSADE) algorithm is developed. The AMSADE algorithm utilizes a surrogate model pool comprising radial basis functions, polynomial response surfaces, and deep neural networks. Additionally, an embedded discrete fracture model (EDFM) is adopted for simulation of fractured reservoir flow and optimization evaluation. The proposed method was applied to a two-dimensional waterflooding reservoir model. The results demonstrate that the algorithm converges rapidly, requiring only 200 numerical simulations to achieve optimal performance. Compared with the classical differential evolution algorithm, the net present value was improved by 17.5%. Overall, the proposed joint optimization framework based on the AMSADE algorithm successfully and simultaneously determines the optimal fracturing and production control parameters.

Xiaopeng Ma, Bin Zhang, Jinsheng Zhao et al. · 0 citations
Preprint Jul 2026

Co-Design of Aeroelastic Systems with Deep Reinforcement Learning

Control co-design considers the physical system and its controller together, enabling the strong coupling between system design and control to be uncovered and exploited. This is especially relevant in aeroelastic flight systems, where structural, aerodynamic, and control design choices jointly determine manoeuvrability and efficiency. This paper presents a model-free nested co-design framework for aeroelastic systems using deep reinforcement learning, in which a design-conditioned control policy is trained with proximal policy optimisation while an outer loop updates a distribution over candidate design parameters. The approach is evaluated on three case studies of increasing complexity: a spring-mass-damper system, a pitch-plunge-flap aerofoil, and a highly flexible high-aspect-ratio glider performing a thermal-soaring mission in a stochastic environment. Across these case studies, the framework is shown to progressively concentrate the design search towards high-performing regions and to outperform policies trained on randomly sampled designs. The results also show that reward shaping plays an important role in enabling stable learning in partially observed and stochastic environments. In the final glider case, the method jointly addresses wing design, flight control, and mission-level behaviour in the presence of aeroelastic coupling and atmospheric uncertainty. These results highlight the potential of model-free co-design for complex aeroelastic systems in which design, control, and mission objectives are tightly coupled.

Y. Li, Urban Fasel · 0 citations
Conference Open access 2026

Passive earth reinforcement of quay walls: Numerical analysis and experimental framework

Deeper berth requirements due to increasing vessel sizes pose a significant challenge, rendering quay wall structures as critical infrastructure for the maritime logistics chain. This paper presents preliminary numerical analyses performed using two-dimensional finite element modelling to evaluate the global structural response of the quay wall for three passive soil-cement block configurations: Small, Wide, and Deep. The presented numerical analyses form the initial phase of a broader research programme focused on establishing a validated design framework for cement-treated soil reinforcement at the passive side of quay wall structures. This phase will be followed by 1g model-scale shaking table tests to verify the performance of the defined reinforcement configurations under dynamic loading conditions. The initial numerical results indicate that the Wide Block configuration provides superior reinforcement performance compared to the Deep Block arrangement. Increasing the improvement depth beyond a certain threshold was found to produce only marginal additional reductions in structural demand. Accordingly, the results are interpreted with reference to soil–structure interaction mechanisms controlling the quay wall response. These findings will guide the configuration of the planned shaking table experiments for dynamic soil– structure interaction assessment and provide insight into how mobilized lateral equilibrium conditions influence and govern overall wall deformation behaviour.

Z. N. Kutlu, İ. E. Kılıç, M. E. Selçuk et al. · 0 citations