ECRS-TRPO: An Edge Cache Replacement Strategy Based on Trust Region Policy Optimization

Abstract

: Edge caching is a key technology for improving video transmission efficiency, but traditional cache replacement strategies struggle to adapt to dynamic network environments and heterogeneous user requests. To address this issue, a novel cache replacement framework named Edge Cache Replacement Strategy based on Trust Region Policy Optimization (ECRS-TRPO) is proposed. This framework leverages Trust Region Policy Optimization (TRPO), a deep reinforcement learning algorithm, to dynamically adjust cached content. Through interactive learning between the agent and the caching environment, long-term rewards are maximized. Furthermore, a Transformer-based soft hit mechanism is introduced. When a cache miss occurs, similar content is provided as a fast alternative access, thereby reducing transmission latency. Experimental results on two benchmark datasets show that the proposed ECRS-TRPO achieves average hit rate improvements of 3.75% and 11.9% over seven baseline models under increasing cache capacity settings, respectively. These results verify its superior performance in edge cache replacement tasks.