A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning
Results show that visual topology, multiplicative representation, and downstream value-estimation design can substantially affect replay-free Q-learning, and that their benefits should be evaluated jointly with computational complexity.