Skip to content
Book Open access

CoT-RC: Self-Improving UMMs via CoT-based Reflecting and Correcting Intermediate Visual Representations

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 39 references

Abstract

Unified Multimodal Models (UMMs) unify multimodal understanding and image generation, but often suffer from an understanding–generation gap, where correct semantic interpretation does not translate into faithful visual outputs. This gap stems from the lack of reflection and correction over intermediate generation processes, as existing methods supervise the final outputs. Since these intermediate states are implicit and not directly observable, we are motivated to exploit UMMs' own multimodal understanding as semantic feedback to supervise and correct the generation process. In this paper, we propose a training-free framework to improve the generation ability of UMMs through CoT-based reflecting and correcting the intermediate visual representations. The proposed framework consists of two complementary modules. The CoT-based Reflection module performs global semantic diagnosis and refines textual conditions to correct high-level inconsistencies, while the Semantic-Driven Region Correction module evaluates token-level reliability and selectively resamples unreliable tokens to prevent local error accumulation. By injecting semantic feedback into intermediate stages, generation trajectories become semantically observable and controllable. Experiments demonstrate substantial improvements in compositional synthesis, boosting the base model from 68% to 79% on GenEval and achieving a 14% absolute gain on T2I-CompBench.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.