Skip to content

PolyComp: A Polycube-based Benchmark for Compositional 3D Spatial Reasoning in Multimodal Models

Aug 2026 · 0 citations · 24 references
Computer Science

Abstract

We introduce PolyComp, a procedurally generated and verified benchmark that stresses visual recognition and compositional spatial reasoning. In each problem, a model must identify which of four options shows a pair of polycube components that can be combined to form a target solid. The benchmark contains 120 problems across four geometry families, and each problem has three different presentation formats using either a single image or multiple images. The random guessing baseline is 25%. Across the three presentations (360 presented problems per model), GPT-5.6 Sol with max effort attains 50.0% accuracy (95% problem-cluster CI 43.3-56.7%) at a mean cost of \$0.951 per presented problem, Claude Fable 5 with max effort attains 39.4% (33.1-46.1%) at \$0.701, and Gemini 3.1 Pro Preview with thinking level high attains 27.5% (22.8-32.5%), near the 25% random guessing baseline, at \$0.350. The observed accuracy spread across geometry families is larger than across presentation formats. We present a problem development and evaluation protocol, cost and token accounting, and release the 120 problems.

View source

Similar papers

Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket

This work introduces a pioneering exploration of Self-Supervised Learning (SSL) within the SNN, and proposes a novel Spiking Self-Attention (SSA) and Spiking Transformer (Spikformer) that achieves 80+% accuracy on ImageNet.

Zhaokun Zhou, Kaiwei Che, Wei Fang et al. · 69 citations · ⚡10
#artificial intelligence Review Dec 2025

Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025

Investigating how experienced developers use agents in building software, including their motivations, strategies, task suitability, and sentiments finds that while experienced developers value agents as a productivity boost, they retain their agency in software design and implementation out of insistence on fundamental software quality attributes.

Ruanqianqian Huang, Avery Reyna, Sorin Lerner et al. · 19 citations · ⚡1
#artificial intelligence Open access Oct 2022

Adaptive surrogate modeling for high-dimensional spatio-temporal output

An adaptive surrogate modeling method for problems with very high-dimensional spatio-temporal outputs is developed that combines exploration and exploitation to improve the surrogate model accuracy with the fewest possible runs of the expensive physics-based model.

B. Kapusuzoglu, S. Mahadevan, Shunsaku Matsumoto et al. · 17 citations
#artificial intelligence Preprint Feb 2025

`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMs

An adaptive jailbreak attack framework for systematic evaluation of both cascaded pipelines and end-to-end large audio-language models under a unified experimental setting that achieves consistently higher attack success rates across diverse audio-based LLM systems.

Linghan Huang, Bo Li, Huaming Chen et al. · 12 citations · ⚡2
#artificial intelligence Review Open access Oct 2025

Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead

This review provides a systematic literature review of LLM-based Verilog code generation, analyzing 102 papers (70 published and 32 high-quality preprints) from SE, AI, and EDA venues and outlines a roadmap highlighting potential opportunities in LLM-assisted hardware design.

Guang Yang, Wei Zheng, Xiang Chen et al. · 11 citations · ⚡1

From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?

This work introduces Behavior-Outcome Freedom (F), a pre-synthesis diagnostic of signed behavior-outcome rank mismatch, and formalizes its candidate-conditional role through Signed Anchor-Rank Transfer, which preserves validated capability resources, removes runtime orchestration, and conditionally inherits pipeline guidance using a calibrated rule over F.

Binyan Xu, Dong Fang, Haitao Li et al. · 10 citations

Related blog posts