Skip to content

Author

Machi Shimmei

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Aug 2026

Vibe Compiler: A Research-Logic Synthesis Tool That Runs without Prompt Engineering -Toward Enhancing Metacognition for Sustaining Agency in the Age of Generative AI-

Used as a capable servant, generative AI has greatly accelerated intellectual work, yet it also risks eroding human epistemic agency by encouraging uncritical acceptance of AI-generated reasoning. Preserving that agency calls for mechanisms that augment human metacognition during AI-assisted work. We therefore propose the Synthesis-Analysis Reciprocity Model, which views intellectual construction as a reciprocal interaction between two cognitive functions. Synthesis selects and combines the components of the artifact; Analysis evaluates them critically against objective indicators and constrains the Synthesis that follows. Grounded in this model, we present the Vibe Compiler, a research-logic compiler that helps researchers turn vague intuitions (Vibes) into coherent research logic. The system attempts to compile those intuitions against a paper ontology of 16 academic parameters. It treats compilation failures as signs that logical components are missing. Rather than fill those gaps autonomously, it returns reflective questions that prompt researchers to develop the missing reasoning themselves. We further characterize the origins of structural gaps along two orthogonal dimensions: cognitive function (Synthesis versus Analysis) and executing agent (human versus AI). The four resulting types of origin give a principled way to identify where breakdowns in intellectual construction arise. Crucially, our design implements the type in which the AI probes its own synthesized output, itself driven by the user's Vibes, and thereby stimulates human metacognition. This choice raises researchers from passive"Makers"of the output into"Managers"who critically direct and validate what the AI produces. In a prototype on NotebookLM and Gemini, AI behavior depended less on prompting than on the structure of the knowledge supplied. The framework spans a learner layer and a researcher layer.

R. Mizoguchi, Tomoki Aburatani, Kento Koike et al. · 0 citations
Open access Jul 2026

How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay Drafts

This study examines feedback in English as a Foreign Language (EFL) writing contexts, focusing on written corrective feedback (WCF). Large language models (LLMs) can provide WCF at scale, but aligning them with pedagogical best practices remains an ongoing challenge. WCF meeting criteria like factuality or relevance may still be unsuitable for learning contexts, highlighting the need for extrinsic evaluation based on the learner's perspective. We deployed WCF systems in a university-level EFL class with nearly 2,000 students, collecting over 20,000 drafts. We evaluated the generated WCF from two perspectives: intrinsic evaluation by experienced English teachers using a rubric, and extrinsic evaluation via student feedback and engagement metrics. Results revealed low alignment between teacher expert ratings and student feedback. These findings suggest that traditional expert evaluation alone may not fully capture WCF's usability or helpfulness from the learner's perspective, highlighting the importance of learner-centered evaluation frameworks for AI-based applications in language education.

Steven Coyne, Diana Galván-Sosa, Ryan Spring et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.