Skip to content
Open access

Understanding Cognition-Induced Risks in Agentic AI Systems

Aug 2026 · IEEE Intelligent Systems · pp. 1-8 · 0 citations · 25 references
Computer Science

TL;DR

This work systematically analyzes risks induced by expanding cognitive capabilities, following a three-level framework defined by their cognitive scope, from physical cognition to social cognition, and finally to self-referential cognition.

Abstract

Frontier agentic systems powered by large language models (LLMs) exhibit human-like patterns of cognition. As these systems become deeply integrated across different domains, their cognitive engagement raises critical concerns for human society that remain insufficiently studied. To address this gap, we systematically analyze risks induced by expanding cognitive capabilities, following a three-level framework defined by their cognitive scope, from physical cognition to social cognition, and finally to self-referential cognition. We study their potential risks to human agency, autonomy, and control capability, corresponding to each cognitive level. We finally propose strategies to mitigate these risks and enhance the controllability of agentic AI systems, ensuring their long-term safe development.

Read PDF

Similar papers

Jul 2026

Cognitive Convergence: Deep Similarities Between Large Language Models and Human Cognition

LLMs are widely regarded as alien intelligences, systems whose cognitive operations are fundamentally unlike our own. Apparent similarities to human cognition are therefore often seen as the result of anthropomorphic projection. We argue that this framing is mistaken. LLMs clearly differ from humans in important respec...

C. Sripada, Richard L. Lewis · 0 citations
Review Aug 2026

A Taxonomy of Cognitive Capability Gaps in Generative and Agentic AI

A taxonomy-driven survey of the major cognitive capability gaps that continue to constrain the development of Cognitive AI and outlines a conceptual Adaptive Cognitive Intelligence Architecture (ACIA), providing a foundation for future progress toward Cognitive AI and, ultimately, Artificial General Intelligence (AGI).

Taye Akinrele, Sindhuja Penchala, Noorbakhsh Amiri Golilarz et al. · 0 citations
Review Open access Aug 2026

Seemingly conscious AI risks

A unified framework connecting empirical hallmarks of consciousness attribution to a structured risk taxonomy of Seemingly Conscious AI (SCAI), AI systems that exhibit hallmarks which elicit consciousness attribution from users is provided.

Ben Bariach, P. Schoenegger, M. Bhaskar et al. · 3 citations
Jul 2026

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

ACE-Brain-0.5 is presented, a unified embodied foundation model that organizes robot intelligence into five coupled functions: spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement, and SSR+, which extends Scaffold-Specialize-Reconcile with a Reactivate stage after task-vector...

Zi-Yang Gong, Hao-Ming Gu, Ze-Hang Luo et al. · 3 citations
Preprint Aug 2026

Assessing mentalization in humans and large language models

Different capacities for mentalization across LLMs are demonstrated, and cognitive computational modeling is highlighted as a formal method for assessing comparative intelligence across humans and machines.

Aamir Sohail, Xintong Zhong, Arkady Konovalov et al. · 0 citations
Open access Aug 2026

Cognitive Entanglement: Toward a Developmental Framework of the Human-AI Coevolutionary Leap

If AI is to support human cognitive growth, design must move beyond answer provision and efficiency maximization toward the organization of productive human-AI relations: relations that challenge users’ initial assumptions while providing support appropriate to the task and the user's level of expertise.

Xiaokun Wu, Min Chen, Giancarlo Fortino · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.