Augmented Intelligence for Professional Recruiting: designing an Advisor-in-the-Loop system to support the job interview process
TL;DR
The central claim is that in high-stakes social evaluation contexts, Augmented Intelligence is best understood not as a compromise between automation efficiency and human oversight but as the epistemically more defensible design choice, because the judgment required cannot be reliably produced by either component in isolation.
Abstract
This paper presents the design and formative evaluation of an Augmented Intelligence (AuI) prototype that supports professional recruiters throughout the job interview process without automating it. We argue for a transition from the conventional Human-in-the-Loop (HITL) paradigm to an Advisor-in-the-Loop (AITL) architecture, in which the human expert operates as a Reasoning Architect — co-constructing the system's evaluative logic before, during, and after each interview rather than functioning as a corrective checkpoint on an otherwise autonomous pipeline. Building on theoretical analysis, interviewer failure taxonomy, and primary field research with recruiters and HR professionals, the prototype's three-layer architecture (Data, Modeling, Governance) and four-phase interaction model (Scenario Designer, Live Interview Dashboard, Reflective Synthesis, Governance Closure) operationalize four non-negotiable governance properties at the interface level: strict traceability, unconditional overrideability, active bias flagging, and explicit accountability closure. A formative evaluation combining heuristic inspection and structured expert review — conducted at prototype stage, prior to summative field deployment — indicates that the design is internally coherent, professionally acceptable, and free of major interaction failures that would undermine its governance objectives. The evaluation also surfaces a behavioural asymmetry between accepting and overriding AI suggestions under time pressure, a finding that refines the design and contributes to the broader HCI literature on automation bias in high-stakes decision-making. The paper's central claim is that in high-stakes social evaluation contexts, Augmented Intelligence is best understood not as a compromise between automation efficiency and human oversight but as the epistemically more defensible design choice, because the judgment required cannot be reliably produced by either component in isolation — a position advanced as a theoretically motivated design argument supported by formative evidence, not as a demonstrated empirical superiority of the Advisor-in-the-Loop model over alternative architectures.