Freshness-constrained audit capacity (FCAC) is developed, a decision-support framework that treats automation as an authorization decision constrained by action risk, evidence freshness, and shared review capacity.
Abstract
Fraud operations must allocate events among automatic approval, analyst review, and automatic blocking even though the labels needed to evaluate these actions are selective and delayed. Predictive scores order cases, but they do not show whether the evidence is current and representative enough to delegate an action to the model. We develop freshness-constrained audit capacity (FCAC), a decision-support framework that treats automation as an authorization decision constrained by action risk, evidence freshness, and shared review capacity. It evaluates candidate action regions from mature randomized audits and a prespecified temporal allowance. Supported regions are automated; unsupported regions remain in review. The resulting decision record reports evidence age, audit demand, total review workload, value exposure, and compatible temporal change. We show that current action risk is unidentified without restricting unobserved label evolution. Under representative randomized audits, label-independent evidence windows, and a prespecified condition linking historical and current action risk, we derive simultaneous finite-sample control of unsafe authorization. Chronological evaluations with simulated audits on IEEE-CIS, ULB-Worldline, and Elliptic++ yield zero-drift automation rates of 84.4%, 67.4%, and 81.3%, with total review workloads of 24.1%, 46.0%, and 43.1%. The experiments reveal an audit-capacity trade-off: sparse auditing delays authorization, whereas intensive auditing eventually increases workload. A separately specified BAF stress test further indicates that fallback thresholds must reflect candidate-specific evidence rather than a common fraction of the risk limit. These findings identify audit freshness and analyst capacity as joint design considerations for fraud decision support.
Auditability in high-risk AI requires more than explanation narratives: reviewers must be able to retrieve and verify decision-bound evidence, including the decision record, model/configuration state, explanation artefact, and audit-log event. We propose a lightweight engineering pattern for auditready Explainable AI (XAI) that (i) packages each decision into a compact evidence bundle, (ii) exposes resolvable trace-links to an evidence store, and (iii) validates structural audit readiness through objective, tool-agnostic checks. The check suite covers both run- and decision-level properties, such as evidence presence, bundle completeness, decision coverage, pinning/provenance coverage, trace-link resolution, and audit-log soundness, and can be layered on top of standard Machine Learning Operations (MLOps) tooling. We demonstrate feasibility on a reproducible mini-case (seed-controlled synthetic intrusion detection system (IDS) anomaly detection) and show how the checks support fast localisation of common audit gaps, including missing bindings, stale links, inconsistent pins, and auditor role-based access control (RBAC) resolution failures. In an audited scope of $N=30$ decisions, all required artefacts were present and independently retrievable under an auditor-equivalent access profile, enabling verifiable navigation from decisions to evidence.
António Gonçalves, J. Ameixa, A. Correia· 2026 6th International Confe...· 0 citations
Automated applicant tracking systems increasingly decide who advances in hiring, and litigation and regulation now demand that those decisions be auditable. Existing tools sit at two extremes. Group fairness metrics such as the disparate impact ratio summarize a whole population but cannot say which individual decisions were unfair or why, while local explainers such as SHAP attribute a single prediction but are not connected to the legal standard by which hiring bias is judged. We present the AI Bias Firewall (AIBF), a method that audits an applicant tracking system one decision at a time. AIBF neutralizes a candidate's protected-attribute proxies, re-scores the decision, and measures the resulting counterfactual shift, which yields a signed per-decision bias in score points, a flag for decisions the protected attributes changed, and a plain-language explanation naming the responsible factors. We evaluate on two real public datasets, Adult and COMPAS, rather than on synthetic data. The per-decision counterfactual shift is faithful, aggregating to reproduce the known group level disparity, for example a mean shift of +7.5 points for the privileged group and -8.0 for the disadvantaged group on Adult, consistent with the measured statistical parity difference. AIBF identifies the decisions that protected attributes flipped with an area under the ROC curve of 0.963 on Adult, against 0.672 for a baseline that flags by group membership, and it identifies the harmed candidates so precisely that reviewing only five percent of decisions surfaces fifty-five percent of them, against six percent under group based review. We also report a limitation: correcting flagged decisions raises the disparate impact ratio substantially but not to legal parity, because features labeled as merit carry residual proxy correlation. AIBF is released under the Apache 2.0 license with code and experiments.
Selective algorithmic auditing requires deciding which process evidence should receive attention when exhaustive review is infeasible. This Review introduces a four-layer framework that connects process representation, learning, inspection allocation, and governance within a single budgeted sequential decision problem over event streams. Unlike prior reviews centered on predictive process monitoring, explainability, cost analysis, or bibliometric structure, the proposed framework examines how these functions interact when human review, computation, latency, and documentation capacity are constrained. A structured and targeted survey of 89 unique publication families is used to illustrate and critically examine event-log, Petri-net, graph, object-centric, neural, uncertainty-aware, sequential, bandit, reinforcement learning, and audit architecture approaches. The reviewed evidence indicates that substantial bodies of work address the individual layers, but cross-layer evaluation remains fragmented and uses heterogeneous datasets, objectives, and validation protocols. The synthesis identifies five priorities: audit-ready benchmarks, explicit inspection budget protocols, calibrated uncertainty, transfer across organizational contexts, and reproducible governance interfaces. The main contribution is a computational framework and a corpus-bounded research agenda that connects representation, learning, inspection allocation, and governance for selective algorithmic auditing.
Héctor R. Becerril Villamil, Vladimir Rodriguez Perez, D. Sanín-Villa et al.· Machine Learning and Knowled...· 0 citations
IT audits require auditors to judge whether heterogeneous organizational evidence satisfies semantic security and compliance controls. This judgment is difficult to automate because relevant evidence is distributed across policies, records, spreadsheets, and operational artifacts, and because audit conclusions depend on evidentiary sufficiency rather than keyword matching. We present IntelliAudit, a retrieval-grounded multi-agent system for IT audit evidence evaluation. Given a control and an evidence corpus, IntelliAudit retrieves relevant artifacts, generates an evidence-grounded assessment, challenges adverse findings, adjudicates disagreements, and produces an auditor-facing recommendation with cited evidence, rationale, missing-evidence analysis, and remediation guidance. We instantiate IntelliAudit on ISO/IEC 27001 and evaluate it across multiple simulated organizations using expert auditor review and audit-readiness user feedback. The evaluation shows that IntelliAudit can support control interpretation, evidence-grounded reasoning, and audit-preparation workflows, while also revealing the importance of human oversight for calibrating sufficiency judgments and correcting overly permissive recommendations. These results suggest that retrieval-grounded multi-agent systems can assist audit evidence review, but should remain decision-support tools rather than autonomous certification systems.
Allison Wilson, S. Sabet, Diar Shakimov et al.· 0 citations
An AI audit record is useful only if its durability and trust boundary are explicit. Returning a guarded decision before any durable write minimizes latency, but it cannot guarantee that evidence survives an immediate crash. We rebuild RuntimeGuard-AI around this constraint. The resulting research prototype binds each deterministic policy decision to the exact policy source, commits a privacy-minimizing record at a caller-selected synchronization boundary, and returns an Ed25519-signed receipt that states whether that boundary completed. After restart, the engine validates framed records, manifests, shard placement, sequence continuity, and replay identity. A separate attestation path groups committed records into chained, signed Merkle epochs that an auditor verifies with an externally obtained key. On an Apple M4 Pro at four worker threads and 2,048-byte prompts, buffered signed evidence reaches 27,193 requests/s with 141.9 microseconds median latency. Per-record data and full synchronization reduce throughput to approximately 242 requests/s and raise median latency to 16.0 ms. Sealing a 100,000-record signed epoch takes 97.0 ms. The result is a measured durability-latency trade-off, not a"free"asynchronous audit path. The prototype does not prove model execution, prevent a compromised signer from forking history, or establish legal conformity.
Public cryptocurrency archives may appear usable when files exist, although factor research requires observations available and executable at each decision time. We audit public Binance BTCUSDT USD-M perpetual-futures data using event, publication, and availability times and separate proposal from deterministic auditing, evaluation, and holdout access. An initial gapless five-minute requirement for trade, mark, index, and open interest failed: the longest unrepaired intersection was 304.5729166666667 days. A disclosed revision made trade, mark, index, and realized funding the core streams and made open interest optional because its publication time was unverified. The revised mask retained 727 complete UTC days and supported a 436/145/146-day train, validation, and historical-holdout split. On 80 frozen known-rule templates, the auditor detected 40/40 violations and rejected 0/40 legal templates. Across ten null-signal paths, full auditing reduced mean false passes from 0.2910 to 0.0625. Under matched valid-candidate budgets, the audited adaptive agent tied random search and did not establish superiority. In the one-time historical holdout, all evaluated runs had positive IC but negative net Sharpe under primary costs. We therefore report a scoped negative result rather than a profitability or agent-superiority claim.
Baocheng Zeng, Jin-Hao Yang, Pei-Lin Han et al.· 0 citations