This work develops a six step BBN framework and illustrates it to model customer intention to consult a doctor in an alternative healthcare system and reveals that while self efficacy appears to be a major factor, its actual causal impact is small.
Abstract
Bayesian Belief Networks (BBNs) are powerful tools for decision-making under uncertainty. However, building their structures and estimating parameters are difficult. Currently, researchers must choose between relying on expert judgement or using large datasets to learn the structure and parameters of the network. We propose a new methodology using Large Language Models to bridge the gap between expert opinion and data-driven learning. This approach uses a panel of AI agents to estimate probabilities based on specific personas and context. We then apply a trimmed-mean rule to remove noise from these responses. We develop a six step BBN framework and illustrate it to model customer intention to consult a doctor in an alternative healthcare system. The model reveals that while self efficacy appears to be a major factor, its actual causal impact is small. In contrast, subjective norms have a much stronger effect in modelling customers'intention. The most effective strategy is to improve both confidence and community norms simultaneously.
Today, the use of Artificial Intelligence (AI) is rapidly increasing in many areas of society. While model performance on various tasks continue to impress, it does so at the cost of increased model complexity, such that most state-of-the-art AI models are effectively black boxes. Where human-made decisions typically are accompanied by human-understandable explanations detailing the reasoning behind the decision, incorporating advanced AI as part of a decision-making process reduces the transparency of that process significantly. Yet, the ability to explain decisions is essential for there to be understanding and trust. As a response to this, Explainable Artificial Intelligence (XAI) has emerged as a field that aims to provide explanations of model behaviour. Methods categorised as post-hoc are designed to generate explanations for black box models after training, at no cost to model performance. In parallel with this, extensive work has been done in the field of causality to formalise the structure of human-understandable, causal explanations. This work presents a comprehensive literature review of the current state of the subfield of XAI that consist of causality-motivated post-hoc XAI methods. In order to clearly define causal XAI, a causal framework for categorising XAI is introduced, and three types of post-hoc XAI methods are identified: observational methods, internally causal methods and externally causal methods. Finally, externally causal XAI is argued a promising direction for reliable and understandable post-hoc XAI, with the ability to generate counterfactual explanations using a meaningful vocabulary, in line with the definition of counterfactual used in causal theory.
Anna Rodum Bjøru, Helge Langseth, Inga Strümke et al.· Machine-mediated learning· 1 citation
While participants rated hedged and unhedged AI as equally trustworthy and likely to be correct, they were significantly less likely to follow hedged advice in a binary choice, and how linguistic markers can be used to calibrate user reliance to model certainty is discussed.
Laura Spillner, Johanna Rockstroh, Nina Wenig et al.· International Conference on...· 0 citations
It is shown that a key factor is the error correlation structure between human and AI predictions, and when the AI's prediction errors are negatively correlated with those of the human, the decision maker can construct robust strategies which guarantee improvements in expected utility.
Amortised Sequential Information Gathering (ASIG), a fine-tuning approach that amortises Bayesian Experimental Design into LLM policies via a multi-turn extension of Group Relative Policy Optimisation with an Expected Information Gain reward, is introduced.
Jakob Hartmann, James Harvey, Jhonathan Navott et al.· 0 citations
Understanding how experts make decisions and being able to transfer that knowledge is important, especially in complex engineering applications. It is highly valuable for training novices, improving the performance of human-machine systems, and potentially enabling fully autonomous systems that perform as well as human experts. However, an expert's decision-making strategy, developed through years of experience, is often not directly accessible, since the implicit preferences and decision rules involved can be difficult to specify explicitly. This has motivated the use of observed decisions made by the expert to learn an interpretable model that captures the expert's decision-making process. In this work, we develop an inverse optimization approach to jointly learn the decision-maker's preferences (or perceived costs) and the decision rules governing their choices. We demonstrate the general applicability of our approach using three case studies that consider a shift assignment problem, a production planning problem, and a real-world routing problem, respectively. Across these case studies, modeling both perceived costs and decision rules leads to better predictions, highlighting the value of the proposed framework and its greater flexibility in capturing and replicating expert decision making.
Anurag Holani, Rishabh Gupta, J. Wassick et al.· 1 citation
Counterfactual explanations (CEs) enhance the interpretability of machine learning models by identifying the smallest change to an input required to obtain a desired output. Although CEs are conventionally formulated as a distance-minimization problem, the theoretical basis of this formulation has received limited attention. We show that a distance-minimization-based CE is mathematically equivalent to the maximum a posteriori (MAP) estimate of a Gibbs posterior within the generalized Bayes framework, specifically when a distance-based prior is used. We call this formulation the Distance-Prior Generalized Bayes CE (DP-GBCE). Building on this posterior perspective, we introduce two decision rules beyond MAP within a unified framework: a Bayes decision that minimizes expected decision loss and CVaR-CE, a risk-averse decision rule. We also propose an extension that uses Bayesian model weights to mix the posterior distributions of multiple models, thereby accounting for model multiplicity, where several models have comparable predictive performance. Finally, we define metrics for evaluating both individual CEs and the posterior distribution as a whole, and use experiments on simulated data and Google Trends data to quantify the trade-offs among the decision rules.