We present a detailed comparison of Bayesian criteria with three non-informative priors - flat, Jeffreys, and scale-invariant - for testing a signal against an unknown background and compare them with the classical frequentist Li-Ma approach in the On/Off problem. We perform Monte Carlo simulations for various background levels and evaluate the Li-Ma and Bayesian criteria by their Type I error rates. We then simulate a nonzero signal and compare the criteria in terms of Type II error rates. We find that both the Li-Ma criterion and the Bayesian criterion with the Jeffreys prior control Type I error well, excluding the case of small background $b<0.5$; the Bayesian criterion with the Jeffreys prior is statistically more powerful than the Li-Ma criterion for $b \lesssim 40$.
Simulations against group-sequential and always-valid frequentist baselines show that flat-prior posterior stopping exactly reproduces naive peeking, that a well-calibrated empirical Bayes prior achieves the lowest estimation error, and that expected-loss stopping minimizes regret only when shipping a null-effect varia...
A computational framework for Bayesian decision-making is developed and it is shown that as long as no action is optimal in every state, Bayes-optimal choice is intractable.
In the application of Bayesian methods, the choice of a prior density for defined model parameters is often a practical issue. In scientific and medical applications, it requires careful justification. In many settings, the properties of the statistical likelihood function and the overall statistical model, within whic...
The most common dynamic borrowing approaches for external data, including how to incorporate multiple historical data sets, are reviewed and used in practice, using publicly available software, to perform important statistical tasks such as model selection, average treatment effect estimation and prior sensitivity anal...
Xin-Xin Chen, E. Braga, Joseph G. Ibrahim et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
