Evaluating covariate balance for long time horizon Markov decision processes
The results demonstrate that, either there is a high risk of bias within existing offline RL studies for treatment recommendations and/or, existing covariate balance metrics are not sufficient to assess such studies, and existing offline RL studies cannot be concluded as being statistically robust.