NORMAN WS model v0.1: QMRF, model card and SusDat predictions

Abstract

A regression model for the solubility of organic chemicals in pure water at 15 to 30 °C (log10 mol/L): the mean of gradient-boosted trees (XGBoost and LightGBM, five seeds each) and a message-passing graph neural network (Chemprop D-MPNN, five repeats, with the melting point as an auxiliary task). It was trained on 12 472 structures with measured values from REACH study results, the EPI Suite data and public collections, and on 508 structures for which only an upper bound is known ("below x mg/L"), which enter the fit one-sidedly. In cluster cross-validation of the recipe (with three seeds per tree learner) on the development set the RMSE is 0.91 log units (R² 0.82; 81 % of the predictions within a factor of 10). The external test used a locked set of 732 structures that no public collection holds: 533 with a measured value from REACH study results and 199 for which REACH gives only a bound or "miscible". On the 203 structures of its strict core (neutral, not ionisable, with a study report of their own) that have a value from all three models, after an outlier rule fixed in advance, the model fitted without that set reached RMSE 0.86 (R² 0.85), against 1.87 for WSKOWWIN and 1.11 for OPERA 2.9, and a mean absolute error of 0.59 against 0.93 and 0.84 (without the outlier rule, on 207 structures, the RMSE is 0.98 against 1.94 and 1.22); on all 533 labels it reached 1.12 (R² 0.78). Two of the four targets set before the test were met (the strict core, and the structures measured below −6 log mol/L) and two were not: on structures for which only a bound is known the model could not be told apart from WSKOWWIN, and the reliability statements frozen before the test were too optimistic (the 90 % interval held 84.4 % of the labels). They were recalibrated with the final set, so no untouched data remain to test them again. Solubility is compressed at both ends: very poorly soluble substances are predicted too soluble (by 2.3 log units on average for the 16 final-set structures measured below −8 log mol/L that were not set aside by the outlier rule), and the reliability statements do not see this. The production model is a refit that also uses the final set: its measured values except 17 labels set aside by the outlier rule, and 117 structures with only a bound "below"; for at least one of the 17 the measured value agrees with published values, so the rule can remove correct values that all three models miss (QMRF 7.9). Every prediction comes with an applicability-domain level and, except for structures that keep a net charge, the probabilities of lying within a factor of 10 (R10) and of 3 (R3) of the measured value and a 90 % prediction interval. The model was built for the screening of exposure and mobility, to fill data gaps in the NORMAN Substance Database (SusDat) and to supply water solubility as an input for other NORMAN models. It does not replace a measurement where a decision hangs on the value. The full documentation is the QMRF (QMRF_NORMAN_WS_model_v0_1.pdf); README.md describes every file of this record. Not in this record: The 15 model files (five XGBoost, five LightGBM and five Chemprop networks). Fitted on all training structures, they reproduce the training values closely, including the REACH study values. The measured values from the REACH study results: every label behind which a value of group R stands (REACH, including what reached public collections through eChemPortal; rows that probably stem from REACH without a visible REACH origin are not recognised by this rule, QMRF 3.7), all labels and bounds of the final set, and the study values behind them. In the QMRF itself, measured values are visible as points in Figures 1 and 2, and its field 7.9 places the REACH value of one substance between two published values; the table there names the 17 substances whose labels were set aside and the direction of the miss, not the values. The locked final set and the full result of the final test per structure (abschlusstest_v0_1_strukturen.csv, md5 in QMRF 9.3) contain such values. Bounds ("below" and "above" a value) are not given either, because nearly all of them come from REACH studies. Both are available on request from the author (peter.vonderohe@uba.de), under the terms of use of the ECHA REACH Study Results: for research, education, safety data sheet compilation and classification and labelling only, and to recipients who accept the same terms in writing. Sister model: NORMAN Kow model v0.1, https://doi.org/10.5281/zenodo.23205470