See if aleatoric uncertainty learning can be improved

Starting the analysis now on 0acf6161

Results when replacing total_unc with sigmas:

energy_efficiency:

before:

non-EiV
------
rmse 0.13730(0.00092)                                                                                                                                                         logdens 0.96744(0.00784)                                                                                                                                                      bias 0.00411(0.00105)                                                                                                                                                         coverage_numerical 0.58894(0.00255)                                                                                                                                           coverage_theory 0.40413(0.00100)                                                                                                                                              coverage_normalized 0.92648(0.00187)                                                                                                                                          res_std 0.83661(0.00386)                                                                                                                                                      EiV                                                                                                                                                                           -----                                                                                                                                                                         rmse 0.13850(0.00102)                                                                                                                                                         logdens -0.65559(0.00980)                                                                                                                                                     bias 0.00380(0.00125)                                                                                                                                                         coverage_numerical 0.64539(0.00185)                                                                                                                                           coverage_theory 0.47878(0.00094)                                                                                                                                              coverage_normalized 0.92402(0.00159)                                                                                                                                          res_std 0.86088(0.00329)

after:

Non-EiV
-----
rmse 0.13722(0.00093)
logdens 0.96792(0.00787)
bias 0.00413(0.00104)
coverage_numerical 0.58875(0.00258)
coverage_theory 0.48670(0.00149)
coverage_normalized 0.87905(0.00231)
res_std 0.83624(0.00386)
EiV
-----
rmse 0.13847(0.00102)
logdens -0.65515(0.00976)
bias 0.00381(0.00125)
coverage_numerical 0.64556(0.00183)
coverage_theory 0.58714(0.00130)
coverage_normalized 0.83774(0.00219)
res_std 0.86052(0.00328)

For kin8nm, with sigma as total unc

Non-EiV
-----
rmse 0.29004(0.00289)
logdens -0.20341(0.00733)
bias 0.01993(0.00615)
coverage_numerical 0.60913(0.00495)
coverage_theory 0.61245(0.00154)
coverage_normalized 0.93569(0.00206)
res_std 0.92381(0.00668)
EiV
-----
rmse 0.28103(0.00239)
logdens -1.79407(0.00566)
bias -0.00536(0.00476)
coverage_numerical 0.67221(0.00348)
coverage_theory 0.67740(0.00148)
coverage_normalized 0.93103(0.00171)
res_std 0.92081(0.00581)

marked this issue as related to #12 (closed)

Check carefully, whether updating std_y is done correctly

added long term label

closed