Aleatoric uncertainty for Errors-in-Variables models in deep regression

This directory lists the source code for the article Aleatoric uncertainty for Errors-in-Variables models in deep regression.

Requirements

The software used to produce the results from the article was written in Python 3. If not already installed, the easiest way to set up Python is usually via Anaconda. To use the software, the installation of some additional packages is required. This is discussed below. To avoid any global impacts on the Python install, especially if the system interpreter is used, it might be preferable to do the following in a virtual environment, either in Anaconda or by using the venv module. The Python version used for the results in the preprint is 3.9.7.

Installing additional packages (except PyTorch)

The Python packages to use this software, except for PyTorch which we will discuss below, can be installed by using the file requirements.txt

pip install -r requirements.txt

When using Anaconda, make sure that python is installed in the virtual environment. If not, use conda install python or conda install python=3.9.7 before running the pip command.

When running into version issues:

There is an according requirements_without_versions.txt file that does not insist on the versions from the preprint. In case of a version conflict, e.g. with pre-installed packages, this file can be used instead. When still running into problems, the packages listed in requirements_without_versions.txt should be installed one after the other.

Installing PyTorch and using the GPU

Much of the source code uses the Python library PyTorch. The best way to install PyTorch will depend on individual requirements such as the availability of a GPU and the CUDA version. The corresponding command can be generated on the PyTorch website

This software can be run with or without a GPU. If a GPU is available and PyTorch is set up to use it, it will be used by the scripts described in the Training and pre-trained models Section. As the training was done using a GPU (Tesla K80 and V100), using the CPU instead might lead to different results.

The results in the preprint were created using the CUDA version 11.1.

Installing the Errors-in-Variables Package

To enhance the portability of the software, all source code that does not directly perform data loading, training or evaluation was bundled in a package that is contained in the directory EIVPackage. It can be installed by

pip install EIVPackage/

Installing this package will make 4 modules available to the python environment: EIVArchitectures (for building EiV Models), EIVTrainingRoutines (containing a general training framework), EIVGeneral and EIVData.

Running scripts

All scripts are designed to run inside the Experiments folder.

Training

For training the following two scripts can be used

• train_eiv.py: to train an EiV model.
• train_noneiv.py: to train a non-EiV model

For training they use the JSON-configurations in configurations according to the provided --data keyword, e.g.

python train_eiv --data sine

The keyword to --data should be the short dataname, cf. the JSON configuration files in configurations.

Evaluation

For evaluation of the trained networks the following scripts can be used

• create_tabular.py
• evaluate_metrics.py
• plot_coverage_vs_q.py
• plot_deviation_scatter.py
• plot_prediction.py
• plot_summary.py
• plot_variety_of_coverage_plots.py

Contributing

Will be completed upon publication. The code will be made publically available on a repository under a BSD-like license.