Uncertainty random forest regression

General Discussion
,
#1

Hi - I am using the tensorflow random forest regression and I’m wondering whether there is any way to get an uncertainty estimate out of it? I know I can get RMSE outputs but I guess typically uncertainty is expressed in the band of the standard deviation or the confidence interval so I was wondering whether there is any way to get those?

Thanks a lot!

#3

Hi Lina,

Short answer, not yet directly. However, the following options are available:

  1. The NGBoost paper presents a method to obtain the uncertainty of a prediction. For example, the method can be used to get the uncertainty as a standard deviation. NGBoost in TF-DF is implemented as we speak, and it should be made available soon.

  2. If your features are compatible, an alternative solution is to use the NGBoost’s author implementation compatible with Scikit learn, and to use the Scikit-learn to TF-DF converter.

  3. Finally, a less efficient but likely correct solution is to train a second model to predict the error of the first one. The main drawback of this solution is that you need more training examples as the two models (the one predicting the value and the one predicting the error) should be trained on different datasets.

  4. The sample efficiently of solution #3 can be improved by running an ensemble scheme: For each ensemble round, randomly split the training dataset in two parts, and use each part to train one of the two models.

1 Like