Hi all,
We’re excited to share a new RFC to enable stochastic rounding in Tensorflow, which is currently open for comment until Nov 30th, 2022. Please take a look, and add your feedback inline or as comments on the pull request.
Summary: This RFC proposes to enable stochastic rounding API in Tensorflow to round big floats to lower-precision floats and integers, as many studies have proven that stochastic rounding has substantial improvements in reducing rounding errors in low precision formats and helps resolve issues like model divergence and quality loss.