I use TensorFlow.js. Tensor data have the same type (dtype). In my case study, all data are integers with discrete distribution (typically, one of my input feature varies between 1 and 20).
My feeling (not certain at all) is that choosing i32 is “natural” and computations at training time are more efficient.
My question is: has the choice between i32 and f32 impact (beyond efficiency) on the model accuracy at training/prediction time? Since most of my input data are discrete with fixed variation intervals, I have the bad feeling that using Gauss distribution for normalization is irrelevant and/or i32 (to the dtriment of f32) is the good choice?
Sorry about this naive question…
Your question is quite insightful and touches on several important aspects of machine learning model development, particularly in the context of TensorFlow.js and data types. Let’s unpack the various elements of your query to provide a comprehensive answer.
Data Types: i32 vs. f32
In TensorFlow.js (as in most machine learning frameworks), i32
represents 32bit integer data, while f32
represents 32bit floatingpoint data. The choice between these two data types can indeed impact both the efficiency and the behavior of your machine learning models, though perhaps not always in the way one might initially think.
Efficiency
 Computational Efficiency: Generally, integer operations can be faster and more memoryefficient than floatingpoint operations, especially on devices optimized for integer arithmetic. However, the degree of this efficiency gain can vary widely depending on the specific hardware and the nature of the computations being performed.

Memory Usage: Using
i32
can also reduce memory consumption, as you’re using the full precision of the storage for meaningful data, without the overhead that comes with representing floatingpoint numbers.
Impact on Model Behavior and Accuracy

GradientBased Optimization: Most machine learning models, especially those involving deep learning, rely on gradientbased optimization techniques (like stochastic gradient descent). These techniques inherently depend on the calculation of derivatives, which implies a continuous space. Even if your input data are integers, the model’s internal computations, including weights and activations, often benefit from the flexibility of floatingpoint representation (
f32
), which allows for fractional adjustments during learning. 
Normalization and Scaling: You mentioned concerns about normalization, particularly using a Gaussian distribution. While it’s true that normalization is often applied to data that follow or approximately follow a Gaussian distribution, the goal of normalization (making the data have a mean of 0 and a standard deviation of 1, for instance) is to standardize the scale of the features, improving the convergence behavior of the training process. This is beneficial regardless of the underlying distribution of the features and is generally applicable to both
i32
andf32
data types. However, the actual normalization computations would typically require floatingpoint arithmetic to avoid losing the nuance in the data distribution.  Quantization: In some cases, especially in deployment scenarios where efficiency is critical, models are quantized to use lowerprecision arithmetic (like integer operations), but this is usually done after training a model in floatingpoint precision to ensure that the training process can fully exploit the continuous nature of the optimization landscape.
Practical Recommendation
Given your case study, where the features are discrete and have a limited range, starting with f32
for model variables and computations is generally advisable for the reasons outlined above, particularly to preserve the fidelity of gradientbased optimization. If efficiency becomes a critical concern, especially in deployment, you might explore quantization techniques to convert the model to use i32
or lowerprecision arithmetic, bearing in mind that this can introduce some degree of approximation and potentially affect model accuracy.
Conclusion
While using i32
for input data that are inherently integer might seem natural and more efficient, the broader context of machine learning model development and training usually benefits from the flexibility and nuance offered by f32
. This doesn’t preclude the use of i32
for certain operations or in posttraining optimization, but during the training phase, f32
is typically preferred to ensure the model can learn effectively.
Hi Tim,
Many many thanks for your long answer. No surprise about efficiency. More confused on the rest… In fact, I got “poor”/“unsatisfactory” (without being “wrong”) results at prediction time (using f32). I train my model with 10.000 records. In fact, my output feature (expected prediction) is also made of discrete/sparse values (between 60000 and 80000). Prediction results are quite far from reality. Expectations in some cases have to be close to min (60000) and max (80000) while all predictions remain around the mean, eg, 70000 ± 5000.
Again, I feel that I have to better control hyperparameters including normailzation, which can be something else that Gauss? Thanks for your help.