How does this basic model reduce the loss so much for random data?

Even though I’m running this on NodeJS, the question is just general, the code is this one from TS flowJS example where:

  • 100 column vectors of 10 units are fitted to
  • 100 single unit outputs.

However, both the column vectors and the units are from a random sample.

After running it locally only 1000 iterations, the loss almost vanishes.

Is this something to be weary about in here? How does it get so low? (I expected it to oscillate around 0.5, or some other value.)

I can’t tell what code in that link you are referring to, but it sounds like this is highlighting and example of overfitting.

With only 100 observations, it doesn’t take a very big model to be able to fit to any output. Normally, you would have a separate hold-out dataset to test the model against. If you are fitting random inputs to a random output, your model will fit your training data very well, and will not fit at all when tested against the hold-out data.

Thank you. Is there a test that uses random data to know at what point the model would start overfitting you think ?