CGI training images

Hi, I am really new to machine learning and have been playing around with an example from a book. Some models are trained with CGI datasets, and the Rock, Paper, Scissors Dataset works pretty well when I use it to predict real world images.

I tried to replicate training with CGI images using images I rendered myself and I am struggling to understand why my two similar datasets give wildly different results.

Dataset 1 works really well when using real world images, classifying all but one image correctly. But the other Dataset 2 does not work at all, with only one or two images correctly classified.

I randomly split the dataset into 20% validation and 80% training data. The code for training is here.

Is it pure luck that makes Dataset 1 work so well or is there a fundamental flaw in my understanding?

Thank you for advice.

Link for dataset 2 is this one.. As I am a new user I can not edit my first post because I can only post 2 links.

I suggest you to create more diversity in your dataset e.g.: randomize your background, lighting, position and with other classical augmentations that you can do in 2d or with your 3d engine, shaders etc.

If you have an Nvidia GPU available you can also play with NVISII to populate your dataset:

Ok thank you for the input. I will give more diversity a try.

I don’t have Nvidia so I will stick with Blender.