Object on table detection

I am trying to train a model that detects objects on a dinner table. The images are from an infrared camera (1280*800 pixels)topdown on the table. See image:

Ideally, we would detect Plates, spoons, forks etc. I’ve tried multiple models on the small dataset below where I only try to detect People and plates. Unfortunately, I keep getting Nan on Loss all the time. Even during the first steps.

INFO:tensorflow:{‘Loss/classification_loss’: nan,
‘Loss/localization_loss’: nan,
‘Loss/regularization_loss’: nan,
‘Loss/total_loss’: nan,
‘learning_rate’: 0.00319994}

Any advice on how to troubleshoot this or how to fix it? I’m also very curious what the best model would be to use in this use case with small objects.

test dataset:
Dataset

Pipeline config file:
Pipeline config

check the loss function if its according to the input dataset. Also check the labels are correct.
Try the learning rate =0.001 may it will solve your problem.