Custom keypoints tracking

What is the best way to train TensorFlow for custom keypoint tracking that can work on the web?
Right now I’m using CenterNet MobileNetV2 FPN Keypoints 512x512 to train, but the outcome is not good enough keypoints confidence is significantly less approx 30%, but the bounding box is fine. So is there any way I can improve the model confidence for keypoints?

Config :
steps 25000
epoch 12
learning rate 0.01
train dataset 1280
test dataset 319

There were no tangible guide to train a key point detection model on custom dataset other than human pose or facial key points.