Open Source MoveNet?

Research & Models
,
#1

Hi MoveNet developers,

I’ve been following MoveNet for a while I love how you guys are creating really useful and performant model to be used in mobile applications. I have a use case where I would like to fine-tune/modify the model. (e.g. modify number of key point, provide more targeted training from custom dataset)

Is there any plan to open source/enable fine tuning on MoveNet models? I’m relatively new to ML field (coming from SW) and looking at TFHub suggests that it’s not fine-tunable. (I did try to import/inspect the model on TF and yeah it looked pretty blackbox) I searched around open source implementation and there’s one written in PyTorch,

I’ve no idea how accurate this implementation is, but I guess I have few options.

  1. Use PyTorch implementation as baseline
  2. Port this implementation/build in TF
  3. Ask original devs for generosity

Let me know what you guys have planned, and any suggestions!

Thanks,
Daniel

1 Like
#3

Hello Daniel,

Thanks a lot for your interest in using MoveNet! You are right the MoveNet releases are currently not fine-tunable. But the model implementation is actually based on the Tensorflow Object Detection API so you should be able to access the model code and modify it for your own purpose. As mentioned in the documentation, the MoveNet model is based on CenterNet with some modifications in the postprocessing logics and ops re-write to boost the performance. The closest model architecture can be found here. You can also find some example training configs in the /models/research/object_detection/configs/tf2/ folder.

I’d recommend you to start with the above codebase and modify from it. There are a few internal modifications which have not been released yet. We are still working on it and will let you know once it is there. Thanks.

2 Likes
BlazePose GHUM 3D: Transfer Learning
#4

That was super helpful, thanks!