Converting MoveNet to CoreMLtf

Felipe_Arrudi · August 30, 2021, 12:51am

Hi,

Not sure if this is the right place but… I’m trying to convert MoveNet (TensorFlow Hub) to CoreML but I’m finding it impossible. Following instructions from CoreMLTools (TensorFlow 2 Conversion) but no success. Any help or guidance?

Thanks in advance.

lgusm · August 30, 2021, 10:52am

I think it might be easier to find help for this specifically on the CoreML forum.

Are you trying to run the model on an iOS device? can’t you run using TFLite interpreter? I don’t think that would need the conversion and maybe the CoreML delegate might help with the performance

Bhack · August 30, 2021, 11:25am

I suppose that we will have quite soon an IOS example

We’ve released a new Android, Raspberry Pi pose estimation sample that lets you try out MoveNet on mobile and IoT devices. (iOS is coming soon)

Felipe_Arrudi · August 30, 2021, 2:15pm

I want to use it with both macOS and IOS.

I’ve read that is possible to use tflite interpreter and find the temporary file in which the .mlmodel is stored. However doesn’t look very pretty isn’t it? Any idea how?

lgusm · August 30, 2021, 4:38pm

Maybe it’s better to wait for the sample for now

Bhack · August 30, 2021, 4:47pm

@lgusm Can we invite Khan in this thread?

Generally it would be nice to have all the Tensorflow developers advocates in the forum.

khanhlvg · August 31, 2021, 2:21am

We’ll publish the iOS MoveNet Swift sample soon. Stay tuned TFLite should work on both Mac and iOS.

If you still want to pursue the CoreML conversion path, unfortunately I can’t help because I’m not familiar with it. I’d suggest you try asking in the CoreML GitHub repo.

Felipe_Arrudi · September 21, 2021, 7:57pm

Thanks for this. I see the IOS version has been published.

However CoreMLDelegate seems not to work with MoveNet (only Metal). Can I ask you why and if I can expect this to get fixed?

Felipe_Arrudi · September 21, 2021, 7:58pm

Published but no CoreMLDelegate support for MoveNet. Why? Will this get fixed?

Bhack · September 21, 2021, 8:49pm

I see the dynamic framework and CoreML in the subspecs:

github.com

tensorflow/examples/blob/master/lite/examples/pose_estimation/ios/Podfile#L12

      
        
            platform :ios, '12.0'
            
            
# ignore all warnings from all pods
            inhibit_all_warnings!
            
            
target 'PoseEstimation' do
              # Comment the next line if you don't want to use dynamic frameworks
              use_frameworks!
            
            
  # Pods for PoseEstimation
              pod 'TensorFlowLiteSwift', '~> 0.0.1-nightly', :subspecs => ['CoreML', 'Metal']
            
            
end

Felipe_Arrudi · September 21, 2021, 9:05pm

Yes but…
https://github.com/tensorflow/examples/blob/master/lite/examples/pose_estimation/ios/PoseEstimation/ML/Models/MoveNet.swift

Line 76: case .npu:
os_log(“WARNING: MoveNet currently doesn’t support NPU yet. Fall back to CPU”, type: .info)