Determining the orientation of eye co ordinates using pose estimation (python)

Hi , I am currently using pose estimation (movenet) to get the x,y co ordinates of the keypoints for the eyes . I am looking for a way to find out the orientation of the eye how it is moving. For simplicity , i am taking the average of both co-ordinates for left and right eye to get mid_x_coordinate and mid_y_cordinate. However i am struggling to find out a way to find how the orientation of eyes are .
For simplicity , i was thinking to use cv2 .drawArrow function to see where the eyes are looking . However I am not sure how to solve it . Since we have only one set of cordinates. Any ideas ? A screen shot of the orientation is shown below .

image

I suggest you to take a look at the problem formulation in section B Data Post-processing of:

https://arxiv.org/abs/2104.12668

Hello Bhack thank you for the tip. I think i would like to just start even simpler. I was thinking to join a line from the ears ( for example left ear) to the nose and extend it both side to get a general gist of the gaze. That is , it would be an arrow points towards the nose . Does that sound legit ?

If I remeber correctly you are still working on 2d coordinates with Movenet so probably It better to work on 3d landmarks also if you want to try a simpler approach:

1 Like

Thanks , that is actually a good point. I wanted to try first with 2d landmarks . I will have a look into this.