How to optimize graphic coordinates

General Discussion
, , , ,
#1

I am currently working on an object detection application for Android (written in Java) that uses Google ML and TensorFlow Lite. The application essentially focuses on drawing a boundingBox around detected objects within a live streamed preview, provided by my device camera (via CameraX). To construct the base of this application, I used CameraX to build my camera view, and then Google ML to build the framework. The custom TF Lite model I use as a library can be found here.

Now onto the problem, HERE is what my object detection application looks like when it detects an object. As you can see, there is a heavy amount of misalignment with the boundingBox, making it appear “off”. Excuse the fact that it’s labelled as a letter opener, but this problem is persistent with any object, at any angle on the screen. It appears to be off by the same metric, always appearing just slightly above any object. My biggest concern lies with how this issue can be resolved, whether the problem is because of the TFLite model or something unrelated. The boundingBox seems to be missing its detected object by a noticeable amount.

The following DrawGraphic.java class is what’s responsible for drawing the boundingbox, so here it is:

@SuppressLint(“ViewConstructor”)
public class DrawGraphic extends View {

Paint borderPaint, textPaint;
Rect rect;
String text;

public DrawGraphic(Context context, Rect rect, String text) {
    super(context);
    this.rect = rect;
    this.text = text;

    borderPaint = new Paint();
    borderPaint.setColor(Color.WHITE);
    borderPaint.setStrokeWidth(10f);
    borderPaint.setStyle(Paint.Style.STROKE);

    textPaint = new Paint();
    textPaint.setColor(Color.WHITE);
    textPaint.setStrokeWidth(50f);
    textPaint.setTextSize(32f);
    textPaint.setStyle(Paint.Style.FILL);
}

@Override
protected void onDraw(Canvas canvas) {
    super.onDraw(canvas);
    canvas.drawText(text, rect.centerX(), rect.centerY(), textPaint);
    canvas.drawRect(rect.left, rect.bottom, rect.right, rect.top, borderPaint);
}

}

Any additional information needed to supplement this question shall be provided upon request.

#3

To avoid cramming this post, here is a link to my Pastebin that has all my MainActivity.java code in it, including the code needed to call this class.

#5

Hi Richard,

Thanks for the information.
From the gist you are using MLKit Object detection.

Quick question, if you change to use the default model is the box on the right place?

1 Like
#6

Hi Richard,

I saw you posted the same question on StackOverflow as well, where there were some suggestions provided. I also added some new ones. In general, ML Kit team doesn’t help debug a specific app, but let me know if those pointers help.

To keep information better organized, maybe let’s just use one channel to continue the discussion. What about the StackOverflow one, since there are already some discussion there.

Best,

2 Likes
#8

@lgusm , I’m glad you asked this question because even when using the default model, the box is not in the right place. Does this help diagnose more of the problem?

#10

Given that, I’d say that the problem is not with the model but on dealing with the coordinates. As mentioned by Steven, maybe there’s something wrong on the image resolution.

#12

This may be a bit of a stretch, but how would I work towards finding the image resolution?

#13

I guess you can inspect it during debug with the image object properties