Are bounding box/bbox definitions consistent across datasets?

Lance_N · March 22, 2022, 11:48pm

Do all Datasets checked into the Tensorflow Datasets definitions follow a common specification for Bounding Boxes (bbox)?

The Stanford Dogs dataset uses a range of (0.0, 0.0)->(1.0, 1.0). Do all of them follow this pattern? If so, where is this documented?

Thank you,

Lance Norskog

Bhack · March 23, 2022, 12:08am

I don’t know if they are all in this range:

TFDS CoCo it seems the same:

github.com/keras-team/keras-cv

Corners bounding box format

opened 08:40AM - 17 Mar 22 UTC

closed 02:04AM - 08 Apr 22 UTC

leondgarse

I got an issue when trying to port two of my currently using bounding box augmen…t related functions [resize_and_crop_bboxes](https://github.com/leondgarse/keras_cv_attention_models/blob/main/keras_cv_attention_models/coco/data.py#L23) and [bboxes_apply_affine](https://github.com/leondgarse/keras_cv_attention_models/blob/main/keras_cv_attention_models/coco/data.py#L45). I think we better set a default expecting bbox input format. For my usage, the corner bounding box is in same format with `tfds COCO`, that I think is `[top, left, bottom, right]` with value in range `[0, 1]`. ```py import tensorflow_datasets as tfds ds, info = tfds.load('coco/2017', with_info=True) aa = ds['train'].as_numpy_iterator().next() print(aa['image'].shape) # (462, 640, 3) print(aa['objects']) # {'area': array([17821, 16942, 4344]), # 'bbox': array([[0.54380953, 0.13464062, 0.98651516, 0.33742186], # [0.50707793, 0.517875 , 0.8044805 , 0.891125 ], # [0.3264935 , 0.36971876, 0.65203464, 0.4431875 ]], dtype=float32), # 'id': array([152282, 155195, 185150]), # 'is_crowd': array([False, False, False]), # 'label': array([3, 3, 0])} imm = aa['image'] plt.imshow(imm) for bb in aa["objects"]["bbox"]: bb = np.array([bb[0] * imm.shape[0], bb[1] * imm.shape[1], bb[2] * imm.shape[0], bb[3] * imm.shape[1]]) plt.plot(bb[[1, 1, 3, 3, 1]], bb[[0, 2, 2, 0, 0]]) ``` ![Figure_1](https://user-images.githubusercontent.com/5744524/158770180-07732b02-261b-41d8-a674-81cc909a17da.png) It will save some effort checking format if we can set a standard. So my questions are: - Currently in [bounding_box.py](https://github.com/keras-team/keras-cv/tree/master/keras_cv/utils/bounding_box.py), we are assuming it's `LEFT, TOP, RIGHT, BOTTOM`. Are we gonna set this as default? - Is value range expecting in `[0, 1]` or scaled with image actual shape?

But it seems preferred not mandatory:

github.com

tensorflow/datasets/blob/master/docs/common_gotchas.md#prefer-more-specific-type-instead-of-tfdsfeaturestensor

# Common implementation gotchas

This page describe the common implementation gotcha when implementing a new
dataset.

## Legacy `SplitGenerator` should be avoided

The old `tfds.core.SplitGenerator` API is deprecated.

```python
def _split_generator(...):
  return [
      tfds.core.SplitGenerator(name='train', gen_kwargs={'path': train_path}),
      tfds.core.SplitGenerator(name='test', gen_kwargs={'path': test_path}),
  ]
```

Should be replaced by:

```python

This file has been truncated. show original

lgusm · March 23, 2022, 5:05pm

I don’t think that’s the case for all datasets.

I couldn’t at least find this anywhere

Lance_N · March 23, 2022, 6:17pm

Yeah, I will assume it is a free-for-all. Thanks!

Bhack · March 23, 2022, 6:24pm

I hope that many will respect this recommendation:

Prefer more specific type instead of tfds.features.Tensor

When possible, prefer the more specific types tfds.features.ClassLabel, tfds.features.BBoxFeatures,… instead of the generic tfds.features.Tensor.
Rationale: In addition of being more semantically correct, specific features provides additional metadata to users and are detected by tools.

Are bounding box/bbox definitions consistent across datasets?

Prefer more specific type instead of tfds.features.Tensor

Prefer more specific type instead of `tfds.features.Tensor`