An example of Compose?

General Discussion
#1

@Bhack could you provide a minimal example of tfa.image.compose_transforms()?

2 Likes
#2

Here you have rotation + translation

2 Likes
#3

Thanks but in practice what are we supposed to be providing to compose_transforms()? Something like the following?

compose_transforms([tf.image.rot90, tf.image.random_flip_left_right])

If so, how it’s supposed to be applied to an image or a batch of images? From the documentation and also the test-case shared above, these questions are not clear.

2 Likes
#4

Internally it is just a matmul loop over single projective tranfroms matrix in the list (or batched):

1 Like
#5

Sadly, I am still not clear as to how to use it with a dataset or even a single image having a leading batch dimension.

1 Like
#6

That method is just going to compose projective transformations.

The real transformation is in transform that is still quite duplicated in addons cause the transform in TF is still a private API (but the transform Matrix format is the same).

See the thread starting at

https://tensorflow-prod.ospodiscourse.com/t/cannot-run-model-fit-locally-help/1648/9

1 Like
#7

Here’s a shot example applying the composition on a standard image:

1 Like
#8

This is the same with internals

from tensorflow.python.keras.layers.preprocessing.image_preprocessing import transform
....
image_transformed = transform(tf.expand_dims(image, axis=0),composed)
1 Like
#9

E.g. the internal one was used in transformations that support models in model gardens SIG.

https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/augment.py

These transformations are available for users in

Most of the transformations available in Addons I think that are duplicated now.

1 Like
#10

Cool. But I think we both could agree that the documentation does not clearly reflect on these points. I guess we cannot expect the majority of users to dig into the source code and figure their way out into using these.

2 Likes
#11

Yes probably some documentation is missing but I think that we can build a clear path and better documentation or tutorials when we will unify a little bit the image processing API that currently it is quite duplicated or sparse in the ecosystem (also generally we cannot promote private API to the end users like from tensorflow.python).

The modern history of this in Addons/Ecosystem it is already quite old:

1 Like