How to multiple image augmentations which don't have layers?

General Discussion
,
#1

I am using

train_data_gen = tf.keras.utils.image_dataset_from_directory(...)

I want to apply multiple image transforms like it is done in Pytorch:

# Define the transformation pipeline
transform = torchvision.transforms.Compose([
    torchvision.transforms.ToTensor(),  # Convert the image to a PyTorch tensor
    Resize((256, 256)),  # Resize the images to a target size while keeping the aspect ratio intact
    torchvision.transforms.Pad(padding=0, fill=0.5, padding_mode='constant'),  # Add padding with the average color of the ImageNet dataset
    torchvision.transforms.RandomRotation(degrees=(-15, 15)),  # Randomly rotate the image between -15 and 15 degrees
    torchvision.transforms.RandomCrop(224),  # Randomly crop and resize the image to the target input size
    torchvision.transforms.RandomHorizontalFlip(),  # Flip the image horizontally with a probability of 50%
    torchvision.transforms.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1, hue=0.1),  # Randomly change the brightness, contrast, saturation, and hue
    torchvision.transforms.RandomGrayscale(p=0.2),  # Convert the image to grayscale with a probability of 20%
    torchvision.transforms.RandomErasing(p=0.2, scale=(0.02, 0.33), ratio=(0.3, 3.3), value=0.5),  # Randomly erase a rectangular region of the image
    Resize((256, 256)),  # Resize the images to a target size while keeping the aspect ratio intact
    torchvision.transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))  # Normalize the image according to the ImageNet dataset
])

So how can I do that? can you show me how its done preferably with a code example.

#3

There are two ways around this. You can either use preprocessing layers as part of your model definition. However there any only a handful of preprocessing layers. The go-to approach is to define all your augmentations (with tf.image) in a function and then apply them on your dataset using the map method. Here is a tutorial from the tensorflow docs. Hope it helps.

#4

Thank you for your response. I read that in the documentation but I couldn’t grasp how it works.

#5

if the label_mode parameter of tf.keras.utils.image_dataset_from_directory(...) is not None, it will return a tensorflow dataset object which is a tuple (image,label). Supposing you wanted to resize and randomly flip you can create a function;

def augmentation(image,label):
    image = tf.image.resize(image,[<resize_shape>])
    image = tf.image.random_flip_left_right(image)
    return image,label

Then apply the augmentation like so train_data = train_data_gen.map(augmentation). The above example assumes the dataset has not been batched otherwise you will have to rewrite the function to work with batched data