Pottery database - Improving model

General Discussion
,
#1

I have a database of 9000+ thin-section images of pottery, each in separate folders divided into their 7 periods (iron age, bronze age, etc) and each looking like something like this https://i.imgur.com/0kNzItI.jpeg.

Archaeologists can look at these thin sections and make out which period they were likely in based on their composition. But I was wondering if a deep learning model could save them time and try to classify new pottery into their period.

This is my code but it’s getting terrible results, like into the 30% validation which is almost guessing at random. Any ideas to improve my model or is it an impossible task. My images do have different image styles depending on the person taking the thin-section image which might affect the model.

import matplotlib.pyplot as plt
import numpy as np
import PIL
import tensorflow as tf
from sklearn.metrics import confusion_matrix
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential

batch_size = 32
img_height = 1024
img_width = 1024
data_dir = r'C:\MajorProject\data'

train_ds = tf.keras.utils.image_dataset_from_directory(
  data_dir,
  validation_split=0.2,
  subset="training",
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)

val_ds = tf.keras.utils.image_dataset_from_directory(
  data_dir,
  validation_split=0.2,
  subset="validation",
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)

num_classes = len(class_names)

model = Sequential([
  layers.Rescaling(1./255, input_shape=(img_height, img_width, 3)),
  layers.Conv2D(16, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(32, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Flatten(),
  layers.Dense(128, activation='relu'),
  layers.Dense(num_classes)
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

epochs=10
history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)
#3

Hi @Frankdent, Could please share the training accuracy you have achieved while training the model. Thank You.

#4

Hi! My training accuracy was 99.07% and my validation was 40.04% after 10 epochs.

#5

Hi @Frankdent, Based upon your training and validation accuracy, your model is overfitting. To overcome this problem you can use regularization techniques, try to shuffle all of your data and split them to the train, test and validation set then train again. Thank You.

#7

Hi! I am new to deep learning. I tried data augmentation with pillow by adding reflections, 90 degree rotations, brightness and color hues. I have balanced the dataset using the augmentations and all the images in are 7 separate folders with around 2000 images each. It’s the same 99% training accuracy and 40% validation accuracy. This is a small sample of my dataset. https://i.imgur.com/8LgUhbU.png
Do you have any tips on trying to improve my model. My image size is fairly large. I figured since I have high quality images, I should use it to its full capability. But does this mean I should be using larger kernal sizes for my pooling.
This is my current model. What would you recommend me try using for a model. Since my model is a little bit unique, i’m not sure what do use. Also let me know if there’s anything wrong with my code.

batch_size = 32
img_height = 900
img_width = 900
data_dir = r'C:\MajorProject\data\Period'
val_dir = r'C:\MajorProject\data\Validation'

train_ds = tf.keras.utils.image_dataset_from_directory(
  data_dir,
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)

num_classes = len(train_ds.class_names)

val_ds = tf.keras.utils.image_dataset_from_directory(
  val_dir,
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)
train_ds = train_ds.prefetch(buffer_size=tf.data.AUTOTUNE)
val_ds = val_ds.prefetch(buffer_size=tf.data.AUTOTUNE)

model = Sequential([
    layers.Rescaling(1./255, input_shape=(img_height, img_width, 3)),
    layers.Conv2D(16, 3, padding='same', activation='relu'),
    layers.MaxPooling2D(),
    layers.Conv2D(32, 3, padding='same', activation='relu'),
    layers.MaxPooling2D(),
    layers.Conv2D(64, 3, padding='same', activation='relu'),
    layers.MaxPooling2D(),
    layers.Dropout(0.2),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dense(num_classes)
])
optimizer = keras.optimizers.Adam(learning_rate=0.001)
model.compile(optimizer=optimizer,
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
#8

Hi @Frankdent, Could you please add layer weight regularizers to the model, which allow you to apply penalties on layer parameters or layer activity during optimization.

  • Also it is a good practice to use 80% of the images for training and 20% for validation.
  • Instead of taking 2 data directories, one for training and other for validation you can place all the images in a single directory with sub folders of classes while importing using image_dataset_from_directory you can split the data by passing value to the validation_split argument.
    For example:
train_ds = tf.keras.utils.image_dataset_from_directory(
  data_dir,
  validation_split=0.2,
  subset="training",
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)

val_ds = tf.keras.utils.image_dataset_from_directory(
  data_dir,
  validation_split=0.2,
  subset="validation",
  seed=123,
  image_size=(img_height, img_width),
  batch_size=batch_size)

Because by taking 2 directories for training and validation, if the training directory has less images then the validation directory for a specific class, it will affect the model performance. Please let us know if the issue is resolved or not by following the above methods. Thank You.

#10

I am aware of setting up the training this way and I had it this way at the start. However, I if I put all the augmented images in one directory, I will be validating on augmented images and that is considered bad practice (I think). What I did was write python code that moved 35% of the data from each class folder into their respected validation class folder and then augmented the training folder data so that the classes are balanced. This will make the validation set around 20% by the end of it. Now that I think about it, the validation set isn’t balanced and that might be causing issues.

I will look into layer weight regularizers.