Hello, my model may have encountered overfitting issue as during the training process, the final accuracy on both the validation and training sets are 1. Moreover, the accuracy on the test set is also 1. However, I am not quite sure whether overfitting has occurred. Here is my log:
Epoch 1/10
2023-05-15 09:30:41.426293: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 3s 97ms/step - loss: 0.3733 - accuracy: 0.9377 - val_loss: 0.6043 - val_accuracy: 0.9208
Epoch 2/10
2023-05-15 09:30:42.535313: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 53ms/step - loss: 0.2549 - accuracy: 0.9836 - val_loss: 0.4312 - val_accuracy: 0.9875
Epoch 3/10
2023-05-15 09:30:43.200824: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.1609 - accuracy: 0.9967 - val_loss: 0.3159 - val_accuracy: 1.0000
Epoch 4/10
2023-05-15 09:30:43.872236: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.1158 - accuracy: 1.0000 - val_loss: 0.2585 - val_accuracy: 1.0000
Epoch 5/10
2023-05-15 09:30:44.548382: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.0969 - accuracy: 1.0000 - val_loss: 0.2022 - val_accuracy: 1.0000
Epoch 6/10
2023-05-15 09:30:45.220167: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.0788 - accuracy: 1.0000 - val_loss: 0.1666 - val_accuracy: 1.0000
Epoch 7/10
2023-05-15 09:30:45.886257: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 55ms/step - loss: 0.0648 - accuracy: 1.0000 - val_loss: 0.1404 - val_accuracy: 1.0000
Epoch 8/10
2023-05-15 09:30:46.562473: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 56ms/step - loss: 0.0542 - accuracy: 1.0000 - val_loss: 0.1242 - val_accuracy: 1.0000
Epoch 9/10
2023-05-15 09:30:47.245306: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 57ms/step - loss: 0.0472 - accuracy: 1.0000 - val_loss: 0.1059 - val_accuracy: 1.0000
Epoch 10/10
2023-05-15 09:30:47.934772: W tensorflow/core/lib/png/png_io.cc:88] PNG warning: iCCP: known incorrect sRGB profile
11/11 [==============================] - 1s 54ms/step - loss: 0.0407 - accuracy: 1.0000 - val_loss: 0.0936 - val_accuracy: 1.0000
This is my first question
my second question is:
when I use the statement tf.keras.models.save_model(model, './my_model.h5') to save the model, I get a warning.
CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
warnings.warn('Custom mask layers require a config and must override ’
Will this affect the accuracy of my saved model? How can I solve this problem? Here is my code:
import tensorflow as tf
import os
import matplotlib.pyplot as plt
base_dir='./daten/dash'
train_dir=os.path.join(base_dir,'train')
val_dir=os.path.join(base_dir,'validation')
BATCH_SIZE=30
IMG_SIZE=(160,160)
train_dataset = tf.keras.utils.image_dataset_from_directory(train_dir,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMG_SIZE)
validation_dataset = tf.keras.utils.image_dataset_from_directory(val_dir,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMG_SIZE)
val_batches=tf.data.experimental.cardinality(validation_dataset)
test_dataset=validation_dataset.take(val_batches//5)
validation_dataset=validation_dataset.skip(val_batches//5)
AUTOTUNE = tf.data.AUTOTUNE
train_dataset = train_dataset.prefetch(buffer_size=AUTOTUNE)
validation_dataset = validation_dataset.prefetch(buffer_size=AUTOTUNE)
test_dataset = test_dataset.prefetch(buffer_size=AUTOTUNE)
data_augmentation=tf.keras.Sequential([
tf.keras.layers.RandomFlip('horizontal'),
tf.keras.layers.RandomRotation(0.2),
tf.keras.layers.RandomCrop(160,160),
tf.keras.layers.RandomZoom(.5, .2)
])
preprocess_input=tf.keras.applications.mobilenet_v2.preprocess_input
IMG_SHAPE= IMG_SIZE+(3,)
base_model=tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
include_top=False,
weights='imagenet')
image_batch,label_batch=next(iter(train_dataset))
feature_batch=base_model(image_batch)
base_model.trainable=False
base_model.summary()
global_average_layer = tf.keras.layers.GlobalAveragePooling2D()
feature_batch_average=global_average_layer(feature_batch)
prediction_layer=tf.keras.layers.Dense(1)
prediction_batch=prediction_layer(feature_batch_average)
inputs=tf.keras.Input(shape=(160,160,3))
x=data_augmentation(inputs)
x=preprocess_input(x)
x=base_model(x,training=False)
x=global_average_layer(x)
x=tf.keras.layers.Dropout(0.2)(x)
outputs=prediction_layer(x)
model=tf.keras.Model(inputs,outputs)
base_learning_rate = 0.0001
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),
loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
model.summary()
initial_epochs = 10
loss0, accuracy0 = model.evaluate(validation_dataset)
history = model.fit(train_dataset,
epochs=initial_epochs,
validation_data=validation_dataset)
loss, accuracy = model.evaluate(test_dataset)
print('Test accuracy :', accuracy)
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']
tf.keras.models.save_model(model, './my_model.h5')
plt.figure(figsize=(8, 8))
plt.subplot(2, 1, 1)
plt.plot(acc, label='Training Accuracy')
plt.plot(val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.ylabel('Accuracy')
plt.ylim([min(plt.ylim()),1])
plt.title('Training and Validation Accuracy')
plt.subplot(2, 1, 2)
plt.plot(loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.ylabel('Cross Entropy')
plt.ylim([0,1.0])
plt.title('Training and Validation Loss')
plt.xlabel('epoch')
plt.show()