"Unsupported color conversion request" error

Research & Models
, , ,
#1

Hello,
I’m currently learning Tensorflow and I’m encountering the following issue when I try to train an image classification model using the PlantCLEF2017 dataset (if relevant).

Unsupported color conversion request
2023-05-17 21:38:04.286414: I tensorflow/core/common_runtime/executor.cc:1197] [/job:localhost/replica:0/task:0/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: jpeg::Uncompress failed. Invalid JPEG data or crop window.
	 [[{{node decode_image/DecodeImage}}]]
2023-05-17 21:38:04.286769: I tensorflow/core/common_runtime/executor.cc:1197] [/job:localhost/replica:0/task:0/device:CPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: jpeg::Uncompress failed. Invalid JPEG data or crop window.
	 [[{{node decode_image/DecodeImage}}]]
	 [[IteratorGetNext]]
2023-05-17 21:38:04.286787: I tensorflow/core/common_runtime/executor.cc:1197] [/job:localhost/replica:0/task:0/device:GPU:0] (DEBUG INFO) Executor start aborting (this does not indicate an error and you can ignore this message): INVALID_ARGUMENT: jpeg::Uncompress failed. Invalid JPEG data or crop window.
	 [[{{node decode_image/DecodeImage}}]]
	 [[IteratorGetNext]]
	 [[IteratorGetNext/_4]]

Regarding the “Unsupported color conversion request” error I could not find anything related to TF.

For “Invalid JPEG data or crop window” I did find some threads and followed the advice there, however that did not fix the issue.

What I’ve tried:

  • Check if .jpg files are corrupted using imagemagick’s identify; No errors reported
  • Check if .jpg files are corrupted using PIL python library and a script that will run .open() and then .verify() on each image; No errors reported
  • Check if .jpg files are corrupted using the method described in Image classification from scratch
  • Tried to use a subset of that dataset
  • Decreased batch size
  • Verified that the color_mode value is set to grayscale

I’m using Ubuntu 22.04 and Jupyter.

I have previously trained image models and I encountered no issues. So this must be related to the dataset somehow. I’m also thinking that the “Unsupported color conversion request” must come from some system library rather than TF itself.

What would you recommend I do in order to identify the actual issue here?

Thank you.

#2

Hi @virgil ,

Yes, it is not related to the TF from the issue description.

I would appreciate it if you could try the following options and report back on whether or not the
problem persists:

Ensure that you are using the latest version of TensorFlow and its dependencies

  1. Convert the images to a different format, such as PNG or TIFF, and try training your model again.
  2. To confirm if the issue is specific to the PlantCLEF2017 dataset, try training your image classification model on a different dataset with the current TF code in the same environment. If the error does not occur with a different dataset, it may suggest that there is an issue with the PlantCLEF2017 dataset itself.
  3. Reach out to the PlantCLEF2017 dataset community for any specific recommendations related to this dataset.

If all the above attempts do not succeed, it is better to go with the other dataset instead of wasting time here.

I hope this helps you.

Thanks.

#3

Hey Laxma and thank you for answering. Just as your reply poped up, I managed to make this work.

What I initially tried was to manually convert all the images to grayscale using imagemagick:

find . -type d -exec sh -c 'mogrify -colorspace gray "$0"/*.jpg' {} \;

This resulted in a couple of images getting corrupted that were correctly identified by previously used methods. After getting rid of those corrupted files, the training process went on successfully.

I tried removing the same files that got corrupted in the non-grayscale mode dataset and the same issue occured.

I’m currently trying out your first suggestion and convert all those images to .png, but it’s a very slow process. I’ll leave it running, but at this point I’m pretty sure this will do the trick.

The only thing I found related to this error message was this old discussion regarding a webp converter: https://groups.google.com/a/webmproject.org/g/webp-discuss/c/MH8q_d6M1vM.
While I checked and could not find any CMYK images in the dataset, I do think that it’s somehow related since manually forcing all the images to grayscale worked.

One question remains though. Shouldn’t color_mode = "grayscale" being set on image_dataset_from_directory() convert the images to grayscale? Even if the original images were CMYK (or other weird mode), shouldn’t they end up as grayscale when used for training?

Thanks :slight_smile:

Edit: Forgot to mention, I’m already using the latest Tensorflow version (2.12.0)

Edit 2: Indeed, converting the files to .png did work.

find . -type d -exec sh -c 'echo "$0" && mogrify -format png "$0"/*.jpg' {} \;

# the original .jpg files remained so I needed to remove them
find . -type d -exec sh -c 'echo "$0" && rm "$0"/*.jpg' {} \;