Training a new tflite image segmentation model?

General Discussion
#1 
Env: Tensorflow:1.15.3
     Python: 3.6
Android App: https://github.com/tensorflow/examples

Using the pre-trained model

$ wget http://download.tensorflow.org/models/deeplabv3_mnv2_pascal_train_aug_2018_01_29.tar.gz"

adding metadata can run on the android platform, but using the following command to train a new model can not be inferred to the object normally, it is always inferred to the background, are there any need to modify ?

$ cd ~
$ mkdir deeplab;cd deeplab
$ git clone --depth 1 https://github.com/tensorflow/models.git
$ cd models/research/deeplab/datasets
$ mkdir pascal_voc_seg

$ curl -sc /tmp/cookie \
  "https://drive.google.com/uc?export=download&id=1rATNHizJdVHnaJtt-hW9MOgjxoaajzdh" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie \
  "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1rATNHizJdVHnaJtt-hW9MOgjxoaajzdh" \
  -o pascal_voc_seg/VOCtrainval_11-May-2012.tar

$ sed -i -e "s/python .\/remove_gt_colormap.py/python3 .\/remove_gt_colormap.py/g" \
      -i -e "s/python .\/build_voc2012_data.py/python3 .\/build_voc2012_data.py/g" \
      download_and_convert_voc2012.sh

$ sh download_and_convert_voc2012.sh

$ cd ../..
$ mkdir -p deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/train
$ mkdir -p deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/eval
$ mkdir -p deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/vis

$ export PATH_TO_TRAIN_DIR=${HOME}/deeplab/models/research/deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/train
$ export PATH_TO_DATASET=${HOME}/deeplab/models/research/deeplab/datasets/pascal_voc_seg/tfrecord
$ export PYTHONPATH=${HOME}/deeplab/models/research:${HOME}/deeplab/models/research/deeplab:${HOME}/deeplab/models/research/slim:${PYTHONPATH}

$ python3 deeplab/train.py \
    --logtostderr \
    --training_number_of_steps=500000 \
    --train_split="train" \
    --model_variant="mobilenet_v3_small_seg" \
    --decoder_output_stride=16 \
    --train_crop_size="513,513" \
    --train_batch_size=8 \
    --dataset="pascal_voc_seg" \
    --save_interval_secs=300 \
    --save_summaries_secs=300 \
    --save_summaries_images=True \
    --log_steps=100 \
    --train_logdir=${PATH_TO_TRAIN_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

$python deeplab/export_model.py --model_variant="mobilenet_v3_small_seg" --crop_size=257 --crop_size=257 --checkpoint_path=${PATH_TO_TRAIN_DIR}/model.ckpt-2508 --export_path=./deeplabv3_mnv2_pascal_trainval/frozen_inference_graph_257.pb

Thanks!

#3

@YYange Welcome to Tensorflow Forum!

Below are the general steps involved in training a new TensorFlow Lite image segmentation model. Do you want to try these and let us know if any of the below works for you:-

Collect a representative dataset of images covering your segmentation task.
Annotate each image with pixel-level masks indicating segmentation boundaries.
Divide the dataset into training, validation, and test sets (e.g., 80%, 10%, 10%).

Choose Architecture:- Select a suitable segmentation model architecture, such as:

  • U-Net
  • DeepLabv3+
  • Mask R-CNN
  • EfficientNet-Lite (for mobile deployment)
  • Consider Factors:
    • Accuracy requirements
    • Computational constraints
    • Latency requirements

Define Model:- Use TensorFlow or Keras to define the model architecture.
Load Data:- Load the prepared dataset using appropriate TensorFlow input pipelines.
Compile Model:- Choose an optimizer (e.g., Adam), loss function (e.g., categorical cross-entropy for pixel-wise classification), and metrics (e.g., IoU, Dice coefficient).
Train Model:- Train the model on the training set, monitoring performance on the validation set.

Convert:- Use the tf.lite.TFLiteConverter to convert the trained model to TFLite format.
Quantization:- Consider quantization for reduced model size and faster inference (optional).

Integrate:- Integrate the TFLite model into your mobile or embedded application. Load the model and perform inference on new images to generate segmentation masks.

Let us know if this helps!