Can someone guide, how to include the Custom DataGenerator functionality from tf.keras with methodskeras.utils.Sequence , on_epoch_end, __len__, __getitem__ ,
in tensorflow alone?
Bhack
January 17, 2022, 6:14pm
#3
I don’t understand your issue, do you have a minimized standalone example or Colab?
1 Like
Yes, here is the link. Uses custom generator based onkeras.utils.Sequence
keras_mnist_sequence.py
"""
Title: Simple MNIST convnet
Author: [fchollet](https://twitter.com/fchollet)
Date created: 2015/06/19
Last modified: 2020/04/21
Description: A simple convnet that achieves ~99% test accuracy on MNIST.
Modified:
- data wrapped as a Sequenceshow original
keras_sequence_fork.log
new process 28285
x_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
main process 28285
TRAIN MnistSequence.__init__(), PID: 28285 len: 58
VAL MnistSequence.__init__(), PID: 28285 len: 9
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param # show original
keras_sequence_spawn.log
new process 27656
x_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
main process 27656
TRAIN MnistSequence.__init__(), PID: 27656 len: 58
VAL MnistSequence.__init__(), PID: 27656 len: 9
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param # show original
There are more than three files. show original
Is there way to implement in native TensorFlow alone (no Keras)?
Bhack
January 18, 2022, 5:08pm
#6
it is just really like a plain python class. Please check yourself the source code at:
class Sequence:
"""Base object for fitting to a sequence of data, such as a dataset.
Every `Sequence` must implement the `__getitem__` and the `__len__` methods.
If you want to modify your dataset between epochs you may implement
`on_epoch_end`.
The method `__getitem__` should return a complete batch.
Notes:
`Sequence` are a safer way to do multiprocessing. This structure guarantees
that the network will only train once
on each sample per epoch which is not the case with generators.
Examples:
```python
from skimage.io import imread
from skimage.transform import resize
import numpy as np
This file has been truncated. show original
1 Like