# Prerequisites
Please answer the following questions for yourself before sub…mitting an issue.
- [x ] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
- [x ] I am reporting the issue to the correct repository. (Model Garden official or research directory)
- [x ] I checked to make sure that this issue has not already been filed.
## 1. The entire URL of the file you are using
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/using_your_own_dataset.md
## 2. Describe the bug
I am trying to write the equivalent of this [code][1] which converts CSV to TF records but instead, I am trying to convert from JSON to TFrecords. I am trying to generate TFrecords for using it in [object detection API][2].
Here is my full error message
Traceback (most recent call last):
File "model_main_tf2.py", line 113, in <module>
tf.compat.v1.app.run()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\absl\app.py", line 299, in run
_run_main(main, args)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\absl\app.py", line 250, in _run_main
sys.exit(main(argv))
File "model_main_tf2.py", line 109, in main
record_summaries=FLAGS.record_summaries)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\model_lib_v2.py", line 561, in train_loop
unpad_groundtruth_tensors)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\model_lib_v2.py", line 342, in load_fine_tune_checkpoint
features, labels = iter(input_dataset).next()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 645, in next
return self.__next__()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 649, in __next__
return self.get_next()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 694, in get_next
self._iterators[i].get_next_as_list_static_shapes(new_name))
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 1474, in get_next_as_list_static_shapes
return self._iterator.get_next()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\data\ops\multi_device_iterator_ops.py", line 581, in get_next
result.append(self._device_iterators[i].get_next())
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 825, in get_next
return self._next_internal()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 764, in _next_internal
return structure.from_compatible_tensor_list(self._element_spec, ret)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\eager\context.py", line 2105, in execution_mode
executor_new.wait()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\eager\executor.py", line 67, in wait
pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[12] = 12 is not in [0, 0)
[[{{node GatherV2_4}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]]
And here is my code, which is an attempt to convert JSON files into TFrecords
Sample JSON file
{
"0.jpg59329": {
"filename": "0.jpg",
"size": 59329,
"regions": [{
"shape_attributes": {
"name": "rect",
"x": 412,
"y": 130,
"width": 95,
"height": 104
},
"region_attributes": {}
}, {
"shape_attributes": {
"name": "rect",
"x": 521,
"y": 82,
"width": 126,
"height": 106
},
"region_attributes": {}
}
}
My Python Code
# Ref 1: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/using_your_own_dataset.md
# Ref 2: https://github.com/datitran/raccoon_dataset/blob/master/generate_tfrecord.py
import json
import glob
from object_detection.utils import dataset_util
import tensorflow as tf
from pathlib import Path
flags = tf.compat.v1.app.flags
flags.DEFINE_string('output_path', '', 'Path to output TFRecord')
FLAGS = flags.FLAGS
def json_to_tf(jsonFile, im):
with open(im, "rb") as image:
encoded_image_data = image.read()
with open(jsonFile) as json_file:
data = json.load(json_file)
for key, value in data.items():
width = 1920
height = 1080
filename = value["filename"]
filename = filename.encode('utf8')
image_format = b'jpeg'
xmins = []
xmaxs = []
ymins = []
ymaxs = []
classes_text = []
classes = []
for x in value["regions"]:
xmins.append(x["shape_attributes"]['x'])
xmaxs.append(x["shape_attributes"]['width'] + x["shape_attributes"]['x'])
ymins.append(x["shape_attributes"]['y'])
ymaxs.append(x["shape_attributes"]['height'] + x["shape_attributes"]['y'])
classes_text.append("cars".encode('utf8'))
classes.append(1)
tf_example = tf.train.Example(features=tf.train.Features(feature={
'image/height': dataset_util.int64_feature(height),
'image/width': dataset_util.int64_feature(width),
'image/filename': dataset_util.bytes_feature(filename),
'image/source_id': dataset_util.bytes_feature(filename),
'image/encoded': dataset_util.bytes_feature(encoded_image_data),
'image/format': dataset_util.bytes_feature(image_format),
'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
'image/object/class/label': dataset_util.int64_list_feature(classes),
}))
return tf_example
writer = tf.compat.v1.python_io.TFRecordWriter("train.record")
for fn in glob.glob("annotation_refined\\*.json"):
for img in glob.glob("images\\*.jpg"):
if Path(fn).stem == Path(img).stem:
tf_example_1 = json_to_tf(fn, img)
writer.write(tf_example_1.SerializeToString())
writer.close()
Can someone give any tips on what is going wrong?
[1]: https://github.com/abdelrahman-gaber/tf2-object-detection-api-tutorial/blob/master/data_gen/generate_tfrecord.py
[2]: https://github.com/tensorflow/models/tree/master/research/object_detection
## 3. Steps to reproduce
I execute the code above to convert JSON to TF records
## 4. Expected behavior
A TF record that works which I can use when executing the following
```
out_dir=output/
mkdir -p $out_dir
python model_main_tf2.py --alsologtostderr --model_dir=$out_dir --checkpoint_every_n=1 \
--pipeline_config_path=mask_rcnn_inception_resnet_v2.config \
--eval_on_train_data 2>&1 | tee $out_dir/train.log
```
Here is my config file
```
# Mask R-CNN with Inception Resnet v2 (no atrous)
# Sync-trained on COCO (with 8 GPUs) with batch size 16 (1024x1024 resolution)
# Initialized from Imagenet classification checkpoint
#
# Train on GPU-8
#
# Achieves 40.4 box mAP and 35.5 mask mAP on COCO17 val
model {
faster_rcnn {
number_of_stages: 3
num_classes: 1
image_resizer {
fixed_shape_resizer {
height: 1024
width: 1024
}
}
feature_extractor {
type: 'faster_rcnn_inception_resnet_v2_keras'
}
first_stage_anchor_generator {
grid_anchor_generator {
scales: [0.25, 0.5, 1.0, 2.0]
aspect_ratios: [0.5, 1.0, 2.0]
height_stride: 16
width_stride: 16
}
}
first_stage_box_predictor_conv_hyperparams {
op: CONV
regularizer {
l2_regularizer {
weight: 0.0
}
}
initializer {
truncated_normal_initializer {
stddev: 0.01
}
}
}
first_stage_nms_score_threshold: 0.0
first_stage_nms_iou_threshold: 0.7
first_stage_max_proposals: 300
first_stage_localization_loss_weight: 2.0
first_stage_objectness_loss_weight: 1.0
initial_crop_size: 17
maxpool_kernel_size: 1
maxpool_stride: 1
second_stage_box_predictor {
mask_rcnn_box_predictor {
use_dropout: false
dropout_keep_probability: 1.0
fc_hyperparams {
op: FC
regularizer {
l2_regularizer {
weight: 0.0
}
}
initializer {
variance_scaling_initializer {
factor: 1.0
uniform: true
mode: FAN_AVG
}
}
}
mask_height: 33
mask_width: 33
mask_prediction_conv_depth: 0
mask_prediction_num_conv_layers: 4
conv_hyperparams {
op: CONV
regularizer {
l2_regularizer {
weight: 0.0
}
}
initializer {
truncated_normal_initializer {
stddev: 0.01
}
}
}
predict_instance_masks: true
}
}
second_stage_post_processing {
batch_non_max_suppression {
score_threshold: 0.0
iou_threshold: 0.6
max_detections_per_class: 100
max_total_detections: 100
}
score_converter: SOFTMAX
}
second_stage_localization_loss_weight: 2.0
second_stage_classification_loss_weight: 1.0
second_stage_mask_prediction_loss_weight: 4.0
resize_masks: false
}
}
train_config: {
batch_size: 16
num_steps: 2
optimizer {
momentum_optimizer: {
learning_rate: {
cosine_decay_learning_rate {
learning_rate_base: 0.008
total_steps: 2
warmup_learning_rate: 0.0
warmup_steps: 1
}
}
momentum_optimizer_value: 0.9
}
use_moving_average: false
}
gradient_clipping_by_norm: 10.0
fine_tune_checkpoint_version: V2
fine_tune_checkpoint: "mask_rcnn_inception_resnet_v2_1024x1024_coco17_gpu-8/checkpoint/ckpt-0"
fine_tune_checkpoint_type: "detection"
data_augmentation_options {
random_horizontal_flip {
}
}
}
train_input_reader: {
label_map_path: "capillary_labelmap.pbtxt"
tf_record_input_reader {
input_path: "train.record"
}
load_instance_masks: true
mask_type: PNG_MASKS
}
eval_config: {
metrics_set: "coco_detection_metrics"
metrics_set: "coco_mask_metrics"
eval_instance_masks: true
use_moving_averages: false
batch_size: 512
include_metrics_per_category: true
}
eval_input_reader: {
label_map_path: "capillary_labelmap.pbtxt"
shuffle: false
num_epochs: 1
tf_record_input_reader {
input_path: "train.record"
}
load_instance_masks: true
mask_type: PNG_MASKS
}
```
## 5. Additional context
```
2020-08-07 10:38:35.337851: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
2020-08-07 10:38:38.463844: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll
2020-08-07 10:38:38.513432: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:09:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.74GHz coreCount: 68 deviceMemorySize: 11.00GiB deviceMemoryBandwidth: 573.69GiB/s
2020-08-07 10:38:38.513852: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
2020-08-07 10:38:38.518276: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll
2020-08-07 10:38:38.522366: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2020-08-07 10:38:38.524155: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2020-08-07 10:38:38.529030: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2020-08-07 10:38:38.531871: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll
2020-08-07 10:38:38.540810: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll
2020-08-07 10:38:38.542364: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-08-07 10:38:38.543192: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-08-07 10:38:38.554233: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x21c5e9cf1e0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-07 10:38:38.554563: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2020-08-07 10:38:38.555417: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:09:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.74GHz coreCount: 68 deviceMemorySize: 11.00GiB deviceMemoryBandwidth: 573.69GiB/s
2020-08-07 10:38:38.555804: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
2020-08-07 10:38:38.556098: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll
2020-08-07 10:38:38.556281: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2020-08-07 10:38:38.556479: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2020-08-07 10:38:38.556678: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2020-08-07 10:38:38.556864: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll
2020-08-07 10:38:38.557049: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll
2020-08-07 10:38:38.558452: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-08-07 10:38:39.469766: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-08-07 10:38:39.470008: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] 0
2020-08-07 10:38:39.470183: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0: N
2020-08-07 10:38:39.471856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8583 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:09:00.0, compute capability: 7.5)
2020-08-07 10:38:39.474601: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x21c05d6bb10 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-08-07 10:38:39.474889: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
I0807 10:38:39.477294 26156 mirrored_strategy.py:341] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
INFO:tensorflow:Maybe overwriting train_steps: None
I0807 10:38:39.482280 26156 config_util.py:552] Maybe overwriting train_steps: None
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0807 10:38:39.482280 26156 config_util.py:552] Maybe overwriting use_bfloat16: False
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W0807 10:38:39.949601 26156 dataset_builder.py:83] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\builders\dataset_builder.py:100: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_deterministic`.
W0807 10:38:39.953629 26156 deprecation.py:323] From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\builders\dataset_builder.py:100: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_deterministic`.
WARNING:tensorflow:From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\builders\dataset_builder.py:175: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
W0807 10:38:39.996484 26156 deprecation.py:323] From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\builders\dataset_builder.py:175: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
WARNING:tensorflow:From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\data_decoders\tf_example_decoder.py:703: calling map_fn (from tensorflow.python.ops.map_fn) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Use fn_output_signature instead
W0807 10:38:41.664895 26156 deprecation.py:506] From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\data_decoders\tf_example_decoder.py:703: calling map_fn (from tensorflow.python.ops.map_fn) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Use fn_output_signature instead
WARNING:tensorflow:From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\util\dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
W0807 10:38:47.547355 26156 deprecation.py:323] From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\util\dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
WARNING:tensorflow:From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\inputs.py:259: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0807 10:38:50.558459 26156 deprecation.py:323] From C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\inputs.py:259: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\eager\context.py", line 2102, in execution_mode
yield
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 758, in _next_internal
output_shapes=self._flat_output_shapes)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 2610, in iterator_get_next
_ops.raise_from_not_ok_status(e, name)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\framework\ops.py", line 6843, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[10] = 10 is not in [0, 0)
[[{{node GatherV2_4}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]] [Op:IteratorGetNext]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "model_main_tf2.py", line 113, in <module>
tf.compat.v1.app.run()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\absl\app.py", line 299, in run
_run_main(main, args)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\absl\app.py", line 250, in _run_main
sys.exit(main(argv))
File "model_main_tf2.py", line 109, in main
record_summaries=FLAGS.record_summaries)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\model_lib_v2.py", line 561, in train_loop
unpad_groundtruth_tensors)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\object_detection\model_lib_v2.py", line 342, in load_fine_tune_checkpoint
features, labels = iter(input_dataset).next()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 645, in next
return self.__next__()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 649, in __next__
return self.get_next()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 694, in get_next
self._iterators[i].get_next_as_list_static_shapes(new_name))
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\distribute\input_lib.py", line 1474, in get_next_as_list_static_shapes
return self._iterator.get_next()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\data\ops\multi_device_iterator_ops.py", line 581, in get_next
result.append(self._device_iterators[i].get_next())
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 825, in get_next
return self._next_internal()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 764, in _next_internal
return structure.from_compatible_tensor_list(self._element_spec, ret)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\eager\context.py", line 2105, in execution_mode
executor_new.wait()
File "C:\ProgramData\anaconda3\envs\4_SOA_OD_v2\lib\site-packages\tensorflow\python\eager\executor.py", line 67, in wait
pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[10] = 10 is not in [0, 0)
[[{{node GatherV2_4}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]]
```
## 6. System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
- TensorFlow installed from (source or binary): Using pip install
- TensorFlow version (use command below): v2.3.0-rc2-23-gb36436b087 2.3.0
- Python version: 3.6
- CUDA/cuDNN version: cudart64_101.dll
- GPU model and memory: GTX 2080TI 11GB
```
== check python ===================================================
python version: 3.6.11
python branch:
python build version: ('default', 'Aug 5 2020 19:41:03')
python compiler version: MSC v.1916 64 bit (AMD64)
python implementation: CPython
== check os platform ===============================================
os: Windows
os kernel version: 10.0.17763
os release version: 10
os platform: Windows-10-10.0.17763-SP0
linux distribution: ('', '', '')
linux os distribution: ('', '', '')
mac version: ('', ('', '', ''), '')
uname: uname_result(system='Windows', node='DESKTOP-8I4BAP8', release='10', version='10.0.17763', machine='AMD64', processor='Intel64 Family 6 Model 158 Stepping 10, GenuineIntel')
architecture: ('64bit', 'WindowsPE')
machine: AMD64
== are we in docker =============================================
No
== compiler =====================================================
c++.exe (MinGW.org GCC Build-20200227-1) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
== check pips ===================================================
numpy 1.19.1
protobuf 3.12.3
tensorflow 2.3.0
tensorflow-addons 0.11.0
tensorflow-datasets 3.2.1
tensorflow-estimator 2.3.0
tensorflow-hub 0.8.0
tensorflow-metadata 0.22.2
tensorflow-model-optimization 0.4.1
== check for virtualenv =========================================
False
== tensorflow import ============================================
tf.version.VERSION = 2.3.0
tf.version.GIT_VERSION = v2.3.0-rc2-23-gb36436b087
tf.version.COMPILER_VERSION = MSVC 192628806
2020-08-07 10:33:58.725634: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset
== nvidia-smi ===================================================
Fri Aug 07 10:34:01 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 442.19 Driver Version: 442.19 CUDA Version: 10.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce RTX 208... WDDM | 00000000:09:00.0 On | N/A |
| 25% 30C P8 28W / 300W | 1352MiB / 11264MiB | 1% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1352 C+G ...mmersiveControlPanel\SystemSettings.exe N/A |
| 0 1756 C+G ...2.0_x64__8wekyb3d8bbwe\WinStore.App.exe N/A |
| 0 1964 C+G C:\Windows\System32\MicrosoftEdgeCP.exe N/A |
| 0 5040 C+G Insufficient Permissions N/A |
| 0 5424 C+G C:\Windows\explorer.exe N/A |
| 0 11260 C+G ...t_cw5n1h2txyewy\ShellExperienceHost.exe N/A |
| 0 12224 C+G ...lmy\AppData\Roaming\Spotify\Spotify.exe N/A |
| 0 12868 C+G ...osoft.LockApp_cw5n1h2txyewy\LockApp.exe N/A |
| 0 12900 C+G C:\Program Files\Hue Sync\HueSync.exe N/A |
| 0 13396 C+G ...1.95.0_x64__8wekyb3d8bbwe\YourPhone.exe N/A |
| 0 13492 C+G ...hell.Experiences.TextInput.InputApp.exe N/A |
| 0 13556 C+G ...0.6242.0_x64__8wekyb3d8bbwe\GameBar.exe N/A |
| 0 15188 C+G ...x64__8wekyb3d8bbwe\Microsoft.Photos.exe N/A |
| 0 16064 C+G Insufficient Permissions N/A |
| 0 16748 C+G ...oftEdge_8wekyb3d8bbwe\MicrosoftEdge.exe N/A |
| 0 17708 C+G ...dqpraam7r\AcrobatNotificationClient.exe N/A |
| 0 17744 C+G ...rosoft Office\root\Office16\OUTLOOK.EXE N/A |
| 0 18160 C+G ...AppData\Local\slack\app-4.7.0\slack.exe N/A |
| 0 19316 C+G ...DIA GeForce Experience\NVIDIA Share.exe N/A |
| 0 21808 C+G ...16211.0_x64__8wekyb3d8bbwe\Video.UI.exe N/A |
| 0 22076 C+G ...dows.Cortana_cw5n1h2txyewy\SearchUI.exe N/A |
| 0 24124 C+G ... Files (x86)\Dropbox\Client\Dropbox.exe N/A |
+-----------------------------------------------------------------------------+
== cuda libs ===================================================
== tensorflow installed from info ==================
Name: tensorflow
Version: 2.3.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: c:\programdata\anaconda3\envs\4_soa_od_v2\lib\site-packages
Required-by: tf-models-official
== python version ==============================================
(major, minor, micro, releaselevel, serial)
(3, 6, 11, 'final', 0)
== bazel version ===============================================
== check python ===================================================
python version: 3.6.11
python branch:
python build version: ('default', 'Aug 5 2020 19:41:03')
python compiler version: MSC v.1916 64 bit (AMD64)
python implementation: CPython
== check os platform ===============================================
os: Windows
os kernel version: 10.0.17763
os release version: 10
os platform: Windows-10-10.0.17763-SP0
linux distribution: ('', '', '')
linux os distribution: ('', '', '')
mac version: ('', ('', '', ''), '')
uname: uname_result(system='Windows', node='DESKTOP-8I4BAP8', release='10', version='10.0.17763', machine='AMD64', processor='Intel64 Family 6 Model 158 Stepping 10, GenuineIntel')
architecture: ('64bit', 'WindowsPE')
machine: AMD64
== are we in docker =============================================
No
== compiler =====================================================
c++.exe (MinGW.org GCC Build-20200227-1) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
== check pips ===================================================
numpy 1.19.1
protobuf 3.12.3
tensorflow 2.3.0
tensorflow-addons 0.11.0
tensorflow-datasets 3.2.1
tensorflow-estimator 2.3.0
tensorflow-hub 0.8.0
tensorflow-metadata 0.22.2
tensorflow-model-optimization 0.4.1
== check for virtualenv =========================================
False
== tensorflow import ============================================
tf.version.VERSION = 2.3.0
tf.version.GIT_VERSION = v2.3.0-rc2-23-gb36436b087
tf.version.COMPILER_VERSION = MSVC 192628806
2020-08-07 10:35:32.098298: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset
== nvidia-smi ===================================================
Fri Aug 07 10:35:34 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 442.19 Driver Version: 442.19 CUDA Version: 10.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce RTX 208... WDDM | 00000000:09:00.0 On | N/A |
| 25% 30C P8 26W / 300W | 1344MiB / 11264MiB | 1% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1352 C+G ...mmersiveControlPanel\SystemSettings.exe N/A |
| 0 1756 C+G ...2.0_x64__8wekyb3d8bbwe\WinStore.App.exe N/A |
| 0 1964 C+G C:\Windows\System32\MicrosoftEdgeCP.exe N/A |
| 0 5040 C+G Insufficient Permissions N/A |
| 0 5424 C+G C:\Windows\explorer.exe N/A |
| 0 11260 C+G ...t_cw5n1h2txyewy\ShellExperienceHost.exe N/A |
| 0 12224 C+G ...lmy\AppData\Roaming\Spotify\Spotify.exe N/A |
| 0 12868 C+G ...osoft.LockApp_cw5n1h2txyewy\LockApp.exe N/A |
| 0 12900 C+G C:\Program Files\Hue Sync\HueSync.exe N/A |
| 0 13396 C+G ...1.95.0_x64__8wekyb3d8bbwe\YourPhone.exe N/A |
| 0 13492 C+G ...hell.Experiences.TextInput.InputApp.exe N/A |
| 0 13556 C+G ...0.6242.0_x64__8wekyb3d8bbwe\GameBar.exe N/A |
| 0 15188 C+G ...x64__8wekyb3d8bbwe\Microsoft.Photos.exe N/A |
| 0 16064 C+G Insufficient Permissions N/A |
| 0 16748 C+G ...oftEdge_8wekyb3d8bbwe\MicrosoftEdge.exe N/A |
| 0 17708 C+G ...dqpraam7r\AcrobatNotificationClient.exe N/A |
| 0 17744 C+G ...rosoft Office\root\Office16\OUTLOOK.EXE N/A |
| 0 18160 C+G ...AppData\Local\slack\app-4.7.0\slack.exe N/A |
| 0 19316 C+G ...DIA GeForce Experience\NVIDIA Share.exe N/A |
| 0 21808 C+G ...16211.0_x64__8wekyb3d8bbwe\Video.UI.exe N/A |
| 0 22076 C+G ...dows.Cortana_cw5n1h2txyewy\SearchUI.exe N/A |
| 0 24124 C+G ... Files (x86)\Dropbox\Client\Dropbox.exe N/A |
+-----------------------------------------------------------------------------+
== cuda libs ===================================================
== tensorflow installed from info ==================
Name: tensorflow
Version: 2.3.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: c:\programdata\anaconda3\envs\4_soa_od_v2\lib\site-packages
Required-by: tf-models-official
== python version ==============================================
(major, minor, micro, releaselevel, serial)
(3, 6, 11, 'final', 0)
== bazel version ===============================================
```
<!--
Collect system information using our environment capture script.
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can also obtain the TensorFlow version with:
1. TensorFlow 1.0
`python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"`
2. TensorFlow 2.0
`python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"`
-->
