Unable to download split of dataset with unknowns size

General Discussion
,
#1

I was trying to download the ‘pass’ image dataset pass  |  TensorFlow Datasets but I soon realized that it was way to big, so I tried to download only a part of it.

I found on Dividir y rebanar  |  TensorFlow Datasets that I could simply use

  • Slices: Slices have the same semantic as python slice notation. Slices can be:
    • Absolute ('train[123:450]', train[:4000]): (see note below for caveat about read order)
    • Percent ('train[:75%]', 'train[25%:75%]'): Divide the full data into 100 even slices. If the data is not divisible by 100, some percent might contain additional examples.
    • Shard (train[:4shard], train[4shard]): Select all examples in the requested shard. (see info.splits['train'].num_shards to get the number of shards of the split)

so I tried

  • ds = tfds.load(‘pass’, split=[‘train[:1%]’])
  • ds = tfds.load(‘pass’, split=[‘train[:10]’])

but it seems to completely ignore the commands and try to downoad a huge part of the dataset anyway

image
image946×203 9.81 KB

Does anybody know how to solve this?
Thanks in advance

#3

I don’t think this API is related to the partial download it is just a partial load after the complete dataset download/cache.

#4

See also:

1 Like
#5

Ok, I see, thanks for the answer