I'm trying to use the new TF features, namely the Data API, and I'm not sure how prefetch works . In the code below
def dataset_input_fn(...)
dataset = tf.data.TFRecordDataset(filenames, compression_type="ZLIB")
dataset = dataset.map(lambda x:parser(...))
dataset = dataset.map(lambda x,y: image_augmentation(...)
, num_parallel_calls=num_threads
)
dataset = dataset.shuffle(buffer_size)
dataset = dataset.batch(batch_size)
dataset = dataset.repeat(num_epochs)
iterator = dataset.make_one_shot_iterator()
Does it matter between each line above i put dataset=dataset.prefetch(batch_size)
? Or maybe it should be after every operation that will use output_buffer_size
if the data set came from tf.contrib.data
?
source
share