我认为文档可能会让人感到困惑,而且我想不同版本的Tensorflow和Keras的行为可能是不同的。例如,在这个帖子中,用户描述了你期望的确切行为。通常,flow_from_directory()
方法允许您直接从目录中读取图像,并在模型正在训练时对其进行增强。正如已经在这里所述,每个时期它都会迭代每个文件夹中的每个样本。通过使用以下示例,您可以检查这一点(在TF 2.7上),只需查看进度条中的每个时期的步骤:
import tensorflow as tf
BATCH_SIZE = 64
flowers = tf.keras.utils.get_file(
'flower_photos',
'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
untar=True)
img_gen = tf.keras.preprocessing.image.ImageDataGenerator(
rescale=1./255,
horizontal_flip=True,
)
train_ds = img_gen.flow_from_directory(flowers, batch_size=BATCH_SIZE, shuffle=True, class_mode='sparse')
num_classes = 5
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(16, 3, padding='same', activation='relu', input_shape=(256, 256, 3)),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Conv2D(32, 3, padding='same', activation='relu'),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu'),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(num_classes)
])
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True))
epochs=10
history = model.fit(
train_ds,
epochs=epochs
)
Found 3670 images belonging to 5 classes.
Epoch 1/10
6/58 [==>...........................] - ETA: 3:02 - loss: 2.0608
如果您将
flow_from_directory
包装在
tf.data.Dataset.from_generator
中,就像这样:
train_ds = tf.data.Dataset.from_generator(
lambda: img_gen.flow_from_directory(flowers, batch_size=BATCH_SIZE, shuffle=True, class_mode='sparse'),
output_types=(tf.float32, tf.float32))
您会注意到进度条看起来像这样,因为
steps_per_epoch
没有被明确定义:
Epoch 1/10
Found 3670 images belonging to 5 classes.
29/Unknown - 104s 4s/step - loss: 2.0364
如果您添加了此参数,您将在进度条中看到步骤:
history = model.fit(
train_ds,
steps_per_epoch = len(from_directory),
epochs=epochs
)
Found 3670 images belonging to 5 classes.
Epoch 1/10
3/58 [>.............................] - ETA: 3:19 - loss: 4.1357
最后回答你的问题:
如何正确使用此生成器与适合所有数据的函数,包括原始的非增强图像和增强图像,并循环多次/步骤?
您可以通过乘以某个因子来简单地增加steps_per_epoch
超过样本数//批量大小
:
history = model.fit(
train_ds,
steps_per_epoch = len(from_directory)*2,
epochs=epochs
)
Found 3670 images belonging to 5 classes.
Epoch 1/10
1/116 [..............................] - ETA: 12:11 - loss: 1.5885
现在每个时期的步骤数从58增加到了116。