使用Keras ImageGenerator训练多输入模型

Question

我已经构建了一个模型，它包含两个分支，然后合并成单一的分支。为了训练这个模型，我想使用ImageGenerator来增强图像数据，但不知道如何将其应用于混合的输入类型。有没有人有任何关于在keras中处理这个问题的想法？

任何帮助都将不胜感激！

最好， Nick

模型 第一个分支以图像作为输入：

img_model = Sequential()
img_model.add(Convolution2D( 4, 9,9, border_mode='valid', input_shape=(1, 120, 160)))
img_model.add(Activation('relu'))
img_model.add(MaxPooling2D(pool_size=(2, 2)))
img_model.add(Dropout(0.5))
img_model.add(Flatten()) 

第二个分支将辅助数据作为输入：

aux_model = Sequential()
aux_model.add(Dense(3, input_dim=3))

然后这些模型会被合并到最终的模型中：

model = Sequential()
model.add(Merge([img_model, aux_model], mode='concat'))
model.add(Dropout(0.5))
model.add(Dense(5))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adadelta', metrics=['accuracy']) 

培训/问题： 我尝试做以下操作，但显然失败了：

datagen = ImageDataGenerator(
            featurewise_center=False,  # set input mean to 0 over the dataset
            samplewise_center=False,  # set each sample mean to 0
            featurewise_std_normalization=False,  # divide inputs by std of the dataset
            samplewise_std_normalization=False,  # divide each input by its std
            zca_whitening=False,  # apply ZCA whitening
            rotation_range=10, #180,  # randomly rotate images in the range (degrees, 0 to 180)
            width_shift_range=0.1,  # randomly shift images horizontally (fraction of total width)
            height_shift_range=0.1,  # randomly shift images vertically (fraction of total height)
            horizontal_flip=False,  # randomly flip images
            vertical_flip=False)  # randomly flip images

model.fit_generator( datagen.flow( [X,I], Y, batch_size=64),
               samples_per_epoch=X.shape[0],
               nb_epoch=20,
               validation_data=([Xval, Ival], Yval))

这会生成以下错误信息：

Traceback (most recent call last):
  File "importdata.py", line 139, in <module>
    model.fit_generator( datagen.flow( [X,I], Y, batch_size=64),
  File "/usr/local/lib/python3.5/dist-packages/keras/preprocessing/image.py", line 261, in flow
    save_to_dir=save_to_dir, save_prefix=save_prefix, save_format=save_format)
  File "/usr/local/lib/python3.5/dist-packages/keras/preprocessing/image.py", line 454, in __init__
    'Found: X.shape = %s, y.shape = %s' % (np.asarray(X).shape, np.asarray(y).shape))
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/numeric.py", line 482, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: could not broadcast input array from shape (42700,1,120,160) into shape (42700)

- Nick Jagiella

Answer 1

我想我找到了一种让它工作的方法。假设我们有多个输入模型。

#declare a final model with multiple inputs.
# final_model ...

train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2)
train_generator = train_datagen.flow_from_directory(train_data_dir, target_size=(224, 224), batch_size=32, class_mode='binary') 

# NOTE: the zip combining multiple image generators with on the fly augmentation.
final_generator = zip(train_generator, train_generator)    
final_model.fit_generator(final_generator, samples_per_epoch=nb_train_samples, nb_epoch=nb_epoch, validation_data=test_generator, nb_val_samples=nb_validation_samples)

- Srikar Appalaraju

---

对我来说不起作用，出现错误：ValueError: 模型期望2个输入数组，但只收到一个数组。找到：形状为（0，299，299，3）的数组。 - Dmitry

有没有办法将 train_generator 保存下来以备后用，比如说 .pickle 或其他什么格式？ - Surya Palaniswamy

Answer 2

使用这个：

def trainGeneratorFunc():
    while True:
        xy = trainGeneratorBasic.next()
        yield [xy[0], xy[0], xy[0]], xy[1]

trainGenerator = trainGeneratorFunc()

- Lokesh Dhaundiyal