如何在Keras模型中停用被称为training=True的dropout层？

Question

如何在Keras模型中停用被称为training=True的dropout层？

3

我希望能够查看训练tf.keras模型的最终输出结果，即来自softmax函数的预测数组，例如[0,0,0,1,0,1]。其他线程建议使用model.predict(training_data)，但在我的情况下，这不起作用，因为我在训练和验证过程中使用了dropout，因此神经元会被随机丢弃，使用相同数据再次进行预测将得到不同的结果。

def get_model():
    inputs = tf.keras.layers.Input(shape=(input_dims,))
    x = tf.keras.layers.Dropout(rate=dropout_rate)(inputs, training=True)
    x = tf.keras.layers.Dense(units=29, activation='relu')(x)
    x = tf.keras.layers.Dropout(rate=dropout_rate)(x, training=True)  
    x = tf.keras.layers.Dense(units=15, activation='relu')(x)
    outputs = tf.keras.layers.Dense(2, activation='softmax')(x)
    model = tf.keras.Model(inputs=inputs, outputs=outputs)
    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',      
                  metrics=['sparse_categorical_accuracy'])
    return model

myModel = get_model()
myModel.summary()
myModel.fit(X_train, y_train,
           batch_size = batch_size,
           epochs= epochs,
           verbose = 1,
           validation_data = (X_val, y_val))

在TensorFlow中，你可以很容易地获取模型训练后的输出。以下是来自Github仓库的一个示例：

input = tf.placeholder(tf.float32, shape=[None, INPUT_DIMS])
labels = tf.placeholder(tf.float32, shape=[None])

hidden = tf.nn.tanh(make_nn_layer(normalized, NUM_HIDDEN))
logits = make_nn_layer(hidden, NUM_CLASSES)
outputs = tf.argmax(logits, 1)

int_labels = tf.to_int64(labels)
cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, int_labels, name='xentropy')
train_step = tf.train.AdamOptimizer().minimize(cross_entropy)

correct_prediction = tf.equal(outputs, int_labels)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())

    validation_dict = {
        input: validation_data[:,0:7],
        labels: validation_data[:,7],}

    for i in range(NUM_BATCHES):
        batch = training_data[numpy.random.choice(training_size, BATCH_SIZE, False),:]
        train_step.run({input: batch[:,0:7], labels: batch[:,7]})

        if i % 100 == 0 or i == NUM_BATCHES - 1:
            print('Accuracy %.2f%% at step %d' % (accuracy.eval(validation_dict) * 100, i))

    output_data = outputs.eval({input: data_vector[:,0:7]})

我能从训练好的模型中得到的唯一输出似乎是一个历史对象。还有一个名为myModel.output的对象，但它是一个张量，如果没有将数据放入其中，我无法对其进行评估。有任何想法吗？

- la_leche

1

你是在询问如何获取模型的可视化模型吗？还是在寻找输出数据，类似于 x = 0, y = 1 的东西？ - user11803894

@Cygnus 是的，我正在寻找输出数据（预测结果），类似于model.predict()的工作方式。 - la_leche

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- today · Accepted Answer

据我所知，在调用层时，通过传递 training=True 无法关闭dropout（除非将权重转移到具有相同架构的新模型中）。不过，你可以在正常情况下构建和训练模型（即在调用中不使用 training 参数），然后通过定义后端函数（即 keras.backend.function()）并设置学习状态（即 keras.backend.learning_phase()）来选择性地在测试阶段打开和关闭 dropout 层：

# build your model normally (i.e. without using `training=True` argument)

# train your model...

from keras import backend as K

func = K.function(model.inputs + [K.learning_phase()], model.outputs)

# run the model with dropout layers being active, i.e. learning_phase == 1
preds = func(list_of_input_arrays + [1])

# run the model with dropout layers being inactive, i.e. learning_phase == 0
preds = func(list_of_input_arrays + [0])

更新: 如我上面所建议的，另一种方法是定义一个具有相同架构但未设置training=True的新模型，然后将训练好的模型的权重传输到这个新模型中。为了实现这一点，我只需在您的get_model()函数中添加一个training参数：

def get_model(training=None):
    inputs = tf.keras.layers.Input(shape=(input_dims,))
    x = tf.keras.layers.Dropout(rate=dropout_rate)(inputs, training=training)
    x = tf.keras.layers.Dense(units=29, activation='relu')(x)
    x = tf.keras.layers.Dropout(rate=dropout_rate)(x, training=training)  
    x = tf.keras.layers.Dense(units=15, activation='relu')(x)
    outputs = tf.keras.layers.Dense(2, activation='softmax')(x)
    model = tf.keras.Model(inputs=inputs, outputs=outputs)
    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',      
                  metrics=['sparse_categorical_accuracy'])
    return model

# build a model with dropout layers active in both training and test phases
myModel = get_model(training=True)
# train the model
myModel.fit(...)

# build a clone of the model with dropouts deactivated in test phase
myTestModel = get_model()  # note: the `training` is `None` by default
# transfer the weights from the trained model to this model
myTestModel.set_weights(myModel.get_weights())
# use the new model in test phase; the dropouts would not be active
myTestModel.predict(...)

如何在Keras模型中停用被称为training=True的dropout层？

生成二维分类数据集