如何将Keras模型插入到scikit-learn管道中？

Question

我正在使用 scikit-learn 自定义管道 (sklearn.pipeline.Pipeline) 与 RandomizedSearchCV 进行超参数优化。这个方法非常好用。

现在，我想将一个 keras 模型作为管道的第一步插入其中。模型的参数应该被优化。计算出的 (拟合的) keras 模型应该在后面的管道步骤中被使用，所以我认为我必须将模型存储为全局变量，以便其他管道步骤可以使用它。这样做对吗？

我知道keras提供一些scikit-learn API的wrappers，但问题是这些wrappers已经执行了分类/回归，而我只想计算keras模型，仅此而已。

怎样才能做到呢？

例如，我有一个返回模型的方法：

def create_model(file_path, argument2,...):
  ...
  return model

该方法需要一些固定参数，如file_path等，但不需要X和y（或可以忽略）。模型的参数应该被优化（层数等）。

- machinery

---

你能解释一下你所说的“计算（拟合）的Keras模型应该在后续流程中被其他步骤使用”是什么意思吗？如果Keras模型是最后一步，那么它如何被后续步骤使用呢？ - amanbirs

可能是 tf.keras.wrappers.scikit_learn。 - JeeyCi

Answer 1

首先，你需要把你的Keras模型封装成Scikit learn模型，然后就可以像平常一样进行操作了。

这里给出一个快速示例（为简洁起见，我省略了导入部分）

这里有一篇完整的博客文章，包括此示例和许多其他示例: Scikit-learn Pipeline Examples

# create a function that returns a model, taking as parameters things you
# want to verify using cross-valdiation and model selection
def create_model(optimizer='adagrad',
                  kernel_initializer='glorot_uniform', 
                  dropout=0.2):
    model = Sequential()
    model.add(Dense(64,activation='relu',kernel_initializer=kernel_initializer))
    model.add(Dropout(dropout))
    model.add(Dense(1,activation='sigmoid',kernel_initializer=kernel_initializer))

    model.compile(loss='binary_crossentropy',optimizer=optimizer, metrics=['accuracy'])

    return model

# wrap the model using the function you created
clf = KerasRegressor(build_fn=create_model,verbose=0)

# just create the pipeline
pipeline = Pipeline([
    ('clf',clf)
])

pipeline.fit(X_train, y_train)

- Felipe

---

在Keras模型中，没有“input_dim”参数。这是打字错误吗？ - Stupid420

这个param_grid似乎是网格搜索或随机搜索的输入，但它并没有set_params，只是设置参数而没有搜索（因此不能有例如epochs的列表）？ - user0

Answer 2

现在似乎Scikeras成为在scikit-learn中使用Keras模型的首选封装器，因为Felipe's answer中使用的Keras/TensorFlow自带的封装器已经消失了。

- Robert Pollak

Answer 3

这是对sklearn documentation中RBM example的修改，但使用了tensorflow后端在keras中实现的神经网络。

# -*- coding: utf-8 -*-
"""
Created on Mon Nov 27 17:11:21 2017

@author: ZED
"""

from __future__ import print_function

print(__doc__)

# Authors: Yann N. Dauphin, Vlad Niculae, Gabriel Synnaeve
# License: BSD

import numpy as np
import matplotlib.pyplot as plt

from scipy.ndimage import convolve

from keras.models import Sequential
from keras.layers.core import Dense,Activation
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils

from sklearn import  datasets, metrics
from sklearn.model_selection import train_test_split
from sklearn.neural_network import BernoulliRBM
from sklearn.pipeline import Pipeline


#%%
# Setting up

def nudge_dataset(X, Y):
    """
    This produces a dataset 5 times bigger than the original one,
    by moving the 8x8 images in X around by 1px to left, right, down, up
    """
    direction_vectors = [
        [[0, 1, 0],
          [0, 0, 0],
          [0, 0, 0]],

        [[0, 0, 0],
          [1, 0, 0],
          [0, 0, 0]],

        [[0, 0, 0],
          [0, 0, 1],
          [0, 0, 0]],

        [[0, 0, 0],
          [0, 0, 0],
          [0, 1, 0]]]

    shift = lambda x, w: convolve(x.reshape((8, 8)), mode='constant',
                                  weights=w).ravel()
    X = np.concatenate([X] +
                        [np.apply_along_axis(shift, 1, X, vector)
                        for vector in direction_vectors])
    Y = np.concatenate([Y for _ in range(5)], axis=0)
    return X, Y

# Load Data
digits = datasets.load_digits()
X = np.asarray(digits.data, 'float32')
X, Y = nudge_dataset(X, digits.target)
X = (X - np.min(X, 0)) / (np.max(X, 0) + 0.0001)  # 0-1 scaling

X_train, X_test, Y_train, Y_test = train_test_split(X, Y,
                                                    test_size=0.2,
                                                    random_state=0)

#%%
def create_model():
  
    model = Sequential()
    model.add(Dense(100, input_dim=64))
    model.add(Activation('tanh'))
    
    """
    #other layer
    model.add(Dense(500))
    model.add(Activation('tanh'))
    """
    
    model.add(Dense(10))
    model.add(Activation('softmax'))
    # Compile model
    model.compile(loss = 'binary_crossentropy', optimizer = 'adadelta', metrics=['accuracy'])
    return model

rbm = BernoulliRBM(random_state=0, verbose=True)

#This is the model you want. it is in sklearn format
clf = KerasClassifier(build_fn=create_model, verbose=0)

classifier = Pipeline(steps=[('rbm', rbm), ('VNN', clf)])

#%%
# Training

# Hyper-parameters. These were set by cross-validation,
# using a GridSearchCV. Here we are not performing cross-validation to
# save time.
rbm.learning_rate = 0.06
rbm.n_iter = 20
# More components tend to give better prediction performance, but larger
# fitting time
rbm.n_components = 64

#adapt targets to hot matrix
yTrain = np_utils.to_categorical(Y_train, 10)
# Training RBM-Logistic Pipeline
classifier.fit(X_train, yTrain)

#%%
# Evaluation

print()
print("NN using RBM features:\n%s\n" % (
    metrics.classification_report(
        Y_test,
        classifier.predict(X_test))))

#%%
# Plotting

plt.figure(figsize=(4.2, 4))
for i, comp in enumerate(rbm.components_):
    plt.subplot(10, 10, i + 1)
    plt.imshow(comp.reshape((8, 8)), cmap=plt.cm.gray_r,
                interpolation='nearest')
    plt.xticks(())
    plt.yticks(())
plt.suptitle('64 components extracted by RBM', fontsize=16)
plt.subplots_adjust(0.08, 0.02, 0.92, 0.85, 0.08, 0.23)

plt.show()

- Ahmad