我目前正在训练一个Keras模型，其对应的fit调用如下：
model.fit(X,y_train,batch_size=myBatchSize,epochs=myAmountOfEpochs,validation_split=0.1,callbacks=myCallbackList)
这个评论在Keras Github页面上解释了"validation_split=0.1"的含义：
验证数据不一定来自每个类别，只是数据的最后10％（假设您要求10％）。
我的问题是：是否有一种简单的方法可以随机选择，例如，将我的训练数据的10％作为验证数据？我想使用随机选择的样本的原因是，在我的情况下，数据的最后10％并不一定包含所有类别。
非常感谢。
Keras没有提供比仅仅从训练数据中取出一部分用于验证更高级的功能。如果您需要更高级的功能，例如分层抽样以确保样本中各类别得到充分代表，则需要在Keras之外手动执行此操作（例如使用scikit-learn或numpy），然后通过model.fit中的validation_data参数将该验证数据传递给Keras。
感谢Matias Valdenegro的评论，我受到启发并想出了以下解决方案来解决我的问题：
from sklearn.model_selection import train_test_split
[input: X and Y]
XTraining, XValidation, YTraining, YValidation = train_test_split(X,Y,stratify=Y,test_size=0.1) # before model building
[The model is built here...]
model.fit(XTraining,YTraining,batch_size=batchSize,epochs=amountOfEpochs,validation_data=(XValidation,YValidation),callbacks=callbackList)
.flow_from_directory方法指定您的训练和验证路径。import split_folders
# Split with a ratio.
# To only split into training and validation set, set a tuple to `ratio`, i.e, `(.8, .2)`.
split_folders.ratio('input_folder', output="output", seed=1337, ratio=(.8, .1, .1)) # default values
# Split val/test with a fixed number of items e.g. 100 for each set.
# To only split into training and validation set, use a single number to `fixed`, i.e., `10`.
split_folders.fixed('input_folder', output="output", seed=1337, fixed=(100, 100), oversample=False) # default values
input/
class1/
img1.jpg
img2.jpg
...
class2/
imgWhatever.jpg
...
...
output/
train/
class1/
img1.jpg
...
class2/
imga.jpg
...
val/
class1/
img2.jpg
...
class2/
imgb.jpg
...
test/ # optional
class1/
img3.jpg
...
class2/
imgc.jpg
...
import tensorflow as tf
import split_folders
import os
main_dir = '/Volumes/WMEL/Independent Research Project/Data/test_train/Data'
output_dir = '/Volumes/WMEL/Independent Research Project/Data/test_train/output'
split_folders.ratio(main_dir, output=output_dir, seed=1337, ratio=(.7, .3))
train_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
rescale=1./224)
train_generator = train_datagen.flow_from_directory(os.path.join(output_dir,'train'),
class_mode='categorical',
batch_size=32,
target_size=(224,224),
shuffle=True)
validation_generator = train_datagen.flow_from_directory(os.path.join(output_dir,'val'),
target_size=(224, 224),
batch_size=32,
class_mode='categorical',
shuffle=True) # set as validation data
base_model = tf.keras.applications.ResNet50V2(
input_shape=IMG_SHAPE,
include_top=False,
weights=None)
maxpool_layer = tf.keras.layers.GlobalMaxPooling2D()
prediction_layer = tf.keras.layers.Dense(4, activation='softmax')
model = tf.keras.Sequential([
base_model,
maxpool_layer,
prediction_layer
])
opt = tf.keras.optimizers.Adam(lr=0.004)
model.compile(optimizer=opt,
loss=tf.keras.losses.CategoricalCrossentropy(),
metrics=['accuracy'])
model.fit(
train_generator,
steps_per_epoch = train_generator.samples // 32,
validation_data = validation_generator,
validation_steps = validation_generator.samples // 32,
epochs = 20)
根据 Keras 入门 FAQ，您可以在 model.fit 中使用 shuffle 参数。
model.fit() 的参数中，validation_data 会覆盖 validation_split，因此没有必要同时配置两者。validation_split: Float between 0 and 1.
Fraction of the training data to be used as validation data.
The model will set apart this fraction of the training data,
will not train on it, and will evaluate
the loss and any model metrics
on this data at the end of each epoch.
validation_data: Data on which to evaluate
the loss and any model metrics at the end of each epoch.
The model will not be trained on this data.
`validation_data` will override `validation_split`
但是有一个选项可以实现您的目的，那就是参数shuffle
shuffle: Boolean (whether to shuffle the training data
before each epoch) or str (for 'batch').
'batch' is a special option for dealing with the
limitations of HDF5 data; it shuffles in batch-sized chunks.
所以你可以做的是：
model.fit(**other_kwargs, validation_split = 0.1, shuffle=True)
评论不够长，所以我在这里发表。
如果你有1000个训练数据、100个测试数据、validation_split=0.1和batch_size=100，它会做的是：在训练数据上进行拆分（批次1：90个训练和10个验证，批次2：90个训练和10个验证，...，所有的原始顺序，90,10,90,10...90,10），这与100个测试数据无关（你的模型永远不会看到它）。所以我猜你只想打乱所有大小为10的验证集，而不触及90大小的训练集。我可能会手动洗牌我的数据的10%部分，因为这就是shuffle=True的作用，它只是洗牌索引并用新的洗牌索引替换旧的训练数据，像这样）：
import numpy as np
train_index = np.arange(1000,dtype=np.int32)
split = 0.1
batch_size = 100
num_batch = int(len(train_index)/batch_size)
train_index = np.reshape(train_index,(num_batch,batch_size))
for i in range(num_batch):
r = np.random.choice(range(10),10,replace=False)
print(r)
train_index[i,int((1-split)*batch_size):] = np.array(r+((1-split)*batch_size)+i*batch_size)
print(train_index[i])
flatten_index = train_index.reshape(-1)
print(flatten_index)
x_train = np.arange(1000,2000)
x_train = x_train[flatten_index]
print(x_train)