有没有人能帮我把 mnist 数据集按照我们所需的比例分成训练、测试和验证集。
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
使用70-20-10的比例进行训练、验证和测试。
有没有人能帮我把 mnist 数据集按照我们所需的比例分成训练、测试和验证集。
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
使用70-20-10的比例进行训练、验证和测试。
tf.keras.datasets.mnist
API提供的默认训练集和测试集拆分,您可以将训练集和测试集合并后,根据您的比例逐步将它们拆分为训练集、验证集和测试集。请注意保留HTML标签。from sklearn.model_selection import train_test_split
import tensorflow as tf
DATASET_SIZE = 70000
TRAIN_RATIO = 0.7
VALIDATION_RATIO = 0.2
TEST_RATIO = 0.1
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
X = np.concatenate([x_train, x_test])
y = np.concatenate([y_train, y_test])
sklearn.model_selection import train_test_split()
函数。以下是一个示例:X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=(1-TRAIN_RATIO))
X_val, X_test, y_val, y_test = train_test_split(X_val, y_val, test_size=((TEST_RATIO/(VALIDATION_RATIO+TEST_RATIO))))
如果您喜欢使用tf Dataset API,那么可以按照以下方式使用.take()
和.skip()
方法:
dataset = tf.data.Dataset.from_tensor_slices((X, y))
train_dataset = dataset.take(int(TRAIN_RATIO*DATASET_SIZE))
validation_dataset = dataset.skip(int(TRAIN_RATIO*DATASET_SIZE)).take(int(VALIDATION_RATIO*DATASET_SIZE))
test_dataset = dataset.skip(int(TRAIN_RATIO*DATASET_SIZE)).skip(int(VALIDATION_RATIO*DATASET_SIZE))
.shuffle()
添加到数据集中以生成随机分区:dataset = dataset.shuffle()
train_test_split
函数来迭代地将数据集分成验证集-测试集-训练集:train_ratio = 0.70
validation_ratio = 0.20
test_ratio = 0.10
# train is now 70% of the entire data set
# the _junk suffix means that we drop that variable completely
x_train, x_test, y_train, y_test = train_test_split(dataX, dataY, test_size=1 - train_ratio)
# test is now 10% of the initial data set
# validation is now 20% of the initial data set
x_val, x_test, y_val, y_test = train_test_split(x_test, y_test, test_size=test_ratio/(test_ratio + validation_ratio))
dataX
和 dataY
的调用吗? - blkpingu