使用caret指定交叉验证折数

Question

你好，首先感谢您的提问。我正在使用caret来交叉验证来自nnet包的神经网络。在trainControl函数的method参数中，我可以指定我的交叉验证类型，但是所有这些类型都会随机选择观测值来进行交叉验证。是否有任何方法可以使用caret对我数据中的特定观测进行交叉验证，无论是通过ID还是硬编码的参数？例如，这是我的当前代码：

library(nnet) 
library(caret) 
library(datasets) 

data(iris) 

train.control <- trainControl( 
    method = "repeatedcv" 
    , number = 4 
    , repeats = 10 
    , verboseIter = T 
    , returnData = T 
    , savePredictions = T 
    ) 

tune.grid <- expand.grid( 
    size = c(2,4,6,8)
    ,decay = 2^(-3:1) 
    ) 

nnet.train <- train( 
    x = iris[,1:4] 
    , y = iris[,5] 
    , method = "nnet" 
    , preProcess = c("center","scale")  
    , metric = "Accuracy" 
    , trControl = train.control 
    , tuneGrid = tune.grid 
    ) 
nnet.train 
plot(nnet.train)

假设我想向数据框iris添加另一列CV_GROUP，并且我希望caret在对具有该列值为1的观测进行神经网络交叉验证时使用它：

iris$CV_GROUP <- c(rep.int(0,times=nrow(iris)-20), rep.int(1,times=20))

这是否可以使用caret实现？

- gtnbz2nyt

Answer 1

使用 index 和 indexOut 控制选项。我编写了一种实现方式，可以让您选择想要的重复次数和折叠次数：

library(nnet)
library(caret)
library(datasets)
library(data.table)
library(e1071)

r <- 2 # number of repeats
k <- 5 # number of folds
data(iris)
iris <- data.table(iris)

# Create folds and repeats here - you could create your own if you want #
set.seed(343)
for (i in 1:r) {
    newcol <- paste('fold.num',i,sep='')
    iris <- iris[,eval(newcol):=sample(1:k, size=dim(iris)[1], replace=TRUE)]
}

folds.list.out <- list()
folds.list <- list()
list.counter <- 1
for (y in 1:r) {
    newcol <- paste('fold.num', y, sep='')
    for (z in 1:k) {
        folds.list.out[[list.counter]] <- which(iris[,newcol,with=FALSE]==z)
        folds.list[[list.counter]] <- which(iris[,newcol,with=FALSE]!=z)
        list.counter <- list.counter + 1
    }
    iris <- iris[,!newcol,with=FALSE]
}

tune.grid <- expand.grid( 
    size = c(2,4,6,8)
    ,decay = 2^(-3:1) 
    ) 

train.control <- trainControl( 
    index=folds.list
    , indexOut=folds.list.out
    , verboseIter = T 
    , returnData = T 
    , savePredictions = T 
    ) 

iris <- data.frame(iris)

nnet.train <- train( 
    x = iris[,1:4] 
    , y = iris[,5] 
    , method = "nnet" 
    , preProcess = c("center","scale")  
    , metric = "Accuracy" 
    , trControl = train.control 
    , tuneGrid = tune.grid 
    ) 

nnet.train
plot(nnet.train)

- Tchotchke

---

抱歉这么晚才确认，我被其他事情分心了，但还是谢谢你！答案很棒！ - gtnbz2nyt