我正在尝试将hmeasure度量Hand,2009作为我在caret中训练SVMs的自定义指标。由于我相对新使用R,我尝试调整twoClassSummary函数。我只需要传递模型(一个svm)中的真实类标签和预测类概率到hmeasure包中的HMeasure函数,而不是在caret中使用ROC或其他分类性能度量即可。
例如,在R中调用HMeasure函数 - HMeasure(true.class,predictedProbs[,2]) - 将计算Hmeasure。但是,使用下面的twoClassSummary代码的调整会返回错误:'x'必须是数字。
也许train函数无法“看到”预测的概率来评估HMeasure函数。我该如何解决这个问题?
我已经阅读了文档,并链接了SO上提出的相关问题处理回归。这使我有所进展。如果有任何帮助或指向解决方案的指针,我将不胜感激。
下面是无法工作的代码。
例如,在R中调用HMeasure函数 - HMeasure(true.class,predictedProbs[,2]) - 将计算Hmeasure。但是,使用下面的twoClassSummary代码的调整会返回错误:'x'必须是数字。
也许train函数无法“看到”预测的概率来评估HMeasure函数。我该如何解决这个问题?
我已经阅读了文档,并链接了SO上提出的相关问题处理回归。这使我有所进展。如果有任何帮助或指向解决方案的指针,我将不胜感激。
library(caret)
library(doMC)
library(hmeasure)
library(mlbench)
set.seed(825)
data(Sonar)
table(Sonar$Class)
inTraining <- createDataPartition(Sonar$Class, p = 0.75, list = FALSE)
training <- Sonar[inTraining, ]
testing <- Sonar[-inTraining, ]
# using caret
fitControl <- trainControl(method = "repeatedcv",number = 2,repeats=2,summaryFunction=twoClassSummary,classProbs=TRUE)
svmFit1 <- train(Class ~ ., data = training,method = "svmRadial",trControl = fitControl,preProc = c("center", "scale"),tuneLength = 8,metric = "ROC")
predictedProbs <- predict(svmFit1, newdata = testing , type = "prob")
true.class<-testing$Class
hmeas<- HMeasure(true.class,predictedProbs[,2]) # suppose its Rocks we're interested in predicting
hmeasure.probs<-hmeas$metrics[c('H')] # returns the H measure metric
hmeasureCaret<-function (data, lev = NULL, model = NULL,...)
{
# adaptation of twoClassSummary
require(hmeasure)
if (!all(levels(data[, "pred"]) == levels(data[, "obs"])))
stop("levels of observed and predicted data do not match")
#lev is a character string that has the outcome factor levels taken from the training data
hObject <- try(hmeasure::HMeasure(data$obs, data[, lev[1]]),silent=TRUE)
hmeasH <- if (class(hObject)[1] == "try-error") {
NA
} else {hObject$metrics[[1]] #hObject$metrics[c('H')] returns a dataframe, need to return a vector
}
out<-hmeasH
names(out) <- c("Hmeas")
#class(out)
}
environment(hmeasureCaret) <- asNamespace('caret')
下面是无法工作的代码。
ctrl <- trainControl(method = "cv", summaryFunction = hmeasureCaret,classProbs=TRUE,allowParallel = TRUE,
verboseIter=TRUE,returnData=FALSE,savePredictions=FALSE)
set.seed(1)
svmTune <- train(Class.f ~ ., data = training,method = "svmRadial",trControl = ctrl,preProc = c("center", "scale"),tuneLength = 8,metric="Hmeas",
verbose = FALSE)