logo标签列表

我该如何告诉h2o深度学习网格使用AUC而不是残差偏差？

rh2oauc
3

我希望通过查找AUC或准确性来衡量模型的性能。在网格搜索中，我得到了残差偏差的结果，如何告诉h2o深度学习网格使用AUC而不是残差偏差，并将结果呈现为类似于下面附加的表格？

train <- read.table(text = "target birds    wolfs     snakes
                              0        9         7 a
                              0        8         4 b
                              1        2         8 c
                              1        2         3 a
                              1        8         3 a
                              0        1         2 a
                              0        7         1 b
                              0        1         5 c
                              1        9         7 c
                              1        8         7 c
                              0        2         7 b
                              1        2         3 b
                              1        6         3 c
                              0        1         1 a
                              0        3         9 a
                              1        1         1 b ",header = TRUE)
trainHex <- as.h2o(train)

g <- h2o.grid("deeplearning",
              hyper_params = list(
                  seed = c(123456789,12345678,1234567),
                  activation = c("Rectifier", "Tanh", "TanhWithDropout", "RectifierWithDropout", "Maxout", "MaxoutWithDropout")
              ),
              reproducible = TRUE,
              x = 2:4,
              y = 1,
              training_frame = trainHex,
              validation_frame = trainHex,
              epochs = 50,
              )
g
model_ids <- g@summary_table
model_ids<-as.data.frame(model_ids)

我得到的结果表格：
     Hyper-Parameter Search Summary: ordered by increasing residual_deviance
             activation      seed                                                  model_ids   residual_deviance
1                Maxout  12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_10 0.07243775676256235
2                Maxout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_16 0.10060885040861599
3     MaxoutWithDropout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_5  0.1706496158406441
4                Maxout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_4 0.17243125875659948
5                  Tanh 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_1 0.18326527198894926
6                  Tanh  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_7 0.18763395264761593
7                  Tanh   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_13 0.18791531211136187
8       TanhWithDropout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_2 0.19808063817007837
9       TanhWithDropout  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_8 0.19815190962052193
10      TanhWithDropout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_14 0.19832946889767458
11            Rectifier 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_0 0.20679125165086842
12    MaxoutWithDropout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_17 0.21971759565380736
13 RectifierWithDropout 123456789  Grid_DeepLearning_train_model_R_1483217086840_112_model_3 0.22337599298253263
14    MaxoutWithDropout  12345678 Grid_DeepLearning_train_model_R_1483217086840_112_model_11 0.22440661112729862
15 RectifierWithDropout   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_15  0.2284671685474275
16 RectifierWithDropout  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_9 0.23163744415703522
17            Rectifier   1234567 Grid_DeepLearning_train_model_R_1483217086840_112_model_12  0.2516917276707789
18            Rectifier  12345678  Grid_DeepLearning_train_model_R_1483217086840_112_model_6  0.2642221616447725
1
顺便提一下，将validation_frame设置为与training_frame相同是默认行为，因此无需指定。请注意，如果不使用验证和测试数据集，则正在优化深度学习参数以过度拟合最佳结果。我甚至不确定您可以从随机种子对结果变化的影响所学到的内容适用于未见过的数据。（当然，这仍然可能是一个有趣的实验：例如，我以前做过这个实验，以查看需要多少隐藏节点/层/时期才能完美地拟合数据。） - Darren Cook
1个回答
3
你可以使用 h2o.getGrid() 来实现这个功能。继续你的示例代码：
g_rmse <- h2o.getGrid(g@grid_id, "rmse")
g_rmse  #Output it

我选择了root-MSE。您的样本数据不支持AUC：它必须是二项分类，而您正在进行回归分析。
您之所以进行回归分析，是因为您的y包含0和1，因此H2O猜测它是数字。您需要在将其上传到H2O后，在该列上使用as.factor()函数。
train <-  ...
trainHex <- as.h2o(train)
trainHex[,1] = as.factor(trainHex[,1])  #Add this

g <- ...

然后你可以这样做：
g_auc <- h2o.getGrid(g@grid_id, "auc", decreasing = TRUE)
g_auc

我已将其设置为decreasing=TRUE，以使最佳AUC位于顶部。
非常感谢您详细的回答，@Darren Cook。 - mql4beginner
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接