如何在OSX上安装支持多线程的XGBoost

6

我正在尝试在我的Mac(OSX 10.12.1)上安装xgboost,按照这里的指南进行,但是遇到了一些问题。


步骤1

通过brew install gcc --without-multilib获取带有openmp支持的gcc-6.x.x

终端

Ben$ brew install gcc --without-multilib
Error: gcc-5.3.0 already installed
To install this version, first `brew unlink gcc`

Ben$ brew unlink gcc
Unlinking /usr/local/Cellar/gcc/5.3.0... 1288 symlinks removed

Ben$ brew install gcc --without-multilib
[26 minutes later]
==> Summary
  /usr/local/Cellar/gcc/6.2.0: 1,358 files, 238.3M, built in 26 minutes 20 seconds

步骤2

克隆存储库 git clone --recursive https://github.com/dmlc/xgboost

终端

Ben$ git clone --recursive https://github.com/dmlc/xgboost
Cloning into 'xgboost'...
remote: Counting objects: 18754, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 18754 (delta 1), reused 0 (delta 0), pack-reused 18733
Receiving objects: 100% (18754/18754), 6.23 MiB | 3.74 MiB/s, done.
Resolving deltas: 100% (11347/11347), done.
Checking connectivity... done.
Submodule 'dmlc-core' (https://github.com/dmlc/dmlc-core) registered for path 'dmlc-core'
Submodule 'rabit' (https://github.com/dmlc/rabit) registered for path 'rabit'
Cloning into '/Users/Ben/xgboost/dmlc-core'...
Cloning into '/Users/Ben/xgboost/rabit'...
Submodule path 'dmlc-core': checked out 'f35f14f30835af238257b979cc1fac3e41ff3291'
Submodule path 'rabit': checked out 'a9a2a69dc1144180a43f7d2d1097264482be7817'

第三步

使用以下命令进行构建:cd xgboost; cp make/config.mk ./config.mk; make -j4

注意:如果您使用OSX El Capitan,则brew会安装最新版本的gcc-6。因此,您可能需要修改Makefile#L46并将gcc-5更改为gcc-6。之后,在make/config.mk中将gcc-5/g++-5更改为gcc-6/g++-6,然后使用以下命令进行构建。

嗯,我不太确定该怎么做。我在xgboost/内看到一个名为“Makefile”的文件。里面有很多我不理解的东西,但是这些行看起来很相关:

Makefile(片段)

# on Mac OS X, force brew gcc-6, since the Xcode c++ fails anyway
# it is useful for pip install compiling-on-the-fly
OS := $(shell uname)
ifeq ($(OS), Darwin)
export CC = $(if $(shell which gcc-6),gcc-6,$(if $(shell which gcc-mp-6), gcc-mp-6, clang))
export CXX = $(if $(shell which g++-6),g++-6,$(if $(shell which g++-mp-6),g++-mp-6, clang++))
endif

Going to ignore this and move on...

Terminal

Ben$ cd xgboost; cp make/config.mk ./config.mk; make -j4
[Tons of log output. Appears to finish succesfully]
...
g++-6 -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude   -Idmlc-core/include -Irabit/include -fPIC -fopenmp -shared -o lib/libxgboost.so build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm  -fopenmp 

步骤4(指令在这里

如果您想使用最新的xgboost版本并且已经编译了xgboost,请使用library(devtools); install('xgboost/R-package')手动安装xgboost包(根据您编译xgboost的位置更改路径)。

新的R会话

library(devtools)
install("/Users/Ben/xgboost/R-package")

Installing xgboost
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  '/Users/Ben/xgboost/R-package' --library='/Library/Frameworks/R.framework/Versions/3.3/Resources/library' --install-tests 

* installing *source* package ‘xgboost’ ...
** libs
g++-5 -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../..//include -I../..//dmlc-core/include -I../..//rabit/include -I../../ -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=1 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_ -I/usr/local/opt/llvm/include   -fopenmp  -fPIC  -Wall -mtune=core2 -g -O2 -c xgboost_R.cc -o xgboost_R.o
make: g++-5: No such file or directory
make: *** [xgboost_R.o] Error 1
ERROR: compilation failed for package ‘xgboost’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
Error: Command failed (1)

失败 :( 我做错了什么?

2个回答

5

更新的解决方案

截至2019年3月,R版本为3.5.3,xgboost版本为0.82.0.1,与我之前的答案不同,以下步骤基于此安装指南,但略有不同。

  1. brew install cmakebrew upgrade cmakecmake --version 显示3.14.0)
  2. brew install gccbrew upgrade gcc(注意安装的gcc版本。它应该位于/usr/local/bin下。我得到的gcc 8.3.0 (gcc-8 --version))
  3. cd some/unimportant/directory
  4. git clone --recursive https://github.com/dmlc/xgboost
  5. cd xgboost/
  6. mkdir build
  7. cd build
  8. CC=gcc-8 CXX=g++-8 cmake .. -DR_LIB=ON
  9. make -j4
  10. sudo make install

现在重新启动/刷新RStudio,它就应该被安装了。

在R中测试

set.seed(222)
N <- 2*10^5
p <- 350
x <- matrix(rnorm(N  * p), ncol = p)
y <- rnorm(N)

system.time(mymodel <- xgboost(
  # nthread = 4,
  data = x,
  label = y, 
  nrounds = 5, 
  objective = "reg:linear", 
  tree_method = "exact",
  max_depth = 10,
  min_child_weight = 1, 
  eta = 1, 
  subsample = 0.66, 
  colsample_bytree = 0.33
))

# Tested on 2018 MPB, xgboost version 0.82.0.1, multi-threaded version
# nthread = default:   7.4 seconds (elapsed)
# nthread = 1:        24.0 seconds (elapsed)
# nthread = 2:        13.7 seconds (elapsed)
# nthread = 4:         7.5 seconds (elapsed)

旧答案

令人惊讶的是,将问题写在StackOverflow上往往直接导向答案。在花费了数小时后,我发现需要在~/.R/Makevars中更改这三行:

CC=gcc-5
CXX=g++-5
CXX1X = g++-5

to

CC=gcc-6
CXX=g++-6
CXX1X = g++-6

同时,我最终从“drat”仓库安装了xgboost

install.packages("drat", repos="https://cran.rstudio.com")
drat:::addRepo("dmlc")
install.packages("xgboost", repos="http://dmlc.ml/drat/", type = "source")

谢谢 - 但是我还需要另外的添加:在我的SLES机器上,我还需要在Makevars中添加CPICFLAGS=-fPICCXXPICFLAGS=-fPICCXX1PICFLAGS=-fPIC - Valentin

1
我的方法是在你完成第一步(更新gcc)和第二步(克隆)之后,不需要构建任何东西(它们是用于其他语言,如Python的)。
3)更新~/.R/Makevars

https://github.com/dmlc/xgboost/issues/1136

如何在OS X中使用OpenMP编译使R软件包xgboost并行运行?

4) 在克隆的xgboost/R-package/configure文件中将ac_pkg_openmp修改为yes https://github.com/dmlc/xgboost/issues/2503

5) 在xgboost/R-package目录下输入R CMD INSTALL . http://xgboost.readthedocs.io/en/latest/build.html#r-package-installation


网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接