R核心中`split`函数背后的算法是什么?

8

split是R核心中尤其重要的函数。在数据处理方面,许多提供基于R基础解决方案的Stack Overflow答案都依赖它。任何分组操作的工作程序都是这个函数。

还有许多问题的解决方案只需要一行split。很多人不知道

  • split.data.frame可以按行拆分矩阵;
  • split.default可以按列拆分数据框。

也许关于split的R文档并不太好。它确实提到了第一个用途,但没有提到第二个。

R核心中split有四种方法:

methods(split)
#[1] split.data.frame split.Date       split.default    split.POSIXct

我会提供一份深入解释split.data.framesplit.default和C级别的.Internal(split(x, f))如何工作的答案。欢迎其他回答涉及“Date”和“POSIXct”对象。

1个回答

9

split.data.frame 是如何工作的?

function (x, f, drop = FALSE, ...) 
lapply(split(x = seq_len(nrow(x)), f = f, drop = drop, ...), 
       function(ind) x[ind, , drop = FALSE])

它调用split.default函数来分割行索引向量seq_len(nrow(x)),然后使用lapply循环将相关行提取到列表条目中。

这并不严格是一个“data.frame”的方法。它可以通过第一维度来分割任何二维对象,包括按行分割矩阵


split.default如何工作?

function (x, f, drop = FALSE, sep = ".", lex.order = FALSE, ...) 
{
if (!missing(...)) 
    .NotYetUsed(deparse(...), error = FALSE)
if (is.list(f)) 
    f <- interaction(f, drop = drop, sep = sep, lex.order = lex.order)
else if (!is.factor(f)) 
    f <- as.factor(f)
else if (drop) 
    f <- factor(f)
storage.mode(f) <- "integer"
if (is.null(attr(x, "class"))) 
    return(.Internal(split(x, f)))
lf <- levels(f)
y <- vector("list", length(lf))
names(y) <- lf
ind <- .Internal(split(seq_along(x), f))
for (k in lf) y[[k]] <- x[ind[[k]]]
y
}
  1. 如果x没有类别(主要是一个原子向量),将使用.Internal(split(x, f))
  2. 否则,它使用.Internal(split())来沿着x分割索引,然后使用一个for循环将相关元素提取到列表条目中。

原子向量(参见?vector)是具有以下模式的向量:

  • “logical”、“integer”、“numeric”、“complex”、“character”和“raw”
  • “list”
  • “expression”

带有类别的对象...额...有这么多!让我举三个例子:

  • “factor”
  • “data.frame”
  • “matrix”

在我看来,split.default写得不太好。有那么多带类别的对象,但是split.default会通过"["以相同方式处理它们。这对于“factor”和“data.frame”很好用(所以我们将沿着列拆分数据框!),但它绝对不适用于矩阵,因为我们期望它会按照另一种方式工作。

A <- matrix(1:9, 3)
#     [,1] [,2] [,3]
#[1,]    1    4    7
#[2,]    2    5    8
#[3,]    3    6    9

split.default(A, c(1, 1, 2))  ## it does not split the matrix by columns!
#$`1`
#[1] 1 2 4 5 7 8
#
#$`2`
#[1] 3 6 9

实际上,已经将回收规则应用于c(1, 1, 2),我们等效地执行如下操作:

split(c(A), rep_len(c(1,1,2), length(A)))

为什么R核心不写另一行代码来创建一个"matrix",例如:
for (k in lf) y[[k]] <- x[, ind[[k]], drop = FALSE]

到目前为止,安全地按列拆分矩阵的唯一方法是先转置它,然后使用 split.data.frame 函数,再转置回去。
lapply(split.data.frame(t(A), c(1, 1, 2)), t)

通过 lapply(split.default(data.frame(A), c(1, 1, 2)), as.matrix) 暂时可以解决问题。但如果 A 是字符矩阵,这种方法就会出现错误。


.Internal(split(x, f)) 的工作原理是什么?

这真的是核心中的核心!下面我将以一个小例子来解释:

set.seed(0)
f <- sample(factor(letters[1:3]), 10, TRUE)
# [1] c a b b c a c c b b
#Levels: a b c

x <- 0:9

基本上有三个步骤。为了提高可读性,每个步骤都提供了等效的R代码。

步骤1:制表(计算每个因子水平的出现次数)

## a factor has integer mode so `tabulate` works
tab <- tabulate(f, nbins = nlevels(f))
[1] 2 4 4

步骤2:分配结果列表的存储空间

result <- vector("list", nlevels(f))
for (i in 1:length(tab)) result[[i]] <- vector(mode(x), tab[i])
names(result) <- levels(f)

我会将这个列表进行注释,其中每行都是一个向量(在此示例中),每个[ ]都代表该向量的一个条目。
$a: [ ] [ ]

$b: [ ] [ ] [ ] [ ]

$c: [ ] [ ] [ ] [ ]

步骤三:元素分配

现在揭示因子的内部整数模式非常有用:

.f <- as.integer(f)
#[1] 3 1 2 2 3 1 3 3 2 2

我们需要扫描x.f,将x[i]填充到result[[.f[i]]]正确条目中,参考一个累加器缓冲向量。

ab <- integer(nlevels(f))  ## accumulator buffer

for (i in 1:length(.f)) {
  fi <- .f[i] 
  counter <- ab[fi] + 1L
  result[[fi]][counter] <- x[i]
  ab[fi] <- counter
  }

在下面的图示中,^ 是指向被访问或更新的元素的指针。
## i = 1

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
     ^

ab: [0] [0] [0]  ## on entry
             ^

$a: [ ] [ ]

$b: [ ] [ ] [ ] [ ]

$c: [0] [ ] [ ] [ ]
     ^

ab: [0] [0] [1]  ## on exit
             ^

## i = 2

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
         ^

ab: [0] [0] [1]  ## on entry
     ^

$a: [1] [ ]
     ^
$b: [ ] [ ] [ ] [ ]

$c: [0] [ ] [ ] [ ]

ab: [1] [0] [1]  ## on exit
     ^

## i = 3

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
             ^

ab: [1] [0] [1]  ## on entry
         ^

$a: [1] [ ]

$b: [2] [ ] [ ] [ ]
     ^
$c: [0] [ ] [ ] [ ]

ab: [1] [1] [1]  ## on exit
         ^

## i = 4

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
                 ^

ab: [1] [1] [1]  ## on entry
         ^

$a: [1] [ ]

$b: [2] [3] [ ] [ ]
         ^
$c: [0] [ ] [ ] [ ]

ab: [1] [2] [1]  ## on exit
         ^

## i = 5

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
                     ^

ab: [1] [2] [1]  ## on entry
             ^

$a: [1] [ ]

$b: [2] [3] [ ] [ ]

$c: [0] [4] [ ] [ ]
         ^

ab: [1] [2] [2]  ## on exit
             ^

## i = 6

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
                         ^

ab: [1] [2] [2]  ## on entry
     ^

$a: [1] [5]
         ^
$b: [2] [3] [ ] [ ]

$c: [0] [4] [ ] [ ]

ab: [2] [2] [2]  ## on exit
     ^

## i = 7

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
                             ^

ab: [2] [2] [2]  ## on entry
             ^

$a: [1] [5]

$b: [2] [3] [ ] [ ]

$c: [0] [4] [6] [ ]
             ^

ab: [2] [2] [3]  ## on exit
             ^

## i = 8

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
                                 ^

ab: [2] [2] [3]  ## on entry
             ^

$a: [1] [5]

$b: [2] [3] [ ] [ ]

$c: [0] [4] [6] [7]
                 ^

ab: [2] [2] [4]  ## on exit
             ^

## i = 9

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
                                     ^

ab: [2] [2] [4]  ## on entry
         ^

$a: [1] [5]

$b: [2] [3] [8] [ ]
             ^
$c: [0] [4] [6] [7]

ab: [2] [3] [4]  ## on exit
         ^

## i = 10

 x: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
.f: [3] [1] [2] [2] [3] [1] [3] [3] [2] [2]
                                         ^

ab: [2] [3] [4]  ## on entry
         ^

$a: [1] [5]

$b: [2] [3] [8] [9]
                 ^
$c: [0] [4] [6] [7]

ab: [2] [4] [4]  ## on exit
         ^

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接