缓冲区大小如何影响NIO通道性能？

Question

我正在阅读Hadoop IPC实现。 https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

/**
 * When the read or write buffer size is larger than this limit, i/o will be 
 * done in chunks of this size. Most RPC requests and responses would be
 * be smaller.
 */
private static int NIO_BUFFER_LIMIT = 8*1024; //should not be more than 64KB.

/**
 * This is a wrapper around {@link WritableByteChannel#write(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * buffer increases. This also minimizes extra copies in NIO layer
 * as a result of multiple write operations required to write a large 
 * buffer.  
 *
 * @see WritableByteChannel#write(ByteBuffer)
 */
private int channelWrite(WritableByteChannel channel, 
                         ByteBuffer buffer) throws IOException {

  int count =  (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
               channel.write(buffer) : channelIO(null, channel, buffer);
  if (count > 0) {
    rpcMetrics.incrSentBytes(count);
  }
  return count;
}


/**
 * This is a wrapper around {@link ReadableByteChannel#read(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * ByteBuffer increases. There should not be any performance degredation.
 * 
 * @see ReadableByteChannel#read(ByteBuffer)
 */
private int channelRead(ReadableByteChannel channel, 
                        ByteBuffer buffer) throws IOException {

  int count = (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
              channel.read(buffer) : channelIO(channel, null, buffer);
  if (count > 0) {
    rpcMetrics.incrReceivedBytes(count);
  }
  return count;
}

逻辑是，如果缓冲区很小，它会读/写一次通道。如果缓冲区很大，它将多次执行，并且每次读/写8kb。

我不理解javadocs以及为什么要这样做。为什么“这是为了避免jdk随着缓冲区大小的增加创建许多直接缓冲区”？大缓冲区大小是否也会影响读取性能？

我理解缓冲区大小如何影响FileInputStream性能（link）。但这里是SocketChannel。所以它无关。

- waltersu

Answer 1

好问题。sun.nio.ch.IOUtil在通道写入时使用，其write(..)函数中包含以下行：

int var7 = var5 <= var6?var6 - var5:0;
ByteBuffer var8 = Util.getTemporaryDirectBuffer(var7);

这里是Util.getTemporaryDirectBuffer。

static ByteBuffer getTemporaryDirectBuffer(int var0) {
    Util.BufferCache var1 = (Util.BufferCache)bufferCache.get();
    ByteBuffer var2 = var1.get(var0);
    if(var2 != null) {
        return var2;
    } else {
        if(!var1.isEmpty()) {
            var2 = var1.removeFirst();
            free(var2);
        }

        return ByteBuffer.allocateDirect(var0);
    }
}

在负载较重且int var0范围较大时，会创建大量新缓冲区并free(..)旧缓冲区。由于bufferCache长度有限（等于系统配置中定义的IOUtil.IOV_MAX。在现代Linux系统上，限制为1024），因此不会存储每个长度的缓冲区。
我认为这是为了避免jdk在缓冲区大小增加时创建许多直接缓冲区。

- dezhik