Hbase ipc.RpcServer 响应过慢问题导致区域服务器崩溃。

Question

我已经配置了 Hbase 1.2.6 和 Hadoop 2.7.4。我在其中配置了 Nutch。 以下是两个 Hbase-site 属性：

<property>
  <name>hbase.rpc.timeout</name>
  <value>1200000</value>
  <description>Changed to 20 minutes</description>
</property>

<property>
  <name>hbase.client.scanner.timeout.period</name>
  <value>1200000</value>
  <description>Changed to 10 minutes</description>
</property>

当我开始一个任务时，一段时间后就会出现以下警告并导致区域服务器宕机。这就是任务失败的原因。

2017-12-22 17:39:25,660 WARN  [B.defaultRpcServer.handler=18,queue=0,port=16020] ipc.RpcServer: (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1513946351644,"responsesize":8238,"method":"Scan","param":"scanner_id: 65 number_of_rows: 2147483647 close_scanner: false next_call_seq: 5 client_handles_partials: true client_handles_heartbeats: true track_scan_metrics: false renew: false","processingtimems":14015,"client":"10.11.21.42:36388","queuetimems":0,"class":"HRegionServer","scandetails":"table: use_webpage region: use_webpage,com.blogsky.shayan-pix:http/category/cat-2,1513328565609.09e0aff4d1e1d0e5733825c90f289be5."}
2017-12-22 17:39:37,626 WARN  [B.defaultRpcServer.handler=24,queue=0,port=16020] ipc.RpcServer: (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1513946357681,"responsesize":8654,"method":"Scan","param":"scanner_id: 64 number_of_rows: 2147483647 close_scanner: false next_call_seq: 5 client_handles_partials: true client_handles_heartbeats: true track_scan_metrics: false renew: false","processingtimems":19945,"client":"10.11.21.42:56252","queuetimems":0,"class":"HRegionServer","scandetails":"table: use_webpage region: use_webpage,ir.persianblog.gae6:http/,1509943454203.6992fa37fa94574aa4797b9ae64a296b."}

- Hafiz Muhammad Shafiq

Answer 1

注意该区域的局部性。如果局部性较低，请对该特定区域进行区域压缩。如果这样无法解决问题，请对该表运行Major Compaction。你会发现几乎所有区域的局部性都变为1。这对我很有用。

- Karthick Ramachandran