Hadoop计数器文档？

Question

在我的MapReduce作业完成后，我得到了大量的Counter信息：

File System Counters
                FILE: Number of bytes read=4386096368
                FILE: Number of bytes written=8805370803
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=54583718086
                HDFS: Number of bytes written=4382090874
                HDFS: Number of read operations=1479
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters
                Launched map tasks=369
                Launched reduce tasks=1
                Data-local map tasks=369
                Total time spent by all maps in occupied slots (ms)=34288552
                Total time spent by all reduces in occupied slots (ms)=232084
                Total time spent by all map tasks (ms)=8572138
                Total time spent by all reduce tasks (ms)=58021
                Total vcore-seconds taken by all map tasks=8572138
                Total vcore-seconds taken by all reduce tasks=58021
                Total megabyte-seconds taken by all map tasks=35111477248
                Total megabyte-seconds taken by all reduce tasks=237654016
        Map-Reduce Framework
                Map input records=14753874
                Map output records=666776
                Map output bytes=4383426830
                Map output materialized bytes=4386098552
                Input split bytes=47970
                Combine input records=0
                Combine output records=0
                Reduce input groups=1
                Reduce shuffle bytes=4386098552
                Reduce input records=666776
                Reduce output records=666776
                Spilled Records=1333552
                Shuffled Maps =369
                Failed Shuffles=0
                Merged Map outputs=369
                GC time elapsed (ms)=1121584
                CPU time spent (ms)=23707900
                Physical memory (bytes) snapshot=152915259392
                Virtual memory (bytes) snapshot=2370755190784
                Total committed heap usage (bytes)=126644912128
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=49449743227
        File Output Format Counters
                Bytes Written=4382090874

哪里可以找到每个字段的解释？其中某些字段很明显（Number of bytes read），但其他字段可能更加模糊（Total time spent by all maps in occupied slots与Total time spent by all map tasks）。我找到了默认计数器列表，但似乎找不到它们的解释或说明。我很惊讶我无法轻松找到有关此输出的文档。是否有人能提供链接或解释？

- Ryan Marcus

---

请查看此链接以获取有关此内容的一些信息：https://dev59.com/QYLba4cB1Zd3GeqPjMn8 - AST

这些计数器的解释可以在Tom White所著的《Hadoop权威指南》第三版的第8章（Map Reduce Features）中找到。希望这能有所帮助。 - Raju

Answer 1

Hadoop权威指南的第8章（华盛顿州立大学提供的链接中包含完整PDF），详细介绍了与MapReduce有关的计数器。这从第225页开始，并列在表8-1中。该资源的更新版本（第4版）可在Safari Books Online上获得（您需要先登录）。

- user6613600