-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathtest-run.txt
666 lines (661 loc) · 46.7 KB
/
test-run.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
A. Set short path of JAVA_HOME in hadoop-3.0.0\etc\hadoop\hadoop-env.cmd
set JAVA_HOME=C:\Progra~1\Java\jdk1.8.0_152
B. Test the installation:
C:\hadoop\hadoop3\hadoop-3.0.0>mkdir input
C:\hadoop\hadoop3\hadoop-3.0.0>cp etc/hadoop/*.xml input
C:\hadoop\hadoop3\hadoop-3.0.0>bin\hadoop jar share\hadoop\mapreduce\hadoop-mapreduce-examples-3.0.0.jar grep input output 'dfs[a-z.]+'
2017-12-15 09:07:21,932 INFO beanutils.FluentPropertyBeanIntrospector: Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
2017-12-15 09:07:22,011 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2017-12-15 09:07:22,786 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2017-12-15 09:07:22,786 INFO impl.MetricsSystemImpl: JobTracker metrics system started
2017-12-15 09:07:24,265 INFO input.FileInputFormat: Total input files to process : 9
2017-12-15 09:07:24,355 INFO mapreduce.JobSubmitter: number of splits:9
2017-12-15 09:07:25,509 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1212895820_0001
2017-12-15 09:07:25,512 INFO mapreduce.JobSubmitter: Executing with tokens: []
2017-12-15 09:07:25,807 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
2017-12-15 09:07:25,814 INFO mapred.LocalJobRunner: OutputCommitter set in config null
2017-12-15 09:07:25,834 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:25,834 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:25,838 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-12-15 09:07:25,817 INFO mapreduce.Job: Running job: job_local1212895820_0001
2017-12-15 09:07:25,922 INFO mapred.LocalJobRunner: Waiting for map tasks
2017-12-15 09:07:25,931 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000000_0
2017-12-15 09:07:25,979 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:25,982 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:26,006 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:26,108 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@4e7f4e2
2017-12-15 09:07:26,121 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/hadoop-policy.xml:0+10206
2017-12-15 09:07:26,281 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:26,284 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:26,285 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:26,286 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:26,287 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:26,296 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:26,354 INFO mapred.LocalJobRunner:
2017-12-15 09:07:26,356 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:26,457 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000000_0 is done. And is in the process of committing
2017-12-15 09:07:26,464 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:26,465 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000000_0' done.
2017-12-15 09:07:26,481 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000000_0: Counters: 18
File System Counters
FILE: Number of bytes read=327508
FILE: Number of bytes written=801018
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=237
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=125
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=0
Total committed heap usage (bytes)=223870976
File Input Format Counters
Bytes Read=10206
2017-12-15 09:07:26,481 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000000_0
2017-12-15 09:07:26,489 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000001_0
2017-12-15 09:07:26,491 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:26,491 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:26,493 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:26,605 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@8e22a45
2017-12-15 09:07:26,613 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/capacity-scheduler.xml:0+7861
2017-12-15 09:07:26,714 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:26,716 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:26,718 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:26,718 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:26,718 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:26,723 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:26,742 INFO mapred.LocalJobRunner:
2017-12-15 09:07:26,751 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:26,782 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000001_0 is done. And is in the process of committing
2017-12-15 09:07:26,786 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:26,786 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000001_0' done.
2017-12-15 09:07:26,787 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000001_0: Counters: 18
File System Counters
FILE: Number of bytes read=336500
FILE: Number of bytes written=801056
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=208
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=130
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=0
Total committed heap usage (bytes)=329252864
File Input Format Counters
Bytes Read=7861
2017-12-15 09:07:26,788 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000001_0
2017-12-15 09:07:26,788 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000002_0
2017-12-15 09:07:26,790 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:26,790 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:26,790 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:26,840 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@192b5f21
2017-12-15 09:07:26,870 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/kms-acls.xml:0+3518
2017-12-15 09:07:26,929 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:26,930 INFO mapreduce.Job: Job job_local1212895820_0001 running in uber mode : false
2017-12-15 09:07:26,932 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:26,933 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:26,934 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:26,934 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:27,096 INFO mapreduce.Job: map 100% reduce 0%
2017-12-15 09:07:27,099 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:27,103 INFO mapred.LocalJobRunner:
2017-12-15 09:07:27,104 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:27,147 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000002_0 is done. And is in the process of committing
2017-12-15 09:07:27,149 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:27,149 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000002_0' done.
2017-12-15 09:07:27,151 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000002_0: Counters: 18
File System Counters
FILE: Number of bytes read=341149
FILE: Number of bytes written=801094
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=135
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=120
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=180
Total committed heap usage (bytes)=282066944
File Input Format Counters
Bytes Read=3518
2017-12-15 09:07:27,151 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000002_0
2017-12-15 09:07:27,151 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000003_0
2017-12-15 09:07:27,153 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:27,153 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:27,153 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:27,191 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@7249073c
2017-12-15 09:07:27,193 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/hdfs-site.xml:0+775
2017-12-15 09:07:27,228 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:27,305 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:27,305 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:27,306 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:27,306 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:27,307 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:27,309 INFO mapred.LocalJobRunner:
2017-12-15 09:07:27,309 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:27,335 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000003_0 is done. And is in the process of committing
2017-12-15 09:07:27,340 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:27,340 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000003_0' done.
2017-12-15 09:07:27,341 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000003_0: Counters: 18
File System Counters
FILE: Number of bytes read=343055
FILE: Number of bytes written=801132
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=21
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=121
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=0
Total committed heap usage (bytes)=387448832
File Input Format Counters
Bytes Read=775
2017-12-15 09:07:27,341 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000003_0
2017-12-15 09:07:27,341 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000004_0
2017-12-15 09:07:27,343 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:27,343 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:27,343 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:27,392 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@7f70cae1
2017-12-15 09:07:27,394 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/core-site.xml:0+774
2017-12-15 09:07:27,445 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:27,501 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:27,501 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:27,501 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:27,501 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:27,502 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:27,505 INFO mapred.LocalJobRunner:
2017-12-15 09:07:27,505 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:27,540 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000004_0 is done. And is in the process of committing
2017-12-15 09:07:27,542 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:27,542 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000004_0' done.
2017-12-15 09:07:27,543 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000004_0: Counters: 18
File System Counters
FILE: Number of bytes read=344960
FILE: Number of bytes written=801170
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=20
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=121
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=0
Total committed heap usage (bytes)=471859200
File Input Format Counters
Bytes Read=774
2017-12-15 09:07:27,543 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000004_0
2017-12-15 09:07:27,543 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000005_0
2017-12-15 09:07:27,544 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:27,545 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:27,545 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:27,587 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@24d1da4
2017-12-15 09:07:27,635 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/mapred-site.xml:0+758
2017-12-15 09:07:27,677 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:27,684 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:27,684 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:27,684 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:27,684 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:27,688 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:27,692 INFO mapred.LocalJobRunner:
2017-12-15 09:07:27,696 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:27,723 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000005_0 is done. And is in the process of committing
2017-12-15 09:07:27,727 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:27,728 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000005_0' done.
2017-12-15 09:07:27,729 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000005_0: Counters: 18
File System Counters
FILE: Number of bytes read=346337
FILE: Number of bytes written=801208
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=21
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=123
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=20
Total committed heap usage (bytes)=487063552
File Input Format Counters
Bytes Read=758
2017-12-15 09:07:27,731 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000005_0
2017-12-15 09:07:27,731 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000006_0
2017-12-15 09:07:27,732 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:27,732 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:27,741 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:27,782 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@195cacf8
2017-12-15 09:07:27,787 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/yarn-site.xml:0+690
2017-12-15 09:07:27,831 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:27,858 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:27,858 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:27,858 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:27,858 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:27,859 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:27,862 INFO mapred.LocalJobRunner:
2017-12-15 09:07:27,865 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:27,903 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000006_0 is done. And is in the process of committing
2017-12-15 09:07:27,907 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:27,907 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000006_0' done.
2017-12-15 09:07:27,909 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000006_0: Counters: 18
File System Counters
FILE: Number of bytes read=347646
FILE: Number of bytes written=801246
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=19
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=121
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=23
Total committed heap usage (bytes)=525336576
File Input Format Counters
Bytes Read=690
2017-12-15 09:07:27,909 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000006_0
2017-12-15 09:07:27,910 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000007_0
2017-12-15 09:07:27,911 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:27,912 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:27,912 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:28,010 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@553b12e
2017-12-15 09:07:28,012 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/kms-site.xml:0+682
2017-12-15 09:07:28,027 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:28,027 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:28,027 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:28,027 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:28,027 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:28,028 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:28,030 INFO mapred.LocalJobRunner:
2017-12-15 09:07:28,032 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:28,063 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000007_0 is done. And is in the process of committing
2017-12-15 09:07:28,064 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:28,065 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000007_0' done.
2017-12-15 09:07:28,066 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000007_0: Counters: 18
File System Counters
FILE: Number of bytes read=348947
FILE: Number of bytes written=801284
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=20
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=120
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=4
Total committed heap usage (bytes)=525860864
File Input Format Counters
Bytes Read=682
2017-12-15 09:07:28,067 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000007_0
2017-12-15 09:07:28,067 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_m_000008_0
2017-12-15 09:07:28,070 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:28,071 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:28,075 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:28,129 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@12d6fd28
2017-12-15 09:07:28,131 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/input/httpfs-site.xml:0+620
2017-12-15 09:07:28,218 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:28,218 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:28,220 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:28,220 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:28,220 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:28,221 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:28,223 INFO mapred.LocalJobRunner:
2017-12-15 09:07:28,223 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:28,281 INFO mapred.Task: Task:attempt_local1212895820_0001_m_000008_0 is done. And is in the process of committing
2017-12-15 09:07:28,285 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:28,285 INFO mapred.Task: Task 'attempt_local1212895820_0001_m_000008_0' done.
2017-12-15 09:07:28,286 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_m_000008_0: Counters: 18
File System Counters
FILE: Number of bytes read=350186
FILE: Number of bytes written=801322
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=17
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=123
Combine input records=0
Combine output records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=15
Total committed heap usage (bytes)=524812288
File Input Format Counters
Bytes Read=620
2017-12-15 09:07:28,286 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_m_000008_0
2017-12-15 09:07:28,287 INFO mapred.LocalJobRunner: map task executor complete.
2017-12-15 09:07:28,296 INFO mapred.LocalJobRunner: Waiting for reduce tasks
2017-12-15 09:07:28,298 INFO mapred.LocalJobRunner: Starting task: attempt_local1212895820_0001_r_000000_0
2017-12-15 09:07:28,341 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:28,341 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:28,347 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:28,476 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@6bb08379
2017-12-15 09:07:28,482 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@69c96565
2017-12-15 09:07:28,484 WARN impl.MetricsSystemImpl: JobTracker metrics system already initialized!
2017-12-15 09:07:28,509 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=367368608, maxSingleShuffleLimit=91842152, mergeThreshold=242463296, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-12-15 09:07:28,513 INFO reduce.EventFetcher: attempt_local1212895820_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-12-15 09:07:28,594 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000007_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,601 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000007_0
2017-12-15 09:07:28,604 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->2
2017-12-15 09:07:28,622 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000004_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,624 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000004_0
2017-12-15 09:07:28,624 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 2, commitMemory -> 2, usedMemory ->4
2017-12-15 09:07:28,635 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000001_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,636 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000001_0
2017-12-15 09:07:28,636 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 3, commitMemory -> 4, usedMemory ->6
2017-12-15 09:07:28,653 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000008_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,654 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000008_0
2017-12-15 09:07:28,655 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 4, commitMemory -> 6, usedMemory ->8
2017-12-15 09:07:28,684 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000005_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,685 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000005_0
2017-12-15 09:07:28,687 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 5, commitMemory -> 8, usedMemory ->10
2017-12-15 09:07:28,695 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000002_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,696 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000002_0
2017-12-15 09:07:28,696 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 6, commitMemory -> 10, usedMemory ->12
2017-12-15 09:07:28,731 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000006_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,741 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000006_0
2017-12-15 09:07:28,745 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 7, commitMemory -> 12, usedMemory ->14
2017-12-15 09:07:28,754 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000003_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,759 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000003_0
2017-12-15 09:07:28,759 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 8, commitMemory -> 14, usedMemory ->16
2017-12-15 09:07:28,805 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1212895820_0001_m_000000_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:28,808 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1212895820_0001_m_000000_0
2017-12-15 09:07:28,809 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 9, commitMemory -> 16, usedMemory ->18
2017-12-15 09:07:28,810 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-12-15 09:07:28,811 INFO mapred.LocalJobRunner: 9 / 9 copied.
2017-12-15 09:07:28,812 INFO reduce.MergeManagerImpl: finalMerge called with 9 in-memory map-outputs and 0 on-disk map-outputs
2017-12-15 09:07:28,842 INFO mapred.Merger: Merging 9 sorted segments
2017-12-15 09:07:28,845 INFO mapred.Merger: Down to the last merge-pass, with 0 segments left of total size: 0 bytes
2017-12-15 09:07:28,850 INFO reduce.MergeManagerImpl: Merged 9 segments, 18 bytes to disk to satisfy reduce memory limit
2017-12-15 09:07:28,851 INFO reduce.MergeManagerImpl: Merging 1 files, 6 bytes from disk
2017-12-15 09:07:28,852 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-12-15 09:07:28,853 INFO mapred.Merger: Merging 1 sorted segments
2017-12-15 09:07:28,855 INFO mapred.Merger: Down to the last merge-pass, with 0 segments left of total size: 0 bytes
2017-12-15 09:07:28,857 INFO mapred.LocalJobRunner: 9 / 9 copied.
2017-12-15 09:07:28,926 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2017-12-15 09:07:28,961 INFO mapred.Task: Task:attempt_local1212895820_0001_r_000000_0 is done. And is in the process of committing
2017-12-15 09:07:28,965 INFO mapred.LocalJobRunner: 9 / 9 copied.
2017-12-15 09:07:28,965 INFO mapred.Task: Task attempt_local1212895820_0001_r_000000_0 is allowed to commit now
2017-12-15 09:07:28,971 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1212895820_0001_r_000000_0' to file:/C:/hadoop/hadoop3/hadoop-3.0.0/grep-temp-361139918
2017-12-15 09:07:28,974 INFO mapred.LocalJobRunner: reduce > reduce
2017-12-15 09:07:28,974 INFO mapred.Task: Task 'attempt_local1212895820_0001_r_000000_0' done.
2017-12-15 09:07:28,979 INFO mapred.Task: Final Counters for attempt_local1212895820_0001_r_000000_0: Counters: 24
File System Counters
FILE: Number of bytes read=350534
FILE: Number of bytes written=801426
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Combine input records=0
Combine output records=0
Reduce input groups=0
Reduce shuffle bytes=54
Reduce input records=0
Reduce output records=0
Spilled Records=0
Shuffled Maps =9
Failed Shuffles=0
Merged Map outputs=9
GC time elapsed (ms)=0
Total committed heap usage (bytes)=524812288
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Output Format Counters
Bytes Written=98
2017-12-15 09:07:28,979 INFO mapred.LocalJobRunner: Finishing task: attempt_local1212895820_0001_r_000000_0
2017-12-15 09:07:28,980 INFO mapred.LocalJobRunner: reduce task executor complete.
2017-12-15 09:07:29,134 INFO mapreduce.Job: map 100% reduce 100%
2017-12-15 09:07:29,137 INFO mapreduce.Job: Job job_local1212895820_0001 completed successfully
2017-12-15 09:07:29,170 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=3436822
FILE: Number of bytes written=8011956
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=698
Map output records=0
Map output bytes=0
Map output materialized bytes=54
Input split bytes=1104
Combine input records=0
Combine output records=0
Reduce input groups=0
Reduce shuffle bytes=54
Reduce input records=0
Reduce output records=0
Spilled Records=0
Shuffled Maps =9
Failed Shuffles=0
Merged Map outputs=9
GC time elapsed (ms)=242
Total committed heap usage (bytes)=4282384384
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=25884
File Output Format Counters
Bytes Written=98
2017-12-15 09:07:29,236 WARN impl.MetricsSystemImpl: JobTracker metrics system already initialized!
2017-12-15 09:07:29,404 INFO input.FileInputFormat: Total input files to process : 1
2017-12-15 09:07:29,437 INFO mapreduce.JobSubmitter: number of splits:1
2017-12-15 09:07:29,494 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1327930606_0002
2017-12-15 09:07:29,496 INFO mapreduce.JobSubmitter: Executing with tokens: []
2017-12-15 09:07:29,660 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
2017-12-15 09:07:29,676 INFO mapreduce.Job: Running job: job_local1327930606_0002
2017-12-15 09:07:29,677 INFO mapred.LocalJobRunner: OutputCommitter set in config null
2017-12-15 09:07:29,678 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:29,678 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:29,678 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2017-12-15 09:07:29,683 INFO mapred.LocalJobRunner: Waiting for map tasks
2017-12-15 09:07:29,684 INFO mapred.LocalJobRunner: Starting task: attempt_local1327930606_0002_m_000000_0
2017-12-15 09:07:29,687 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:29,687 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:29,687 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:29,743 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@163f4c33
2017-12-15 09:07:29,746 INFO mapred.MapTask: Processing split: file:/C:/hadoop/hadoop3/hadoop-3.0.0/grep-temp-361139918/part-r-00000:0+86
2017-12-15 09:07:29,766 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2017-12-15 09:07:29,766 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
2017-12-15 09:07:29,766 INFO mapred.MapTask: soft limit at 83886080
2017-12-15 09:07:29,766 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
2017-12-15 09:07:29,767 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
2017-12-15 09:07:29,772 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2017-12-15 09:07:29,784 INFO mapred.LocalJobRunner:
2017-12-15 09:07:29,785 INFO mapred.MapTask: Starting flush of map output
2017-12-15 09:07:29,814 INFO mapred.Task: Task:attempt_local1327930606_0002_m_000000_0 is done. And is in the process of committing
2017-12-15 09:07:29,826 INFO mapred.LocalJobRunner: map
2017-12-15 09:07:29,826 INFO mapred.Task: Task 'attempt_local1327930606_0002_m_000000_0' done.
2017-12-15 09:07:29,830 INFO mapred.Task: Final Counters for attempt_local1327930606_0002_m_000000_0: Counters: 17
File System Counters
FILE: Number of bytes read=666819
FILE: Number of bytes written=1599995
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=0
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=134
Combine input records=0
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=0
Total committed heap usage (bytes)=525336576
File Input Format Counters
Bytes Read=98
2017-12-15 09:07:29,830 INFO mapred.LocalJobRunner: Finishing task: attempt_local1327930606_0002_m_000000_0
2017-12-15 09:07:29,830 INFO mapred.LocalJobRunner: map task executor complete.
2017-12-15 09:07:29,832 INFO mapred.LocalJobRunner: Waiting for reduce tasks
2017-12-15 09:07:29,832 INFO mapred.LocalJobRunner: Starting task: attempt_local1327930606_0002_r_000000_0
2017-12-15 09:07:29,834 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2
2017-12-15 09:07:29,834 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2017-12-15 09:07:29,835 INFO util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux.
2017-12-15 09:07:29,878 INFO mapred.Task: Using ResourceCalculatorProcessTree : org.apache.hadoop.yarn.util.WindowsBasedProcessTree@42cc682f
2017-12-15 09:07:29,879 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@46f3e4ae
2017-12-15 09:07:29,881 WARN impl.MetricsSystemImpl: JobTracker metrics system already initialized!
2017-12-15 09:07:29,883 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=367735584, maxSingleShuffleLimit=91933896, mergeThreshold=242705488, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2017-12-15 09:07:29,897 INFO reduce.EventFetcher: attempt_local1327930606_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2017-12-15 09:07:29,901 INFO reduce.LocalFetcher: localfetcher#2 about to shuffle output of map attempt_local1327930606_0002_m_000000_0 decomp: 2 len: 6 to MEMORY
2017-12-15 09:07:29,901 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1327930606_0002_m_000000_0
2017-12-15 09:07:29,902 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->2
2017-12-15 09:07:29,902 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning
2017-12-15 09:07:29,903 INFO mapred.LocalJobRunner: 1 / 1 copied.
2017-12-15 09:07:29,904 INFO reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2017-12-15 09:07:29,966 INFO mapred.Merger: Merging 1 sorted segments
2017-12-15 09:07:29,968 INFO mapred.Merger: Down to the last merge-pass, with 0 segments left of total size: 0 bytes
2017-12-15 09:07:29,972 INFO reduce.MergeManagerImpl: Merged 1 segments, 2 bytes to disk to satisfy reduce memory limit
2017-12-15 09:07:29,973 INFO reduce.MergeManagerImpl: Merging 1 files, 6 bytes from disk
2017-12-15 09:07:29,974 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
2017-12-15 09:07:29,974 INFO mapred.Merger: Merging 1 sorted segments
2017-12-15 09:07:29,975 INFO mapred.Merger: Down to the last merge-pass, with 0 segments left of total size: 0 bytes
2017-12-15 09:07:29,976 INFO mapred.LocalJobRunner: 1 / 1 copied.
2017-12-15 09:07:29,988 INFO mapred.Task: Task:attempt_local1327930606_0002_r_000000_0 is done. And is in the process of committing
2017-12-15 09:07:29,989 INFO mapred.LocalJobRunner: 1 / 1 copied.
2017-12-15 09:07:29,989 INFO mapred.Task: Task attempt_local1327930606_0002_r_000000_0 is allowed to commit now
2017-12-15 09:07:29,996 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1327930606_0002_r_000000_0' to file:/C:/hadoop/hadoop3/hadoop-3.0.0/output
2017-12-15 09:07:29,997 INFO mapred.LocalJobRunner: reduce > reduce
2017-12-15 09:07:29,997 INFO mapred.Task: Task 'attempt_local1327930606_0002_r_000000_0' done.
2017-12-15 09:07:29,997 INFO mapred.Task: Final Counters for attempt_local1327930606_0002_r_000000_0: Counters: 24
File System Counters
FILE: Number of bytes read=666863
FILE: Number of bytes written=1600009
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Combine input records=0
Combine output records=0
Reduce input groups=0
Reduce shuffle bytes=6
Reduce input records=0
Reduce output records=0
Spilled Records=0
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=0
Total committed heap usage (bytes)=525336576
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Output Format Counters
Bytes Written=8
2017-12-15 09:07:29,997 INFO mapred.LocalJobRunner: Finishing task: attempt_local1327930606_0002_r_000000_0
2017-12-15 09:07:29,998 INFO mapred.LocalJobRunner: reduce task executor complete.
2017-12-15 09:07:30,678 INFO mapreduce.Job: Job job_local1327930606_0002 running in uber mode : false
2017-12-15 09:07:30,678 INFO mapreduce.Job: map 100% reduce 100%
2017-12-15 09:07:30,680 INFO mapreduce.Job: Job job_local1327930606_0002 completed successfully
2017-12-15 09:07:30,682 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=1333682
FILE: Number of bytes written=3200004
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
Map-Reduce Framework
Map input records=0
Map output records=0
Map output bytes=0
Map output materialized bytes=6
Input split bytes=134
Combine input records=0
Combine output records=0
Reduce input groups=0
Reduce shuffle bytes=6
Reduce input records=0
Reduce output records=0
Spilled Records=0
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=0
Total committed heap usage (bytes)=1050673152
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=98
File Output Format Counters
Bytes Written=8
C:\hadoop\hadoop3\hadoop-3.0.0>