Hadoop实战-PHP-MapReduce

admin 发布于:2018-6-8 8:51 分类:系统架构  有 1754 人浏览,获得评论 0 条 标签: Hadoop MapReduce 

1. 编写Mapper的代码

#vi WordMap.php

#!/usr/bin/php
<?php
while (($line = fgets(STDIN)) !== false) 
{  
   $words = preg_split('/(\s+)/', $line);    
   foreach ($words as $word) 
   {  
       echo $word."\t"."1".PHP_EOL;  
   }  
}  
?> 

 


2.编写Reducer的代码
#vi WordReduce.php

 

#!/usr/bin/php
<?php
$result=array();
while (($line = fgets(STDIN)) !== false) 
{  
    $line = trim($line);
 list($k, $v) = explode("\t", $line);
 $result[$k] += $v; 
}  
ksort($result);
foreach($result as $word => $count)
{
 echo $word."\t".$count.PHP_EOL;  
}
?>

 

 

 

3.运行WordMapReduce

#chmod 0777 WordMap.php
#chmod 0777 WordReduce.php
#bin/hadoop  jar share/hadoop/tools/lib/hadoop-streaming-2.9.1.jar  -mapper WordMap.php -reducer WordReduce.php  -input HdfsInput/* -output HdfsOutput 


4.查看运行结果

#hadoop fs -ls HdfsOutput
#hadoop fs -cat HdfsOutput/*
#hadoop fs -get HdfsOutput LocalOutput
#cat LocalOutput/*

 

上一篇:hive安装
下一篇:Hadoop实战-MapReduce