Pentaho MapReduce - wordcount 0 / - 2010/07/19 21:35:45.843 - 2010/07/19 21:35:45.843 ID_JOBYID_JOBCHANNEL_IDYCHANNEL_IDJOBNAMEYJOBNAMESTATUSYSTATUSLINES_READYLINES_READLINES_WRITTENYLINES_WRITTENLINES_UPDATEDYLINES_UPDATEDLINES_INPUTYLINES_INPUTLINES_OUTPUTYLINES_OUTPUTLINES_REJECTEDYLINES_REJECTEDERRORSYERRORSSTARTDATEYSTARTDATEENDDATEYENDDATELOGDATEYLOGDATEDEPDATEYDEPDATEREPLAYDATEYREPLAYDATELOG_FIELDYLOG_FIELD ID_BATCHYID_BATCHCHANNEL_IDYCHANNEL_IDLOG_DATEYLOG_DATEJOBNAMEYTRANSNAMEJOBENTRYNAMEYSTEPNAMELINES_READYLINES_READLINES_WRITTENYLINES_WRITTENLINES_UPDATEDYLINES_UPDATEDLINES_INPUTYLINES_INPUTLINES_OUTPUTYLINES_OUTPUTLINES_REJECTEDYLINES_REJECTEDERRORSYERRORSRESULTYRESULTNR_RESULT_ROWSYNR_RESULT_ROWSNR_RESULT_FILESYNR_RESULT_FILESLOG_FIELDNLOG_FIELDCOPY_NRNCOPY_NR ID_BATCHYID_BATCHCHANNEL_IDYCHANNEL_IDLOG_DATEYLOG_DATELOGGING_OBJECT_TYPEYLOGGING_OBJECT_TYPEOBJECT_NAMEYOBJECT_NAMEOBJECT_COPYYOBJECT_COPYREPOSITORY_DIRECTORYYREPOSITORY_DIRECTORYFILENAMEYFILENAMEOBJECT_IDYOBJECT_IDOBJECT_REVISIONYOBJECT_REVISIONPARENT_CHANNEL_IDYPARENT_CHANNEL_IDROOT_CHANNEL_IDYROOT_CHANNEL_ID N Pentaho MapReduce (Wordcount) HadoopTransJobExecutorPlugin Pentaho MapReduce (Wordcount) ${Internal.Job.Filename.Directory}/wordcount-mapper.ktr ${Internal.Job.Filename.Directory}/wordcount-reducer.ktr Y Hadoop Input Hadoop Output Hadoop Input Hadoop Output Y 5 /wordcount/input org.apache.hadoop.mapred.TextInputFormat /wordcount/output Y N N N N org.apache.hadoop.mapred.TextOutputFormat hadoop-server 8020 hadoop-server 8021 2 1 /var/tmp N Y 0 299 158 START SPECIAL Y N N 0 0 60 12 0 1 1 N Y 0 19 158 Failure ABORT N Y 0 440 292 Success SUCCESS N Y 0 439 158 Copy Files to HDFS HadoopCopyFilesPlugin Y N N N N N N Y ./ hdfs://hadoop-server:8020/wordcount/input .*README.* N Y 0 159 158 Pentaho MapReduce (Wordcount) Success 0 0 Y Y N Pentaho MapReduce (Wordcount) Failure 0 0 Y N N Copy Files to HDFS Pentaho MapReduce (Wordcount) 0 0 Y Y Y START Copy Files to HDFS 0 0 Y Y Y SETUP INSTRUCTIONS: 1. Create an input directory in HDFS and place text file(s) in the input directory that you want to use to test the wordcount example 2. Update the 'Pentaho MapReduce' step (Job Setup and Cluster tabs) to configure the correct paths and server names including: - Input Path - the path in HDFS from which to read files for counting - Output Path - where the processed count of words will be placed - HDFS Hostname - Job Tracker Hostname 20 40 443 73 Microsoft Sans Serif 8 N N 0 0 0 255 165 0 100 100 100 Y