org.apache.nutch.mapReduce
Class TaskTracker

java.lang.Object
  org.apache.nutch.mapReduce.TaskTracker

All Implemented Interfaces:

MapOutputProtocol, MRConstants, Runnable, TaskUmbilicalProtocol

public class TaskTracker

extends Object

implements MRConstants, TaskUmbilicalProtocol, MapOutputProtocol, Runnable

TaskTracker is a process that starts and tracks MR Tasks in a networked environment. It contacts the JobTracker for Task assignments and reporting results.

Author:

Mike Cafarella

Nested Class Summary

static class

TaskTracker.Child The main() for child processes.

Field Summary

static Logger

LOG

Fields inherited from interface org.apache.nutch.mapReduce.MRConstants

FILE_NOT_FOUND, HEARTBEAT_INTERVAL, SUCCESS, TASKTRACKER_EXPIRY_INTERVAL

Constructor Summary

TaskTracker()
Start with the local machine name, and the default JobTracker

TaskTracker(InetSocketAddress jobTrackAddr)
Start with the local machine name, and the addr of the target JobTracker

Method Summary

void	close() Close down the TaskTracker and all its components.
void	done(String taskid) The task is done.
MapOutputFile	getFile(String mapTaskId, String reduceTaskId, IntWritable partition) Returns the output from the named map task destined for this partition.
InterTrackerProtocol	getJobClient() The connection to the JobTracker, used by the TaskRunner for locating remote files.
Task	getTask(String taskid) Called upon startup by the child process, to fetch Task data.
static void	main(String[] argv) Start the TaskTracker, point toward the indicated JobTracker
void	progress(String taskid, FloatWritable progress) Called periodically to report Task progress, from 0.0 to 1.0.
void	run() The server retry loop.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final Logger LOG

Constructor Detail

TaskTracker

public TaskTracker()
            throws IOException

Start with the local machine name, and the default JobTracker

TaskTracker

public TaskTracker(InetSocketAddress jobTrackAddr)
            throws IOException

Start with the local machine name, and the addr of the target JobTracker

Method Detail

close

public void close()
           throws IOException

Close down the TaskTracker and all its components. We must also shutdown any running tasks or threads, and cleanup disk space. A new TaskTracker within the same process space might be restarted, so everything must be clean.

Throws:

IOException

getJobClient

public InterTrackerProtocol getJobClient()

The connection to the JobTracker, used by the TaskRunner for locating remote files.

run

public void run()

The server retry loop. This while-loop attempts to connect to the JobTracker. It only loops when the old TaskTracker has gone bad (its state is stale somehow) and we need to reinitialize everything.

Specified by:

run in interface Runnable

getFile

public MapOutputFile getFile(String mapTaskId,
                             String reduceTaskId,
                             IntWritable partition)

Description copied from interface: MapOutputProtocol

Returns the output from the named map task destined for this partition.

Specified by:

getFile in interface MapOutputProtocol

getTask

public Task getTask(String taskid)
             throws IOException

Called upon startup by the child process, to fetch Task data.

Specified by:

getTask in interface TaskUmbilicalProtocol

Throws:

IOException

progress

public void progress(String taskid,
                     FloatWritable progress)
              throws IOException

Called periodically to report Task progress, from 0.0 to 1.0.

Specified by:

progress in interface TaskUmbilicalProtocol

Parameters:

progress - value between zero and one

Throws:

IOException

done

public void done(String taskid)
          throws IOException

The task is done.

Specified by:

done in interface TaskUmbilicalProtocol

Throws:

IOException

main

public static void main(String[] argv)
                 throws IOException

Start the TaskTracker, point toward the indicated JobTracker

Throws:

IOException