org.apache.nutch.io
Class SequenceFile.Sorter

java.lang.Object
  org.apache.nutch.io.SequenceFile.Sorter

Enclosing class:

SequenceFile

public static class SequenceFile.Sorter

extends Object

Sorts key/value pairs in a sequence-format file.

For best performance, applications should make sure that the Writable.readFields(DataInput) implementation of their keys is very efficient. In particular, it should avoid allocating memory.

Constructor Summary

SequenceFile.Sorter(NutchFileSystem nfs, Class keyClass, Class valClass)
Sort and merge files containing the named classes.

SequenceFile.Sorter(NutchFileSystem nfs, WritableComparator comparator, Class valClass)
Sort and merge using an arbitrary WritableComparator.

Method Summary

int	getFactor() Get the number of streams to merge at once.
int	getMemory() Get the total amount of buffer memory, in bytes.
void	merge(String[] inFiles, String outFile) Merge the provided files.
void	setFactor(int factor) Set the number of streams to merge at once.
void	setMemory(int memory) Set the total amount of buffer memory, in bytes.
void	sort(String inFile, String outFile) Perform a file sort.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SequenceFile.Sorter

public SequenceFile.Sorter(NutchFileSystem nfs,
                           Class keyClass,
                           Class valClass)

Sort and merge files containing the named classes.

SequenceFile.Sorter

public SequenceFile.Sorter(NutchFileSystem nfs,
                           WritableComparator comparator,
                           Class valClass)

Sort and merge using an arbitrary WritableComparator.

Method Detail

setFactor

public void setFactor(int factor)

Set the number of streams to merge at once.

getFactor

public int getFactor()

Get the number of streams to merge at once.

setMemory

public void setMemory(int memory)

Set the total amount of buffer memory, in bytes.

getMemory

public int getMemory()

Get the total amount of buffer memory, in bytes.

sort

public void sort(String inFile,
                 String outFile)
          throws IOException

Perform a file sort.

Throws:

IOException

merge

public void merge(String[] inFiles,
                  String outFile)
           throws IOException

Merge the provided files.

Throws:

IOException