Uses of Class
org.apache.nutch.fs.NutchFileSystem

Packages that use NutchFileSystem

org.apache.nutch.db	Web database: tracks page fetches and link structure.
org.apache.nutch.fetcher	The Nutch robot.
org.apache.nutch.fs
org.apache.nutch.indexer	Maintain Lucene full-text indexes.
org.apache.nutch.io	Generic i/o code for use when reading and writing data to the network, to databases, and to files.
org.apache.nutch.mapReduce	A system for scalable, fault-tolerant, distributed computation over large data collections.
org.apache.nutch.searcher	Search API
org.apache.nutch.segment
org.apache.nutch.tools

Uses of NutchFileSystem in org.apache.nutch.db

Methods in org.apache.nutch.db with parameters of type NutchFileSystem

static void	EditSectionGroupWriter.createEditGroup(NutchFileSystem nfs, File dbDir, String label, int numSections, int keySpaceType) Initialize an EditSectionGroup.
static void	WebDBWriter.createWebDB(NutchFileSystem nfs, File dbDir) Create the WebDB for the first time.
static void	DistributedWebDBWriter.createDB(NutchFileSystem nfs, File root, int totalMachines) Method useful for the first time we create a distributed db project.

Constructors in org.apache.nutch.db with parameters of type NutchFileSystem

DBSectionReader(NutchFileSystem nfs, File sectionFile, WritableComparator comparator)
Right now we assume we're getting a File that is a MapFile.Reader directory.

EditSectionGroupWriter(NutchFileSystem nfs, int machineNum, int totalMachines, String label, Class keyClass, Class valClass, EditSectionGroupWriter.KeyExtractor extractor)
Start a EditSectionGroupWriter at the indicated location, for a single emitter.

WebDBWriter(NutchFileSystem fs, File dbDir)
Create a WebDBWriter.

WebDBReader(NutchFileSystem nfs, File dbDir)
Open a web db reader for the named directory.

EditSectionGroupReader(NutchFileSystem nfs, String label, int readerNum, int totalMachines)
Open the EditSectionGroupReader for the appropriate file.

EditSectionWriter(NutchFileSystem nfs, String label, int targetNum, int writerNum, Class keyClass, Class valClass)
Make a EditSectionWriter for the appropriate file.

DistributedWebDBWriter(NutchFileSystem nfs, File root, int machineNum)
Open the db files.

DistributedWebDBReader(NutchFileSystem nfs, File root)
Open a web db reader for the named directory.

Uses of NutchFileSystem in org.apache.nutch.fetcher

Constructors in org.apache.nutch.fetcher with parameters of type NutchFileSystem

Fetcher(NutchFileSystem nfs, String directory, boolean parsing)

Uses of NutchFileSystem in org.apache.nutch.fs

Subclasses of NutchFileSystem in org.apache.nutch.fs

class	LocalFileSystem Implement the NutchFileSystem interface for the local disk.
class	NDFSFileSystem Implement the NutchFileSystem interface for the NDFS system.

Methods in org.apache.nutch.fs that return NutchFileSystem

static NutchFileSystem	NutchFileSystem.parseArgs(String[] argv, int i) Parse the cmd-line args, starting at i.
static NutchFileSystem	NutchFileSystem.get() Returns the default filesystem implementation.
static NutchFileSystem	NutchFileSystem.getNamed(String name) Returns a named filesystem.

Methods in org.apache.nutch.fs with parameters of type NutchFileSystem

static boolean	FileUtil.fullyDelete(NutchFileSystem nfs, File dir)
static boolean	FileUtil.copyContents(NutchFileSystem nfs, File src, File dst, boolean overwrite) Copy a file's contents to a new location.
static void	FileUtil.recursiveCopy(NutchFileSystem nfs, File src, File dst) Copy a file and/or directory and all its contents (whether data or other files/dirs)

Constructors in org.apache.nutch.fs with parameters of type NutchFileSystem

TestClient(NutchFileSystem nfs)

Uses of NutchFileSystem in org.apache.nutch.indexer

Constructors in org.apache.nutch.indexer with parameters of type NutchFileSystem

IndexSegment(NutchFileSystem nfs, long maxDocs, File srcDir, File localWorkingDir)
Index a segment in the given NFS.

IndexMerger(NutchFileSystem nfs, File[] segments, File outputIndex, File localWorkingDir)
Merge all of the segments given

Uses of NutchFileSystem in org.apache.nutch.io

Methods in org.apache.nutch.io with parameters of type NutchFileSystem

static void	MapFile.rename(NutchFileSystem nfs, String oldName, String newName) Renames an existing map directory.
static void	MapFile.delete(NutchFileSystem nfs, String name) Deletes the named map file.
static long	MapFile.fix(NutchFileSystem nfs, File dir, Class keyClass, Class valueClass, boolean dryrun) This method attempts to fix a corrupt MapFile by re-creating its index.

Constructors in org.apache.nutch.io with parameters of type NutchFileSystem

MapFile.Writer(NutchFileSystem nfs, String dirName, Class keyClass, Class valClass)
Create the named map for keys of the named class.

MapFile.Writer(NutchFileSystem nfs, String dirName, WritableComparator comparator, Class valClass)
Create the named map using the named key comparator.

MapFile.Reader(NutchFileSystem nfs, String dirName)
Construct a map reader for the named map.

MapFile.Reader(NutchFileSystem nfs, String dirName, WritableComparator comparator)
Construct a map reader for the named map using the named comparator.

SetFile.Writer(NutchFileSystem nfs, String dirName, Class keyClass)
Create the named set for keys of the named class.

SetFile.Writer(NutchFileSystem nfs, String dirName, WritableComparator comparator)
Create the named set using the named key comparator.

SequenceFile.Writer(NutchFileSystem nfs, String name, Class keyClass, Class valClass)
Create the named file.

SequenceFile.Reader(NutchFileSystem nfs, String file)
Open the named file.

SequenceFile.Sorter(NutchFileSystem nfs, Class keyClass, Class valClass)
Sort and merge files containing the named classes.

SequenceFile.Sorter(NutchFileSystem nfs, WritableComparator comparator, Class valClass)
Sort and merge using an arbitrary WritableComparator.

ArrayFile.Writer(NutchFileSystem nfs, String file, Class valClass)
Create the named file for values of the named class.

SetFile.Reader(NutchFileSystem nfs, String dirName)
Construct a set reader for the named set.

SetFile.Reader(NutchFileSystem nfs, String dirName, WritableComparator comparator)
Construct a set reader for the named set using the named comparator.

ArrayFile.Reader(NutchFileSystem nfs, String file)
Construct an array reader for the named file.

Uses of NutchFileSystem in org.apache.nutch.mapReduce

Methods in org.apache.nutch.mapReduce that return NutchFileSystem

NutchFileSystem

JobClient.getFs() Get a filesystem handle.

Methods in org.apache.nutch.mapReduce with parameters of type NutchFileSystem

RecordWriter	OutputFormat.getRecordWriter(NutchFileSystem fs, JobConf job, String name) Construct a RecordWriter.
RecordWriter	SequenceFileOutputFormat.getRecordWriter(NutchFileSystem fs, JobConf job, String name)
RecordWriter	TextOutputFormat.getRecordWriter(NutchFileSystem fs, JobConf job, String name)
abstract RecordReader	InputFormatBase.getRecordReader(NutchFileSystem fs, FileSplit split, JobConf job)
protected File[]	InputFormatBase.listFiles(NutchFileSystem fs, JobConf job) Subclasses may override to, e.g., select only files matching a regular expression.
FileSplit[]	InputFormatBase.getSplits(NutchFileSystem fs, JobConf job, int numSplits) Splits files returned by {#listFiles(NutchFileSystem,JobConf) when they're too big.
RecordReader	TextInputFormat.getRecordReader(NutchFileSystem fs, FileSplit split, JobConf job)
FileSplit[]	InputFormat.getSplits(NutchFileSystem fs, JobConf job, int numSplits) Splits a set of input files.
RecordReader	InputFormat.getRecordReader(NutchFileSystem fs, FileSplit split, JobConf job) Construct a RecordReader for a FileSplit.
RecordReader	SequenceFileInputFormat.getRecordReader(NutchFileSystem fs, FileSplit split, JobConf job)

Uses of NutchFileSystem in org.apache.nutch.searcher

Constructors in org.apache.nutch.searcher with parameters of type NutchFileSystem

FetchedSegments(NutchFileSystem nfs, String segmentsDir)
Construct given a directory containing fetcher output.

Uses of NutchFileSystem in org.apache.nutch.segment

Fields in org.apache.nutch.segment declared as NutchFileSystem

NutchFileSystem

SegmentReader.nfs

Methods in org.apache.nutch.segment with parameters of type NutchFileSystem

static boolean	SegmentReader.isParsedSegment(NutchFileSystem nfs, File segdir)
static boolean	SegmentReader.fixSegment(NutchFileSystem nfs, File dir, boolean withContent, boolean withParseText, boolean withParseData, boolean dryrun) Attempt to fix a partially corrupted segment.

Constructors in org.apache.nutch.segment with parameters of type NutchFileSystem

SegmentReader(NutchFileSystem nfs, File dir)
Open a segment for reading.

SegmentReader(NutchFileSystem nfs, File dir, boolean autoFix)
Open a segment for reading.

SegmentReader(NutchFileSystem nfs, File dir, boolean withContent, boolean withParseText, boolean withParseData, boolean autoFix)
Open a segment for reading.

SegmentWriter(NutchFileSystem nfs, File dir, boolean force)

SegmentWriter(NutchFileSystem nfs, File dir, boolean force, boolean isParsed)

SegmentWriter(NutchFileSystem nfs, File dir, boolean force, boolean isParsed, boolean withContent, boolean withParseText, boolean withParseData)
Open a segment for writing.

SegmentSlicer(NutchFileSystem nfs, File[] input, File output, boolean withContent, boolean withParseText, boolean withParseData, boolean autoFix, long maxCount, boolean plusSign, org.apache.oro.text.regex.Pattern pattern)
Create new SegmentSlicer.

Uses of NutchFileSystem in org.apache.nutch.tools

Methods in org.apache.nutch.tools with parameters of type NutchFileSystem

void	UpdateDatabaseTool.updateForSegment(NutchFileSystem nfs, String directory) Iterate through items in the FetcherOutput.

Constructors in org.apache.nutch.tools with parameters of type NutchFileSystem

ParseSegment(NutchFileSystem nfs, String directory, boolean dryRun)
ParseSegment constructor

LinkAnalysisTool(NutchFileSystem nfs, File dbDir)
We need a DistributedAnalysisTool in order to get things done!

SegmentMergeTool(NutchFileSystem nfs, File[] segments, File output, long maxCount, boolean runIndexer, boolean delSegs)
Create a SegmentMergeTool.

UpdateSegmentsFromDb(NutchFileSystem fs, String dbDir, String segmentsDir, String tempDir)
Updates all segemnts in the named directory from the named db.

FetchListTool(NutchFileSystem nfs, File dbDir, boolean refetchOnly, float cutoffScore, int seed)
FetchListTool takes a page db, and emits a RECNO-based subset of it.

DistributedAnalysisTool(NutchFileSystem nfs, File dbDir)
Give the pagedb and linkdb files and their cache sizes