|
|||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.nutch.tools.FetchListTool
This class takes an IWebDBReader, computes a relevant subset, and then emits the subset.
Nested Class Summary | |
static class |
FetchListTool.SortableScore
SortableScore is just a WritableComparable Float! |
Field Summary | |
static Logger |
LOG
|
Constructor Summary | |
FetchListTool(NutchFileSystem nfs,
File dbDir,
boolean refetchOnly,
float cutoffScore,
int seed)
FetchListTool takes a page db, and emits a RECNO-based subset of it. |
Method Summary | |
void |
emitFetchList(File segmentDir,
long topN,
long curTime)
Spit out the fetchlist, to a BDB at the indicated filename. |
void |
emitMultipleLists(File dir,
int numLists,
long topN,
long curTime)
Spit out several fetchlists, so that we can fetch across several machines. |
static void |
main(String[] argv)
Generate a fetchlist from the pagedb and linkdb |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
public static final Logger LOG
Constructor Detail |
public FetchListTool(NutchFileSystem nfs, File dbDir, boolean refetchOnly, float cutoffScore, int seed) throws IOException, FileNotFoundException
Method Detail |
public void emitMultipleLists(File dir, int numLists, long topN, long curTime) throws IOException
IOException
public void emitFetchList(File segmentDir, long topN, long curTime) throws IOException
IOException
public static void main(String[] argv) throws IOException, FileNotFoundException
IOException
FileNotFoundException
|
|||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |