org.apache.nutch.analysis.lang
Class LanguageIndexingFilter

java.lang.Object
  org.apache.nutch.analysis.lang.LanguageIndexingFilter

All Implemented Interfaces:

IndexingFilter

public class LanguageIndexingFilter

extends Object

implements IndexingFilter

An IndexingFilter that add a lang (language) field to the document. It tries to find the language of the document by:

• First, checking if HTMLLanguageParser add some language information
• Then, checking if a Content-Language HTTP header can be found
• Finaly by statisticaly analyzing the document content

Author:

Sami Siren, Jérôme Charron

Field Summary

Fields inherited from interface org.apache.nutch.indexer.IndexingFilter

X_POINT_ID

Constructor Summary

LanguageIndexingFilter()
Constructs a new Language Indexing Filter.

Method Summary

Document

filter(Document doc, Parse parse, FetcherOutput fo) Adds fields or otherwise modifies the document that will be indexed for a parse.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LanguageIndexingFilter

public LanguageIndexingFilter()

Constructs a new Language Indexing Filter.

Method Detail

filter

public Document filter(Document doc,
                       Parse parse,
                       FetcherOutput fo)
                throws IndexingException

Description copied from interface: IndexingFilter

Adds fields or otherwise modifies the document that will be indexed for a parse.

Specified by:

filter in interface IndexingFilter

Throws:

IndexingException