Package nltk :: Package chunk :: Module util :: Class ChunkScore
[hide private]
[frames] | no frames]

Class ChunkScore

source code

object --+
         |
        ChunkScore

A utility class for scoring chunk parsers. ChunkScore can evaluate a chunk parser's output, based on a number of statistics (precision, recall, f-measure, misssed chunks, incorrect chunks). It can also combine the scores from the parsing of multiple texts; this makes it signifigantly easier to evaluate a chunk parser that operates one sentence at a time.

Texts are evaluated with the score method. The results of evaluation can be accessed via a number of accessor methods, such as precision and f_measure. A typical use of the ChunkScore class is:

   >>> chunkscore = ChunkScore()
   >>> for correct in correct_sentences:
   ...     guess = chunkparser.parse(correct.leaves())
   ...     chunkscore.score(correct, guess)
   >>> print 'F Measure:', chunkscore.f_measure()
   F Measure: 0.823
Instance Methods [hide private]
 
__init__(self, **kwargs)
x.__init__(...) initializes x; see x.__class__.__doc__ for signature
source code
 
_updateMeasures(self) source code
 
score(self, correct, guessed)
Given a correctly chunked sentence, score another chunked version of the same sentence.
source code
float
precision(self)
Returns: the overall precision for all texts that have been scored by this ChunkScore.
source code
float
recall(self)
Returns: the overall recall for all texts that have been scored by this ChunkScore.
source code
float
f_measure(self, alpha=0.5)
Returns: the overall F measure for all texts that have been scored by this ChunkScore.
source code
list of chunks
missed(self)
Returns: the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order.
source code
list of chunks
incorrect(self)
Returns: the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order.
source code
list of chunks
correct(self)
Returns: the chunks which were included in the correct chunk structures, listed in input order.
source code
list of chunks
guessed(self)
Returns: the chunks which were included in the guessed chunk structures, listed in input order.
source code
 
__len__(self) source code
String
__repr__(self)
Returns: a concise representation of this ChunkScoring.
source code
String
__str__(self)
Returns: a verbose representation of this ChunkScoring.
source code

Inherited from object: __delattr__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__

Instance Variables [hide private]
list of Token _fn
List of false negatives
int _fn_num
Number of false negatives.
list of Token _fp
List of false positives
int _fp_num
Number of false positives
list of Token _tp
List of true positives
int _tp_num
Number of true positives
  kwargs
Keyword arguments:
Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self, **kwargs)
(Constructor)

source code 

x.__init__(...) initializes x; see x.__class__.__doc__ for signature

Overrides: object.__init__
(inherited documentation)

score(self, correct, guessed)

source code 

Given a correctly chunked sentence, score another chunked version of the same sentence.

Parameters:
  • correct (chunk structure) - The known-correct ("gold standard") chunked sentence.
  • guessed (chunk structure) - The chunked sentence to be scored.

precision(self)

source code 
Returns: float
the overall precision for all texts that have been scored by this ChunkScore.

recall(self)

source code 
Returns: float
the overall recall for all texts that have been scored by this ChunkScore.

f_measure(self, alpha=0.5)

source code 
Parameters:
  • alpha (float) - the relative weighting of precision and recall. Larger alpha biases the score towards the precision value, while smaller alpha biases the score towards the recall value. alpha should have a value in the range [0,1].
Returns: float
the overall F measure for all texts that have been scored by this ChunkScore.

missed(self)

source code 
Returns: list of chunks
the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order.

incorrect(self)

source code 
Returns: list of chunks
the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order.

correct(self)

source code 
Returns: list of chunks
the chunks which were included in the correct chunk structures, listed in input order.

guessed(self)

source code 
Returns: list of chunks
the chunks which were included in the guessed chunk structures, listed in input order.

__repr__(self)
(Representation operator)

source code 

repr(x)

Returns: String
a concise representation of this ChunkScoring.
Overrides: object.__repr__

__str__(self)
(Informal representation operator)

source code 

str(x)

Returns: String
a verbose representation of this ChunkScoring. This representation includes the precision, recall, and f-measure scores. For other information about the score, use the accessor methods (e.g., missed() and incorrect()).
Overrides: object.__str__

Instance Variable Details [hide private]

kwargs

Keyword arguments:
  • max_tp_examples: The maximum number actual examples of true positives to record. This affects the correct member function: correct will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure)
  • max_fp_examples: The maximum number actual examples of false positives to record. This affects the incorrect member function and the guessed member function: incorrect will not return more than this number of examples, and guessed will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure)
  • max_fn_examples: The maximum number actual examples of false negatives to record. This affects the missed member function and the correct member function: missed will not return more than this number of examples, and correct will not return more than this number of true negative examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure)
  • chunk_node: A regular expression indicating which chunks should be compared. Defaults to '.*' (i.e., all chunks).