Regression Tests

Sequential Taggers

Add tests for:
  • make sure backoff is being done correctly.
  • make sure ngram taggers don't use previous sentences for context.
  • make sure ngram taggers see 'beginning of the sentence' as a unique context
  • make sure regexp tagger's regexps are tried in order
  • train on some simple examples, & make sure that the size & the generated models are correct.
  • make sure cutoff works as intended
  • make sure that ngram models only exclude contexts covered by the backoff tagger if the backoff tagger gets that context correct at all locations.