Class TabTokenizer

source code

    object --+    
             |    
api.TokenizerI --+
                 |
                TabTokenizer

A tokenizer that divides a string into substrings by treating any single tab character as a separator. If you are performing the tokenization yourself (rather than building a tokenizer to pass to some other piece of code), consider using the string split() method instead:

>>> words = s.split('\t')