Corpus tokenizer# CorpusTokenizer([tokenizer_model_path]) Top-level worker for generating pronunciations from a corpus and a Pynini tokenizer model TokenizerValidator([utterances_to_tokenize]) Simple tokenizer# SimpleTokenizer(word_break_markers, ...[, ...])
