Runs basic tokenization such as punctuation spliting, lower casing.
Split text on whitespace, dropping the runs between words.
Tokenizes a piece of text.
@spec split_by_whitespace(binary()) :: [String.t()]
@spec tokenize(String.t(), boolean()) :: [String.t()]