\Camspiers\StatisticalClassifier\TokenizerTokenizerInterface

Summary

Methods
Constants
tokenize()
No constants found
No protected methods found
N/A
No private methods found
N/A

Methods

tokenize()

tokenize(string $document) : array

Split document into tokens

Parameters

string $document

The document to split

Returns

array —

An array of tokens