Home
last modified time | relevance | path

Searched defs:max_chars_per_subtoken (Results 1 – 3 of 3) sorted by relevance

/aosp_15_r20/external/libtextclassifier/native/utils/
H A Dwordpiece_tokenizer.cc50 const std::string& suffix_indicator, const int max_chars_per_subtoken, in LongestMatchStartingAt()
161 const int max_chars_per_subtoken, const std::string& suffix_indicator, in TokenizeL2RGreedy()
211 const int max_chars_per_subtoken, const std::string& suffix_indicator, in WordpieceTokenize()
H A Dbert_tokenizer.h60 int max_chars_per_subtoken = kDefaultMaxCharsPerSubToken; member
/aosp_15_r20/external/tflite-support/tensorflow_lite_support/cc/text/tokenizers/
H A Dbert_tokenizer.h61 int max_chars_per_subtoken = kDefaultMaxCharsPerSubToken; member