Home
last modified time | relevance | path

Searched refs:get_spm_processor (Results 1 – 1 of 1) sorted by relevance

/aosp_15_r20/external/executorch/examples/mediatek/aot_utils/llm_utils/tokenizers_/
H A Dtokenization_llama.py210 self.sp_model = self.get_spm_processor(kwargs.pop("from_slow", False))
232 def get_spm_processor(self, from_slow=False): member in LlamaTokenizer