Home
last modified time | relevance | path

Searched defs:clean_up_tokenization_spaces (Results 1 – 4 of 4) sorted by relevance

/aosp_15_r20/external/executorch/examples/mediatek/aot_utils/llm_utils/tokenizers_/
H A Dtokenization_llama.py174 clean_up_tokenization_spaces=False, argument
/aosp_15_r20/external/executorch/examples/mediatek/models/llm_models/weights/llama3-8B-instruct/
H A Dtokenizer_config.json2054 "clean_up_tokenization_spaces": true, boolean
/aosp_15_r20/external/executorch/examples/mediatek/models/llm_models/weights/Llama-3.2-1B-Instruct/
H A Dtokenizer_config.json2054 "clean_up_tokenization_spaces": true, boolean
/aosp_15_r20/external/executorch/examples/mediatek/models/llm_models/weights/Llama-3.2-3B-Instruct/
H A Dtokenizer_config.json2054 "clean_up_tokenization_spaces": true, boolean