Searched refs:_byte_pair_encode (Results 1 – 1 of 1) sorted by relevance
231 static std::vector<uint64_t> _byte_pair_encode( in _byte_pair_encode() function311 auto tokens = _byte_pair_encode(piece, _encoder); in _encode()