ferritin_plms::esm::tokenization

Function get_model_tokenizers

source 
pub fn get_model_tokenizers(model: &str) -> Result<TokenizerCollection>