apr tokenize
Tokenizer training pipeline (plan/apply) — BPE vocabulary learning
Category: Training
Synopsis
apr tokenize [OPTIONS]
Example
apr tokenize "Hello world" --tokenizer qwen2.5-coder-1.5b
Full help
Run apr tokenize --help for the complete option list.