How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications

Dharma-AI describes VocabX, a method that rebuilds a small model's tokenizer around the cost of each token rather than its frequency, combining sub-word merges with multi-word "macro" tokens under a shared cost criterion. In a production Portuguese essay grader built on Qwen3-4B, the approach cut average response length from 350 to 101 tokens and measured inference cost by 77.0% with the same base model and training recipe, while the bench score rose from 0.703 to 0.810. The article previews results from a VocabX paper still under review, noting the numbers may change before publication.