When Tokenizers Drift: Hidden Costs and Security Risks in LLM Deployments

This report explains the risk of "tokenizer drift," where altered tokenizer/normalizer files—whether accidental or malicious—can double or triple token counts (raising cost, CPU, and latency) and can also transform harmless text into router-interpretable control sequences that trigger unintended actions; it includes experiments, attack scenarios (malicious uploads and normalization tampering), a demo, and practical mitigations such as hashing, calibration tests, monitoring, and supply-chain controls.