Byte Back: Next-Generation Malware Classification Using Binary Transformers

CrowdStrike introduces Binary Transformers, a token-free, byte-level Transformer architecture designed to understand malware files directly from binary data, leveraging strided convolution and shifted window attention to handle long sequences efficiently. In tests on benign and malicious shellcode, the approach significantly outperformed traditional tree-based models, demonstrating strong potential for improved malware detection and classification, with ongoing work focused on distillation and compression for lightweight deployment.