Getting the Most Out of Transformers in Elastic

This blog details Elastic’s experiments fine-tuning RoBERTa/DistilRoBERTa models for classifying malicious command lines, importing them into Elastic with Eland, and optimizing real-time inference using multi-node/threading, dynamic quantization, and knowledge distillation, yielding up to ~2.6–2.7x speedups and cutting test-time from ~4 hours to ~35 minutes; performance comparisons show transformers offer higher sensitivity on malicious samples while a tree-based model better handles benign data, motivating an ensemble approach and future inference caching.