With open-weight AI, more of the security burden falls on enterprises

Microsoft and NVIDIA warn organizations that running open-weight AI models on their own infrastructure exposes them to security and safety risks such as tampered or backdoored models, malicious model file formats (e.g., Pickle), and weakened safety behavior after fine-tuning or compression. The paper cites research showing practical backdoors (ProjectDiscovery’s Qwen backdoor, BadEngram memory backdoors), and advises establishing provenance (cryptographic signing), recording integrity as an open risk for unsigned models, deploying layered controls (input filtering, output moderation, access restrictions, rate limiting, human oversight), and continuous monitoring of model repositories and new misuse techniques; the guidance and a risk taxonomy are maintained on GitHub.