AI hallucinates software packages and devs download them – even if potentially poisoned with malware

Security researcher Bar Lanyado showed that generative AI can repeatedly hallucinate non-existent package names that attackers could register on public repositories to drive supply-chain compromise, demonstrating the risk with a benign PyPI package ('huggingface-cli') that received 15,000+ downloads and appeared in Alibaba’s GraphTranslator README. His tests across GPT-4/3.5, Gemini, and Cohere found persistent hallucinations, with Python and npm ecosystems notably more susceptible than Go and .NET, highlighting a viable TTP despite no known malicious use yet.