概要: Large language models are reportedly hallucinating software package names, some of which are uploaded to public repositories and integrated into real code. One such package, huggingface-cli, was downloaded over 15,000 times. This behavior enables "slopsquatting," a term coined by Seth Michael Larson of the Python Software Foundation, where attackers register fake packages under AI-invented names and put supply chains at serious risk.
Editor Notes: See Bar Lanyado's report at: https://www.lasso.security/blog/ai-package-hallucinations. See Spracklen, et al's preprint here, "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs," here: https://arxiv.org/abs/2406.10279. See Zhou, et al's study, "Larger and more instructable language models become less reliable," here: https://doi.org/10.1038/s41586-024-07930-y.
Alleged: OpenAI , Google , Cohere , Meta , DeepSeek AI と BigScience developed an AI system deployed by Developers using AI-generated suggestions と Bar Lanyado, which harmed Developers and businesses incorporating AI-suggested packages , Alibaba , Organizations that incorporated fake dependencies , Software ecosystems , Users downstream of software contaminated by hallucinated packages と Trust in open-source repositories and AI-assisted coding tools.
関与が疑われるAIシステム: LLM-powered coding assistants , ChatGPT 3.5 , ChatGPT 4 , Gemini Pro , Command , LLaMA , CodeLlama , DeepSeek Coder , BLOOM , Python Package Index (PyPI) , npm (Node.js) , GitHub と Google Search / AI Overview
インシデントのステータス
Risk Subdomain
A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
2.2. AI system security vulnerabilities and attacks
Risk Domain
The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
- Privacy & Security
Entity
Which, if any, entity is presented as the main cause of the risk
AI
Timing
The stage in the AI lifecycle at which the risk is presented as occurring
Post-deployment
Intent
Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
Unintentional
インシデントレポート
レポートタイムライン
Loading...
translated-ja-Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI.
Not only that but someone, having spotted this reoccurring hallucination, had turned that made-up…
Loading...

AIモデルは、どうにも勝手に勝手に作り出すのをやめられないようです。最近の2つの研究が指摘しているように、この傾向は、本当に重要な事柄についてはAIのアドバイスに頼るべきではないという過去の警告を裏付けています。
AIが頻繁に作り出すものの一つが、ソフトウェアパッケージの名前です。
今年初めに指摘したように、Lasso Securityは、大規模言語モデル(LLM)がサンプルソースコードを生成する際に、ソフトウェアパッケージの依存関係の名前を(実際には存在しない)作り出すこと…
Loading...