LiteLLM Supply Chain Attack Exposes Thousands of CI/CD Pipelines
A supply chain attack targeting the LiteLLM Python package potentially exposed over 2,500 companies and 434,000 CI/CD pipelines by compromising the Trivy scanner's release process.
A sophisticated supply chain attack has put thousands of companies and their continuous integration and continuous delivery (CI/CD) pipelines at risk, following the compromise of the popular LiteLLM Python package. The incident, attributed to a threat actor known as TeamPCP, saw malicious code injected into the build process, leading to the publication of poisoned LiteLLM packages on the Python Package Index (PyPI). While the malicious releases were only live for approximately 40 minutes, the potential impact is significant, with researchers identifying over 2,500 organizations and 434,000 CI/CD pipeline runs potentially affected.
The attack vector began when threat actors compromised the release process of the Trivy security scanner. LiteLLM's build pipeline, which incorporated Trivy without pinning to a specific, verified version, inadvertently pulled the compromised component. This allowed malicious code to infiltrate LiteLLM's build environment, ultimately resulting in the creation and distribution of tainted LiteLLM packages on PyPI. The malware was designed to execute as a Python startup file, capable of stealing sensitive credentials from developer workstations and CI/CD runners.
Once executed, the payload actively sought to exfiltrate a wide array of secrets. This included cloud service keys, AI service keys, repository tokens, SSH keys, Kubernetes tokens, and environment variables. The stolen data was encrypted and then transmitted to a typosquatted destination. In cases where direct exfiltration failed, the malware was programmed to create a new public repository within the victim's GitHub account and upload the stolen data as a release asset, a tactic designed to obscure the malicious activity by making it appear as if the victim themselves had published the data.
CloudSEK analysts, who identified the campaign, emphasized that while their dataset linked thousands of organizations to the affected path, this does not confirm a breach or data theft in every instance. They advise organizations to validate whether the affected LiteLLM versions were actually downloaded, cached, or executed, rather than assuming a compromise or complete safety. The credentials stolen could remain valuable long after the malicious packages were removed from PyPI, highlighting the persistent threat posed by such supply chain incidents.
In response to the incident, security teams are urged to identify all installations of the affected LiteLLM versions (1.82.7 and 1.82.8) and isolate any related runners, hosts, container images, and caches. A critical step involves rotating all credentials accessible by the compromised processes, not just those directly related to LiteLLM or AI service providers. Rebuilding affected environments from known-good sources and meticulously reviewing audit logs for cloud services, source control, and Kubernetes clusters are also recommended.
To prevent future occurrences, CloudSEK recommends implementing dependency pinning to verified hashes, reducing the lifespan and scope of credentials, and utilizing workload identity mechanisms over static keys wherever possible. This incident underscores the broader trend of attackers targeting trusted software components within development pipelines, a strategy that offers significant leverage due to the widespread use and implicit trust placed in these tools.
The compromise of LiteLLM serves as a stark reminder of the inherent risks within software supply chains, particularly in the rapidly evolving AI ecosystem. As AI systems become more integrated into critical infrastructure and development workflows, securing these components and the pipelines that manage them is paramount. The incident highlights the need for continuous vigilance, robust security practices, and a proactive approach to managing dependencies and credentials.