Cisco Research Reveals AI Model Lineage is More Complex Than Country of Origin Labels

Cisco research highlights a critical blind spot in how AI models are assessed for security risks: country-of-origin labels are often misleading. The US government, for instance, views Chinese AI as a national security threat, prompting many organizations to avoid any AI explicitly labeled as such. However, Cisco's findings, in collaboration with VAIL, demonstrate that this simplistic approach can lead to a false sense of security.

In a report titled “The ‘U.S. vs. China’ AI Trap: An Incomplete Proxy for AI Security,” Cisco researchers detail a phenomenon they call "provenance entanglement." This occurs because AI model developers frequently fine-tune existing models rather than training them from scratch. Consequently, a model might inherit weights, biases, and behavioral patterns from an upstream model that originated in a different country than its own label suggests. This means a US-developed model could inadvertently contain characteristics from a Chinese model, and vice-versa.

To investigate this, researchers analyzed Nemotron and Qwen models, as it's known that some Nemotron models are built upon Qwen base weights. They employed two distinct AI model fingerprinting methods: Cisco's Model Provenance Kit, which analyzes model weights internally, and VAIL's Behavioral Fingerprinting, which examines the model's inference behavior externally. Both methods confirmed a significant similarity between Nemotron models built from Qwen weights and the original Qwen models, indicating that post-training modifications and new publisher names do not erase detectable lineage relationships.

The implications of this "provenance entanglement" are akin to the software supply chain threats that led to the development of Software Bills of Materials (SBOMs). If an upstream model is later discovered to contain a backdoor, systemic bias, or exploitable vulnerabilities, organizations need a clear understanding of which downstream models might be affected. Without this visibility, security teams could be unknowingly exposed to risks inherited from compromised foundational models.

Cisco suggests improvements are needed across three key areas. Enterprises should treat publisher identity as just one piece of the puzzle, supplementing it with due diligence on lineage, training dependencies, behavior analysis, and operational control. Regulators require a deeper understanding of upstream dependencies to accurately assess vulnerabilities and biases stemming from model lineage. Finally, AI developers should embrace lineage disclosure as a routine practice, providing transparency that allows users to identify potential risks before integrating models into their technology stacks.

The research emphasizes that while country of origin labels offer some value—indicating the accountable developer, jurisdiction, and procurement process—they do not provide an accurate assessment of the internal components or inherent risks of an AI model. The analogy used is that "models do not have passports. They have supply chains."

This research underscores the need for a more sophisticated approach to AI security, moving beyond geopolitical labels to a comprehensive understanding of an AI model's development history and dependencies. The development of a "model bill of materials" that records base checkpoints, derivation methods, major datasets, and other critical information could significantly enhance responsible AI adoption and security.