Organizations are rapidly adopting AI models, but many still lack visibility into where those models come from or how they’ve been modified along the way.
Cisco is aiming to close that gap with the release of its open-source Model Provenance Kit, a tool designed to verify the origins of AI models and improve trust across the AI supply chain.
“We’re at the AI equivalent of the early internet, when systems were focused on capability advancements,” said Amy Chang, Head of AI Threat Intelligence & Security Research at Cisco in an email to eSecurityPlanet.
She explained, “Model provenance is emerging as the missing layer that can shed light into an AI model’s lineage and training, which can inform organizations about where it came from and whether it can be trusted.”
Amy also added, “As AI continues to advance into regulated, high-stakes domains, provenance will become foundational to governance, accountability, and enforceable trust.”
Cisco’s Approach to AI Model Provenance
As enterprises accelerate adoption of third-party and open-source AI models, understanding model lineage is quickly becoming a foundational requirement for managing risk.
Modern AI systems are rarely built from scratch — they are continuously fine-tuned, compressed, merged, or otherwise modified, producing layers of derivative models.
Each transformation introduces the potential to inherit not only capabilities, but also vulnerabilities, hidden dependencies, and licensing obligations.
Without a reliable way to trace these relationships, organizations face growing challenges across compliance, incident response, and overall supply chain security.
How Cisco’s Model Provenance Kit Works
Cisco’s Model Provenance Kit is designed to address this gap by giving organizations a way to verify where models come from and how they are related.
The tool fingerprints models at the weight level — the underlying parameters that define model behavior — allowing security teams to determine whether one model is derived from another with a high degree of confidence.
Complementing this, Cisco introduced the Model Provenance Constitution, a formal framework that defines what constitutes a legitimate derivation relationship and, just as importantly, what does not.
Defining Provenance at the Weight Level
At the core of Cisco’s approach is a precise and restrictive definition of provenance based on weight-level derivation.
Under this model, two AI systems are considered related only if there is a direct or indirect causal chain connecting their trained parameters.
This includes common development paths such as fine-tuning from a base model, knowledge distillation from a teacher model, or mechanical transformations like quantization, pruning, or model merging.
By anchoring provenance in verifiable weight relationships, the framework provides a consistent and technically grounded standard that can be applied across organizations.
What the Framework Excludes
Equally important is what the framework deliberately excludes.
Superficial similarities, such as shared architectures, overlapping training datasets, or comparable benchmark performance, are not treated as evidence of derivation.
This distinction is critical in practice. Without it, organizations could mistakenly classify unrelated models as dependent, leading to false positives in vulnerability tracking, unnecessary licensing concerns, and increased noise in governance processes.
By drawing a clear boundary between true derivation and coincidental similarity, the framework reduces ambiguity and improves decision-making accuracy.
Model Provenance Constitution
The Model Provenance Constitution further strengthens this approach by explicitly outlining the conditions under which models are considered related, including direct descent, indirect descent, mechanical transformation, and transitive relationships across multiple stages.
It also catalogs common false signals — such as independently developed models that happen to resemble one another — helping teams avoid misclassification.
This structured taxonomy ensures that every model comparison can be evaluated against a consistent set of criteria.
Why Provenance Matters for AI Security
The need for this level of rigor is driven by the evolving threat landscape.
Weak model provenance has already been identified as a growing risk in AI environments, especially in the context of supply chain attacks.
Adversaries can exploit poorly documented model dependencies to introduce malicious code, backdoors, or vulnerabilities into widely reused components.
Industry frameworks such as OWASP’s Top 10 for LLM applications and MITRE ATLAS highlight supply chain compromise as a primary threat vector, reinforcing the importance of traceability and verification.
Building Trust Through Verifiable Evidence
To support real-world use, Cisco’s approach emphasizes verifiable evidence over assumptions.
Provenance can be established through official documentation, technical validation of model checkpoints, or authoritative third-party analysis.
By relying on weight-level verification instead of manipulable metadata or naming, the framework helps prevent attempts to obscure a model’s origin.
Together, these capabilities give organizations clearer visibility into model dependencies and a stronger foundation for managing AI supply chain risk.
How to Reduce AI Model Risk
As organizations integrate AI into critical business processes, managing model risk is becoming a core security priority.
AI systems introduce new challenges across data, dependencies, and dynamic behavior that require a more comprehensive approach to risk reduction.
Addressing these risks requires safeguards across the entire AI lifecycle, from development through deployment and operations.
- Implement model provenance and supply chain controls by verifying lineage, validating third-party models, and treating models as managed dependencies.
- Establish strong governance policies that require documentation of model origins, transformations, and risk classification aligned to frameworks like NIST AI RMF.
- Secure data across the AI lifecycle by protecting training and inference pipelines, preventing data leakage, and validating datasets against poisoning risks.
- Enforce identity and access controls using least privilege and zero trust principles for all users, APIs, and systems interacting with models.
- Continuously monitor and log model behavior to detect anomalies, drift, or signs of tampering and enable effective forensic analysis.
- Apply model and application-layer protections such as adversarial testing, guardrails, output filtering, and environment isolation to reduce misuse and exploitation risk.
- Develop and regularly test AI-specific incident response plans to ensure readiness for model compromise, data exposure, or malicious outputs.
Collectively, these measures help organizations build resilience and reduce exposure to AI model risks.
Rise of AI Supply Chain Risk
Cisco’s Model Provenance Kit highlights an ongoing shift in how organizations approach AI risk management.
As AI systems become more modular and interconnected, the traditional concept of a software supply chain is expanding to include models, datasets, and training pipelines.
In this environment, establishing clear provenance is increasingly important for maintaining security, supporting compliance efforts, and building operational confidence.
Without better visibility into how models are developed and related, organizations may face challenges in identifying dependencies, assessing risk, and managing potential inherited vulnerabilities.
These challenges reinforce the need for zero trust that helps organizations continuously verify systems, data, and dependencies across AI environments.