Security Certifications Offer a Limited View, Not a Guarantee
New research from Masaryk University reveals that many vulnerabilities in certified products are disclosed after their certification date, challenging the perceived security assurance of certifications like Common Criteria and FIPS-140.
Procurement managers often face a deluge of information when evaluating security-certified products, with documents like Common Criteria's Security Target or FIPS-140's Security Policy presenting a complex picture. While these documents outline the product's name, version, assurance level, and evaluated configuration, their utility is limited by their static nature. A certification represents a snapshot in time, detailing the product's security posture at the point of evaluation, but it offers no insight into vulnerabilities discovered or changes made thereafter.
Mariia Bakhtina, a research specialist at Masaryk University, highlighted this limitation in a recent discussion. Her team's analysis found that approximately 60% of vulnerabilities associated with certified products, excluding smartcards, were disclosed after the certification date had passed. This means that a product certified today might already have known security flaws that were not accounted for during its evaluation, undermining the confidence buyers might place in the certification as a guarantee of ongoing security.
The term "certified" can be misleading, implying a comprehensive validation of security. However, certifications like Common Criteria (CC) and FIPS 140-2/3 operate within specific scopes and processes. A CC certificate confirms that a product was evaluated by an accredited lab against the vendor's specific security claims, often based on a Protection Profile. It validates the claims and the evaluation process at a stated assurance level (EAL), rather than declaring the product universally secure.
Similarly, FIPS 140 validates cryptographic modules against specific requirements, focusing on the module's internal security and defined boundaries. It does not extend to the broader application or system in which the module operates. Buyers often misinterpret these certifications, assuming they guarantee the absence of vulnerabilities, cover the entire product, or apply to the most current version. The reality is that scope can be limited, versions may differ, and certifications have a finite validity period, typically five years.
Furthermore, a higher assurance level in CC does not inherently mean a product is more secure overall. Instead, it signifies a more thorough examination. While products with higher EALs may exhibit less severe vulnerabilities, they are not necessarily less numerous. This distinction is crucial for buyers to understand, as it reframes the value of the assurance level from a security guarantee to an indicator of evaluation depth.
To address these challenges, Bakhtina's lab has developed open-source tools like sec-certs and SCRUTINY. These tools aim to make certification documents more accessible and searchable. By transforming public certification documents into a linked dataset, they allow buyers to quickly verify a certificate's active status, check for newer versions or re-certifications, and trace dependencies on underlying hardware or software components.
These tools are particularly valuable because they allow buyers to answer critical questions in minutes that previously required extensive consultant engagement or days of manual research. For instance, checking if a certificate is still valid, if a newer version exists, or understanding the components the certified product relies upon, can now be done rapidly. This increased transparency empowers buyers to make more informed decisions by quickly accessing and cross-referencing relevant documentation.
Ultimately, security certifications serve as a valuable, but incomplete, indicator of a product's security. They are a point-in-time assessment, not a continuous guarantee. Tools like sec-certs and SCRUTINY are essential in bridging the gap between the static nature of certifications and the dynamic reality of cybersecurity, enabling buyers to better navigate vendor claims and assess the true security posture of the products they procure.