CVE-2018-17540

Vulnerability

The gmp plugin in strongSwan before version 5.7.1 contains a heap buffer overflow vulnerability. It was introduced by the patch that fixes CVE-2018-16151/2, affecting any strongSwan version with that patch applied (versions 4.4.0 to 5.7.0). The bug occurs during RSA signature verification: a length check if (data.len > keylen - 11) uses an unsigned size_t for keylen, causing an integer underflow when keylen is less than 11 bytes. This underflow leads to a heap buffer overflow when the encoding is written to a buffer of only keylen bytes [1][4].

Exploitation

A remote attacker can exploit this vulnerability by sending a crafted certificate containing a very short RSA public key during an IKE handshake. The signature verification function in the gmp plugin processes the certificate, triggering the integer underflow and subsequent heap buffer overflow. No authentication or prior access is required; the attacker only needs network connectivity to the target strongSwan server [1][2].

Impact

Successful exploitation most likely crashes the IKE daemon, resulting in a denial of service. The Ubuntu security notice also states that arbitrary code execution may be possible, though the primary impact is a denial-of-service condition [1][2].

Mitigation

The vulnerability is fixed in strongSwan version 5.7.1. For versions 4.4.0 through 5.7.0, a patch is available from the strongSwan security page [4]. Ubuntu and Gentoo have released updated packages (USN-3774-1 and GLSA 201811-16, respectively) [2][3]. No workaround is known; upgrading to the fixed version is the recommended mitigation.