CVE-2010-3737

Vulnerability

A memory leak exists in the Relational Data Services component of IBM DB2 UDB 9.5 prior to Fix Pack 6a (FP6a). The leak is triggered when a remote authenticated user executes a user-defined function (UDF) or stored procedure while the client application and database server use different code pages, and the connection concentrator is enabled. The connection concentrator is activated when max_connections exceeds max_coordagents [1].

Exploitation

An attacker must have valid authentication credentials to the database. The attacker then repeatedly executes a UDF or stored procedure from a client with a code page different from the server's. The connection concentrator must be enabled. Each execution leaks heap memory, which accumulates over time. The db2pd -db -memblocks appctl top command can be used to observe the growing application heap [1].

Impact

Successful exploitation leads to progressive heap memory consumption, eventually exhausting the application heap. The database server may return error SQL0954C: Not enough storage is available in the application heap to process the statement (SQLSTATE=57011), causing a denial of service for the affected database instance [1].

Mitigation

IBM released Fix Pack 6a (FP6a) and subsequent fix packs (FP7, FP8, FP9, FP10) for DB2 9.5 that resolve this issue [1]. As a workaround, administrators can disable the connection concentrator by setting max_connections equal to or less than max_coordagents [1]. No other mitigations are documented.