psd-tools composite/numpy has uncontrolled memory allocation via crafted PSD geometry

Summary

PSDImage.composite() (and .numpy()) allocate the output image buffer from the PSD's header geometry (width × height × channels × depth, and per-layer rectangles) before validating those values against the actual file contents. A tiny crafted PSD declaring huge dimensions causes a multi-gigabyte allocation. Critically, composite() then returns a (black) image with only a warning, no exception is raised, so a caller cannot detect or guard against it.

Impact

On psd-tools 1.17.2 (latest), default usage, a 49-byte PSD makes composite() commit ~3 GB and return successfully (warning only); .numpy() reaches ~7.5 GB, and the per-layer rectangle is a second lever (up to ~32 GB), all from an input under 100 bytes (input-to-commit amplification over 1000×). Because the buffer is committed before validation and no exception is thrown, any service that composites untrusted PSDs is exposed to denial of service: on a host with less RAM than the attacker-declared geometry the allocation is an unrecoverable OOM-kill.

Steps to reproduce

# pip install psd-tools==1.17.2
from psd_tools import PSDImage
psd = PSDImage.open("psd-psdtools-grammar-d23.psd")
psd.composite()   # commits ~3 GB from a 49-byte file and returns (warning only)

PoC (49 bytes), reconstruct with: ``sh base64 -d > psd-psdtools-grammar-d23.psd <<'EOF' OEJQUwABAAAAAAAAAAYAACg4AAAXTAAIAAMAAAAAAAAAAAAAAAAAAUNIUIFU+yQtDw== EOF ``

Verify: `` 7d8ebf03a54393cb0359ecf4b676d1b08c9a8c6afdd06671ef406d6893cce826 psd-psdtools-grammar-d23.psd ``

Root cause

The composite/numpy buffer is sized from the declared image (and per-layer) dimensions and channel/depth without checking them against the available data length or a sane maximum.

Suggested fix

Validate the declared dimensions, channel count, and per-layer rectangles against the actual file length (and a configurable maximum pixel/byte budget) before allocating; raise an error on overflow instead of committing the buffer and returning a black image.