PDFs encrypted with RC4 40-bit (R=2, V=1) — generated by mPDF 7.1.9 — throw BadPasswordException: Bad user password even though no password is set and the file opens normally in every other PDF reader (Acrobat, qpdf, PDFBox, Evince).
mPDF writes the /P permissions field as an unsigned 32-bit integer (e.g. 4294963412) instead of the signed value required by the PDF spec (e.g. -3884). The bit pattern is identical — it is purely an interpretation issue.
PdfNumber stores all numbers internally as double. intValue() does:
public int intValue() {
return (int) value; // value = 4294963412.0
}
Per JLS §5.1.3, casting a double to int does not use modular arithmetic. When the value exceeds Integer.MAX_VALUE, the result is clamped to 2147483647 instead of the correct -3884. The wrong pValue is fed into the MD5 that derives the RC4 key, which no longer matches the /U entry in the PDF.
The fix is a one-liner in PdfNumber.intValue():
// before
return (int) value;
// after — goes through long to preserve two's complement semantics
return (int)(long) value;
To Reproduce
// PDF generated by mPDF 7.1.9 with RC4 encryption (PDF 1.4 default)
// Encryption dict contains: /P 4294963412 (should be /P -3884)
new PdfReader("mpdf-rc4-encrypted.pdf", new byte[0]); // throws BadPasswordException
Replacing /P 4294963412 with /P -3884 (same byte length, padded with spaces) in the raw file bytes makes PdfReader open the document successfully, confirming the bug is isolated to the intValue() conversion.
Expected behavior:
PdfReader should open the document without a password, as the user password is empty and all other PDF readers handle it correctly.
System
- OS: Linux
- OpenPDF version: 3.0.3
Additional context:
The /P field in the PDF encryption dictionary is defined as a signed 32-bit integer by the PDF spec (ISO 32000). Some producers (including mPDF) write it as an unsigned value. Other readers handle this by converting through long before casting to int: (int)(long) value.
PDFs encrypted with RC4 40-bit (R=2, V=1) — generated by mPDF 7.1.9 — throw BadPasswordException: Bad user password even though no password is set and the file opens normally in every other PDF reader (Acrobat, qpdf, PDFBox, Evince).
mPDF writes the /P permissions field as an unsigned 32-bit integer (e.g. 4294963412) instead of the signed value required by the PDF spec (e.g. -3884). The bit pattern is identical — it is purely an interpretation issue.
PdfNumber stores all numbers internally as double. intValue() does:
public int intValue() {
return (int) value; // value = 4294963412.0
}
Per JLS §5.1.3, casting a double to int does not use modular arithmetic. When the value exceeds Integer.MAX_VALUE, the result is clamped to 2147483647 instead of the correct -3884. The wrong pValue is fed into the MD5 that derives the RC4 key, which no longer matches the /U entry in the PDF.
The fix is a one-liner in PdfNumber.intValue():
// before
return (int) value;
// after — goes through long to preserve two's complement semantics
return (int)(long) value;
To Reproduce
// PDF generated by mPDF 7.1.9 with RC4 encryption (PDF 1.4 default)
// Encryption dict contains: /P 4294963412 (should be /P -3884)
new PdfReader("mpdf-rc4-encrypted.pdf", new byte[0]); // throws BadPasswordException
Replacing /P 4294963412 with /P -3884 (same byte length, padded with spaces) in the raw file bytes makes PdfReader open the document successfully, confirming the bug is isolated to the intValue() conversion.
Expected behavior:
PdfReader should open the document without a password, as the user password is empty and all other PDF readers handle it correctly.
System
Additional context:
The /P field in the PDF encryption dictionary is defined as a signed 32-bit integer by the PDF spec (ISO 32000). Some producers (including mPDF) write it as an unsigned value. Other readers handle this by converting through long before casting to int: (int)(long) value.