PDFium compile cache: drop the go:linkname shortcut, rely on wazero's content-addressed cache; long-term artifact pinning, SBOM and provenance #1150

Open
opened 2026-11-09 13:41:46 +00:00 by Cordy · 1 comment
Owner

Tracking issue from v0.6.404 (#1101, fix fx6t, commit aa330df7).

What the shortcut is. The pod-local compile cache stores PDFium as native machine code compiled by wazero. Cairn writes a marker file so it can empty the folder when the module changes. To put the exact module hash in that marker, fx6t reads go-pdfium's embedded pdfium.wasm with //go:linkname embeddedWASM github.com/klippa-app/go-pdfium/internal/pdfium_wasm.Module. That is a private (internal) variable of another module: Go's internal-package rule forbids importing it, and linkname bypasses that rule. Go 1.23 already blocks this for standard-library internals (VERIFIED, https://go.dev/doc/go1.23 "Linker"); for third-party modules it still links, but any go-pdfium release may rename or move the variable. Today that would turn the cache off with a WARN and fail TestCompileCacheEmbeddedWASMFx6t, so it cannot break silently, but it is not a professional dependency.

Finding: the marker is redundant for correctness. wazero v1.12.0 (VERIFIED in the module source):

  • internal/wasm/module.go AssignModuleID: module ID = sha256 of the wasm bytes (plus listener and termination flags).
  • internal/engine/wazevo/engine_cache.go fileCacheKey: file key = sha256(module ID + "WAZEVO" + CPU features).
  • cache.go:102: the cache directory is wazero-<version>-<GOARCH>-<GOOS>.
  • deserializeCompiledModule checks the wazero version in each entry header and deletes stale entries on read.
    So a different pdfium.wasm, wazero version or CPU can never load a stale entry. In addition the default folder is in the pod's /tmp (emptyDir), which starts empty with every new pod, so an upgrade never meets an old cache.

Short-term fix (small, planned with batch 5 / v0.6.405): remove the linkname and the wasm hash from the marker; keep the folder safety checks (0700, owner, no group/other write, re-check before each compile) which are the real security boundary; keep a marker only for housekeeping (marker version + Go version + GOOS/GOARCH), and delete stale wazero-* subdirectories other than the current one. Update the marker sentence in security-standards.md, configuration-reference.md, ARCHITECTURE.md and the CHANGELOG.

Long-term professional setup (industry practice):

  1. Pin the artifact. go.sum already pins the go-pdfium module zip (which contains pdfium.wasm) through the Go checksum database, a public transparency log; keep it that way (no GOFLAGS=-insecure, GONOSUMDB empty for it). If Cairn ever needs a patched PDFium, ship its own checksummed artifact and use webassembly.InitWithWASM with Config.WASM, so the hash is of bytes Cairn owns.
  2. SBOM. Generate a CycloneDX (or SPDX) SBOM in the release pipeline that lists the Go modules and the components inside pdfium.wasm (PDFium, FreeType, libjpeg-turbo, zlib, lcms2, OpenJPEG, ICU, musl, LLVM runtime), attached to each release (pairs with #1148).
  3. Provenance. Reproducible build recipe and, ideally, a SLSA provenance attestation for pdfium.wasm from upstream (#1147); otherwise Cairn's own reproducible build of the wasm in CI.
  4. Upstream accessor. If a hash of the embedded module is ever needed again, ask go-pdfium for a public accessor instead of linkname (#1147 item 3).
Tracking issue from v0.6.404 (#1101, fix fx6t, commit aa330df7). **What the shortcut is.** The pod-local compile cache stores PDFium as native machine code compiled by wazero. Cairn writes a marker file so it can empty the folder when the module changes. To put the exact module hash in that marker, fx6t reads go-pdfium's embedded `pdfium.wasm` with `//go:linkname embeddedWASM github.com/klippa-app/go-pdfium/internal/pdfium_wasm.Module`. That is a private (internal) variable of another module: Go's internal-package rule forbids importing it, and linkname bypasses that rule. Go 1.23 already blocks this for standard-library internals (VERIFIED, https://go.dev/doc/go1.23 "Linker"); for third-party modules it still links, but any go-pdfium release may rename or move the variable. Today that would turn the cache off with a WARN and fail `TestCompileCacheEmbeddedWASMFx6t`, so it cannot break silently, but it is not a professional dependency. **Finding: the marker is redundant for correctness.** wazero v1.12.0 (VERIFIED in the module source): - `internal/wasm/module.go` `AssignModuleID`: module ID = sha256 of the wasm bytes (plus listener and termination flags). - `internal/engine/wazevo/engine_cache.go` `fileCacheKey`: file key = sha256(module ID + "WAZEVO" + CPU features). - `cache.go:102`: the cache directory is `wazero-<version>-<GOARCH>-<GOOS>`. - `deserializeCompiledModule` checks the wazero version in each entry header and deletes stale entries on read. So a different pdfium.wasm, wazero version or CPU can never load a stale entry. In addition the default folder is in the pod's /tmp (emptyDir), which starts empty with every new pod, so an upgrade never meets an old cache. **Short-term fix (small, planned with batch 5 / v0.6.405):** remove the linkname and the wasm hash from the marker; keep the folder safety checks (0700, owner, no group/other write, re-check before each compile) which are the real security boundary; keep a marker only for housekeeping (marker version + Go version + GOOS/GOARCH), and delete stale `wazero-*` subdirectories other than the current one. Update the marker sentence in security-standards.md, configuration-reference.md, ARCHITECTURE.md and the CHANGELOG. **Long-term professional setup (industry practice):** 1. **Pin the artifact.** go.sum already pins the go-pdfium module zip (which contains pdfium.wasm) through the Go checksum database, a public transparency log; keep it that way (no `GOFLAGS=-insecure`, `GONOSUMDB` empty for it). If Cairn ever needs a patched PDFium, ship its own checksummed artifact and use `webassembly.InitWithWASM` with `Config.WASM`, so the hash is of bytes Cairn owns. 2. **SBOM.** Generate a CycloneDX (or SPDX) SBOM in the release pipeline that lists the Go modules and the components inside pdfium.wasm (PDFium, FreeType, libjpeg-turbo, zlib, lcms2, OpenJPEG, ICU, musl, LLVM runtime), attached to each release (pairs with #1148). 3. **Provenance.** Reproducible build recipe and, ideally, a SLSA provenance attestation for pdfium.wasm from upstream (#1147); otherwise Cairn's own reproducible build of the wasm in CI. 4. **Upstream accessor.** If a hash of the embedded module is ever needed again, ask go-pdfium for a public accessor instead of linkname (#1147 item 3).
Author
Owner

The short-term fix shipped in v0.6.405 (ccm6405). The go:linkname is gone, the marker is housekeeping only, and stale wazero-* folders are removed.

What is left is the long-term part: pinning, SBOM and provenance. That fits v0.9 (signed releases, CRA) and is tracked together with #1151, #1147 and #1148.

The short-term fix shipped in v0.6.405 (ccm6405). The go:linkname is gone, the marker is housekeeping only, and stale `wazero-*` folders are removed. What is left is the long-term part: pinning, SBOM and provenance. That fits v0.9 (signed releases, CRA) and is tracked together with #1151, #1147 and #1148.
Sign in to join this conversation.
No labels
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Cordy/Cairn#1150
No description provided.