BuildStream: how extraction writes outside its directory

1 Why fetching sources includes extraction

BuildStream describes build steps and their sources in element files. For a kind: tar source, url identifies the archive and ref records its SHA-256. After obtaining the archive, the tool prepares its files for a content-addressed source cache. Later builds consume that prepared tree, so source fetching includes filesystem work.

Take the pre-fix 2.8.0 commit, 979d31ae9861e8ec1d0a42a7889ce98c78d821f0, as the pinned reference. ElementSources._fetch_source() checks the source cache. When it needs the original source, it obtains the archive and calls SourceCache.commit(). Lines 69–81 of that function create a staging-temp directory, invoke the source plugin there and then import the files into the cache. This temporary directory lives on the host filesystem.

Obtain or reuse the original archive
  → SourceCache.commit()
  → TarSource.stage() in a host temporary directory
  → Import the extracted tree into the source cache
  → Later builds consume that tree

Source._stage() passes a normal directory path to the plugin's stage() method. The extraction itself is in TarSource.stage(), lines 138–159. The upstream regression test for this vulnerability expects failure during bst source fetch; a build command is not required to reach it.

Caching affects whether a particular job reaches this code. An existing content-cache entry or a successful remote source-cache retrieval can avoid local extraction of the original tar archive. A successful warm-cache build therefore leaves the repair untested. Validation needs a path that actually prepares the source, and any node producing a shared source cache belongs in the upgrade inventory as well.

2 One member changes where the next file lands

A tar archive contains an ordered sequence of members. A member can represent a directory, an ordinary file or a symbolic link. The link stores a destination path. When a later filesystem operation encounters it, path resolution continues at that destination. Members created during extraction can therefore change the location reached by later members.

The two members in the diagram are enough to explain the problem. The first is link, pointing outside the staging directory to /outside. The second is link/file.txt. Its name can be joined into /stage/link/file.txt, which begins with /stage. Once the first link exists, opening that path reaches /outside/file.txt. These paths are illustrative examples.

In 2.8.0, _assert_safe() applies abspath() to the joined name and checks that the resulting string starts with the staging directory. abspath() can normalize dot components, but it does not resolve filesystem symlinks. The function separately checks linkname for hard links. Symbolic-link destinations receive no equivalent check. A source comment assumes that links will be harmless once inside a sandbox, overlooking their use by extraction itself.

# Excerpt from BuildStream 2.8.0, _assert_safe()
final_path = os.path.abspath(os.path.join(target_dir, member.path))
if not final_path.startswith(target_dir):
    raise SourceError(...)

The timing of the check adds to the problem. stage() first walks every member, completes its preliminary checks and then passes the whole list to extractall(). On Python below 3.12, that call supplies no extraction filter. The preliminary pass sees a list of names; the eventual write sees a filesystem containing links created by earlier members. A check performed only while constructing the list misses that change.

Version 2.8.0 uses Python's standard library for the actual write. As an illustration of that operation, makefile(), lines 2588–2602, in the implementation bundled with 2.8.1 opens the destination with open(targetpath, "wb") and then copies the contents. This excerpt comes from the repaired release's bundled code. Normal operating-system path resolution applies when the file is opened, and existing files can be overwritten. The process account, containers, mounts and file permissions determine its reach. Host file writes are the confirmed capability in the advisory; any subsequent execution impact depends on the actual destination and how it is used.

Scroll horizontally to read the diagram.

3 Version 2.8.1 checks again before each write

Version 2.8.1 keeps preliminary member preparation, including removal of the base-dir prefix and skipping device nodes, while moving the safety decision into the extraction filter. The repaired stage() always supplies filter=self._extract_filter. For each member, the underlying extractall() calls that filter before performing extraction.

The filter resolves both the destination directory and the proposed write path, then uses commonpath() to require that the latter remains within the former. This part of _get_filtered_attrs() sees symlinks left by earlier members and compares path components when checking containment. An escape raises OutsideDestinationError. The plugin reports the extraction failure as SourceError, and the source is not imported as a successful cache result.

The maintainers also preserve a useful source-tree property: legitimate symbolic links, including some absolute links, must survive extraction. _extract_filter(), lines 201–207, selects tar_filter for symbolic links and the stricter data_filter for other members. The latter adds restrictions on hard-link destinations, file types and attributes. Both branches require the member's own resolved write location to remain inside the extraction directory.

# The branch in BuildStream 2.8.1, _extract_filter()
if member.issym():
    return tarfile.tar_filter(member, target_dir)
else:
    return tarfile.data_filter(member, target_dir)

An absolute symlink remaining in the extracted tree can therefore be an expected result. A later member attempting to write outside through it must be rejected. Upstream retains the valid-link assertions in test_symlinks and adds test_symlink_escape, which expects source fetching to report a path outside the destination. A test that rejects every link would confuse a compatibility break with a successful repair.

The patch covers older Python runtimes too. The import branch uses the standard library on Python 3.12 and later, and BuildStream's bundled _tarfile.py on earlier supported versions. The bundled implementation was imported from Python 3.11.16 and adapted for runtimes without os.path.ALLOW_MISSING. BuildStream 2.8.1 itself requires Python 3.10 or later, so nodes on older interpreters need a runtime migration as part of the update.

4 Identify the Python runtime that actually runs bst

Apache's advisory lists BuildStream through 2.8.0 and says that existing filtering blocks this escape in BuildStream 2.3.0 and later when used with Python 3.12 or later. Our source comparison covers 2.8.0 and 2.8.1; we have not verified every release in that historical range. Retain the advisory's runtime conditions when inventorying systems, and use 2.8.1 or a maintainer-confirmed backport as the acceptance target. The interpreter version alone should not close the issue. The same project configuration can take different extraction paths across workers.

Check versions inside the virtual environment, container or launch script that the CI job actually uses. In the following commands, python must be the interpreter running bst. Another Python found in an interactive host shell says nothing about the job's runtime. Printing the module location also helps expose an old virtual environment in PATH or an unchanged image layer still used by workers.

bst --version
python -c "import sys, buildstream; from importlib.metadata import version; print(sys.executable); print(sys.version); print(version('BuildStream')); print(buildstream.__file__)"

Python has backported extraction filters to some older releases, and its documentation recommends testing hasattr(tarfile, 'data_filter') for feature availability. BuildStream 2.8.0's own branch compares sys.version_info directly. A function being available in the interpreter and the application actually calling it during extraction are separate facts to verify. Python security updates remain important; apply this application's repair according to the BuildStream advisory as well.

Source hashes still do useful work. DownloadableFileSource.track() and fetch() record the hash of new content and check a download against the existing reference. Replacing an already reviewed, pinned archive in transit produces a mismatch and fails fetching. If a project accepts a malicious archive together with its correct hash, integrity verification succeeds; the extractor must still constrain where those bytes can write. Reviewing new ref changes directly controls which content reaches that step.

Distribution packages need a separate check. On September 26, Debian listed bookworm's 1.6.8-2 and trixie's 1.6.9-2 as vulnerable, with forky and sid fixed in 2.8.1-1. Ubuntu listed its 24.04, 22.04 and 20.04 packages as needing evaluation, and 26.04 as not containing the package. Pending maintainer evaluation cannot be reported as a confirmed repair. Avoid mixing a pip installation into the system environment simply to obtain a newer version string. Updating through the existing package channel, or moving to a validated build image, makes dependencies and rollback easier to identify.

The published severity assessments differ. Apache assigns moderate; CISA ADP assigns CVSS 3.1 9.8. At the research cutoff, NVD remained Awaiting Analysis and displayed CISA ADP's 9.8. The September 25 CISA KEV catalog did not include this CVE, and CISA ADP's September 23 SSVC entry in the CVE record said Exploitation: none. The finder published a technical advisory and demonstration program on September 24. These observations describe severity, a known-exploitation catalog and public technical material respectively; they do not establish the number of affected victims. In operational scheduling, prioritize workers that ingest new sources and run with broad write authority.

5 Validate a path that really extracts the archive

After updating the build image or virtual environment, verify its launch identity again and fetch and build a trusted project using a fresh, isolated source cache. Existing caches can remain available to reduce normal migration costs; the validation run needs to exercise extraction. Directory layout, executable bits, hard links and the symbolic links required by the project should retain their expected behavior, and the build result should match the existing baseline.

For teams maintaining their own BuildStream packages, upstream's two link tests provide useful repair checks. Valid links retain their contents; the escape case is rejected during source fetch, with an error identifying a location outside the destination. Run such tests in an isolated environment containing only test directories and no production credentials, and separately verify that sentinel files outside the test extraction directory remain unchanged. The new upstream assertions check command failure and error text. Retaining the unchanged-file observation in package acceptance directly tests whether rejection occurred early enough.

Add a distinct source-integrity check. Pin a trusted archive, modify its test copy and fetch it with another empty cache, requiring failure on a hash mismatch. Then restore the original archive and require normal fetching and building to succeed. This preserves positive and negative checks on source management alongside the extraction-boundary checks.

Temporary containment has concrete costs. Suspending new sources delays dependency updates. Isolating the whole bst process in a container or virtual machine requires reviewing cache, output and credential mounts. A writable host workspace or shared credential mount still gives the process authority over that data. Resume suspended source updates after the permanent repair and acceptance checks pass. If the update fails, retain the restrictions. Roll back only to an image or distribution package confirmed to contain this repair; restoring the older Python and older BuildStream combination would restore the affected path.

A node that processed a suspicious source before the repair also needs investigation. Preserve project reference changes, archive hashes, job logs and filesystem-write records. Focus on build configuration, other projects and credential-related files writable by the job account. Evidence of an actual out-of-directory modification calls for rebuilding the affected node, revoking relevant credentials and reviewing artifacts produced afterward. Where historical records are insufficient, leave that impact unresolved. A successful post-update build cannot answer what an earlier job changed.

The mechanism analysis here uses pinned source, Apache's advisory, the finder's account and upstream regression tests. On Windows with Python 3.12.14, we separately invoked filter functions extracted from the fixed source: ordinary-file and absolute-symlink metadata were accepted, while an outside relative path and outside hard-link metadata were rejected. The local account lacked permission to create a symbolic link, so the dynamic existing-link check was not run. We did not run a complete BuildStream vulnerability reproduction. The finder reports testing on Linux with Python 3.10; that is a separate verification scope.

The patch identifies a concrete place to inspect in build systems: every step that turns external bytes into host files needs to confirm the destination when it writes. Source pinning preserves the selected content, extraction constrains where it can write, and build isolation constrains the commands that follow. Locating each protection at its actual execution stage exposes permissions that would otherwise disappear behind the phrase “the build runs in a sandbox.”