Skip to content

Fix #38: preserve file metadata in artifacts and images - #2215

Closed
LeonxLJX wants to merge 1 commit into
apache:masterfrom
LeonxLJX:fix/38-preserve-file-metadata
Closed

LeonxLJX wants to merge 1 commit into
apache:masterfrom
LeonxLJX:fix/38-preserve-file-metadata

Conversation

@LeonxLJX

Copy link
Copy Markdown

Fixes #38.

Problem

BuildStream's CAS backend was not storing or restoring critical file metadata including full UNIX file mode (especially setuid/setgid bits) and directory mode bits. When artifacts were serialized to CAS and checked out, setuid/setgid bits were lost, breaking bootable images and setuid/setgid tooling.

Solution

Use the Remote Execution API's existing NodeProperties.unix_mode protobuf field to preserve full file mode throughout the storage pipeline:

  1. Capture full mode when adding files to CAS via stat.S_IMODE(os.lstat(path).st_mode)
  2. Serialize mode via node_properties.unix_mode.value for files, directories, and symlinks
  3. Deserialize and store mode in _IndexEntry and expose via FileStat.mode
  4. Restore full mode on CAS checkout (falling back to legacy is_executable for old artifacts)
  5. Use stored mode in export_to_tar() instead of hardcoded values
  6. Fix FileBasedDirectory chmod on import (previously hardcoded 0o755, stripping setuid/setgid)

Files Changed

  • src/buildstream/storage/_casbaseddirectory.py
  • src/buildstream/storage/_filebaseddirectory.py
  • src/buildstream/storage/directory.py
  • src/buildstream/_cas/cascache.py

All changes are backwards-compatible with existing artifacts.

The BuildStream CAS backend was not storing or restoring critical file
metadata, including file permission bits (especially setuid/setgid bits),
when serializing/deserializing directory trees or exporting to tar.

Key changes:
- Add a 'mode' field to _IndexEntry to store full unix file mode (0o7777)
- Add a 'mode' field to the public FileStat API
- Capture full file mode (including setuid/setgid/sticky bits) when adding
  files to the CAS directory tree (via os.lstat + stat.S_IMODE)
- Serialize/unserialize full mode using the protobuf unix_mode field in
  NodeProperties (which is already part of the Remote Execution API)
- Restore full mode on checkout (both files and directories), falling
  back to the legacy is_executable behavior for old artifacts
- Use stored mode in export_to_tar() instead of hardcoded values
- In FileBasedDirectory, read full mode from os.stat_result and restore
  it when importing from CAS-based directories (previously it would
  unconditionally chmod to 0o755 for executable files, stripping setuid/
  setgid)
- FileBasedDirectory.export_to_tar() now uses gettarinfo() which preserves
  full mode (UID/GID are still normalized to 0 for reproducibility)

This ensures that bootable images and tooling relying on setuid/setgid
binaries are correctly produced.
@juergbi

juergbi commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution. However, from a brief look it doesn't seem this has been tested in any real scenario as it doesn't capture mode bits of build outputs (and there are also no tests in this PR). Furthermore, this would require configurability for security and backwards compatibility reasons. This is at least partially discussed in #38.

I.e., as my impression is that this was never tested and it's far from a proper solution to #38, I'm closing this.

If you want to seriously work on fixing #38, the solution should address the aspects mentioned in the issue and tests should demonstrate that it actually works. Possibly with a design proposal before a PR.

@juergbi juergbi closed this Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lost file metadata in artifacts and images

2 participants