Publish py_image examples and a public library boundary

.bcr/README.md

@@ -21,8 +21,9 @@ bazel run //tools:release -- --output /tmp/rules_layer_optimizer_release --verif
 ```
 
 This creates a deterministic archive, `source.json`, and
-`installation.MODULE.bazel`. Verification runs independent smoke and image
-consumers against the archive. Archives use `git archive` and the exclusions in `.gitattributes`. Local
+`installation.MODULE.bazel`. Verification runs the distroless contract and a
+py_image example against the archive. Archives use `git archive` and the
+exclusions in `.gitattributes`. Local
 preparation packages **committed HEAD**, excluding dirty and untracked changes.
 Commit changes before rehearsing a release. Release automation archives the
 version tag itself and reads its module version. Test consumers omit the version

.github/workflows/ci.yaml

@@ -60,18 +60,6 @@ jobs:
       - if: matrix.module == '.'
         name: Verify source archive installation
         run: bazel run //tools:release -- --output "$RUNNER_TEMP/rules_layer_optimizer_release" --verify
-      - if: matrix.module == 'examples' || matrix.module == 'e2e/minimum'
-        name: Run both Python adapters with both image backends
-        working-directory: ${{ matrix.module }}
-        run: |
-          bazel run //python_image:load
-          docker run --rm --network none rules-layer-optimizer-example:local | grep -F 'hello from reusable layers'
-          bazel run //python_image:img_load
-          docker run --rm --network none rules-layer-optimizer-aspect-img:local | grep -F 'hello from reusable layers'
-          bazel run //rules_python_image:load
-          docker run --rm --network none rules-layer-optimizer-rules-python:local | grep -F 'hello from reusable layers'
-          bazel run //rules_python_image:img_load
-          docker run --rm --network none rules-layer-optimizer-rules-python-img:local | grep -F 'hello from reusable layers'
 
   all-tests-passed:
     runs-on: ubuntu-24.04

CONTRIBUTING.md

@@ -9,16 +9,21 @@ install both hook stages:
 pre-commit install --hook-type pre-commit --hook-type pre-push
 pre-commit run --all-files
 bazel test //...
-(cd examples && bazel test //...)
+(cd examples/py_image_with_rules_oci_and_aspect_rules_py && bazel test //...)
+(cd examples/py_image_with_rules_img_and_rules_python && bazel test //...)
+(cd e2e/layer_plan && bazel test //...)
+(cd e2e/aspect_layers && bazel test //...)
+(cd e2e/rules_python_layers && bazel test //...)
+(cd e2e/image_backends && bazel test //...)
 (cd e2e/minimum && bazel test //...)
-(cd e2e/smoke && bazel test //...)
+(cd e2e/distroless && bazel test //...)
 (cd e2e/inference && bazel test //...)
 (cd e2e/rules_python && bazel test //...)
 ```
 
-The root contains focused unit and analysis tests. The examples module builds
-real package and image archives; the smoke module exercises external-repository
-inference with a small APT fixture. Behavioral tests must accompany rule changes.
+The root contains focused unit and analysis tests. The example modules build
+runnable images. Each package under `e2e/` checks one behavior. Behavioral
+tests must accompany rule changes.
 After committing changes, rehearse archive installation with
 `bazel run //tools:release -- --output /tmp/rules_layer_optimizer_release --verify`.
 This packages committed HEAD, not working-tree edits.
@@ -35,7 +40,8 @@ in that example's package, then review the full diff before committing.
 
 ## Releases
 
-Maintainers update `MODULE.bazel` and the README installation version, then push a matching
+Maintainers update `MODULE.bazel` and the README installation version, then
+push a matching
 `vX.Y.Z` tag to trigger the [Release workflow](.github/workflows/release.yaml).
 GitHub tag protection limits release tags to maintainers and administrators.
 The shared Bazel workflow verifies the archive and publishes the GitHub release

README.md

@@ -1,6 +1,18 @@
 # rules_layer_optimizer
 
-This Bazel ruleset provides building blocks for creating optimized application OCI images based on an executable entrypoint, e.g. a `py_binary` target. It provides dependency inference, reusable package layers, and size-aware OCI layer optimization. It provides everything you need to create your own efficient `py_image` macro.
+This Bazel ruleset provides building blocks for creating optimized application
+OCI images based on an executable entrypoint, e.g. a `py_binary` target.
+
+The following are provided:
+
+- Build separate image layers for each apt or python package, shared across all
+  consuming applications
+- Sort layers by size
+- Flatten the long-tail of small packages, to stay under a configurable layer
+  cap or size threshold
+- Infer apt package dependencies and environment variables
+- Examples for how to combine all of the above into a simple `py_image` macro,
+  for producing an OCI-compatible image from a `py_binary`
 
 ## Building blocks
 
@@ -24,19 +36,48 @@ Add the dependency to your `MODULE.bazel`:
 bazel_dep(name = "rules_layer_optimizer", version = "0.1.0")
 ```
 
-Then, there are 3 basic steps. Each stage is modular, so hand-rolled replacements work fine too.
+Then, there are 3 basic steps. Each stage is modular, so hand-rolled
+replacements work fine too.
 
 ### 1. Produce layer candidates with size hints
 
-This is the step with the most variability, as it depends on which dependencies your repo uses (`rules_oci` | `rules_img`, and `rules_python` | `aspect_rules_py`). Regardless of your choices, the goal remains the same: produce candidate image layers that are efficient and also accompanied by size hints via the `SizeHintInfo` provider:
+This is the step with the most variability, as it depends on which dependencies
+your repo uses (`rules_oci` | `rules_img`, and `rules_python` |
+`aspect_rules_py`). Regardless of your choices, the goal remains the same:
+produce candidate image layers that are efficient and also accompanied by size
+hints via the `SizeHintInfo` provider:
+
+```starlark
+load("@rules_layer_optimizer//python:aspect_rules_py.bzl", "py_image_layer")
+
+py_image_layer(
+    name = "app_layers",
+    binary = ":app",
+    interpreter_tar = ":interpreter",
+    python_toolchain = "@python_interpreters//:current_py_toolchain",
+    strip_prefix = "my/package/app",
+)
+```
+
+With `rules_python`, load `python:rules_python.bzl` and omit `interpreter_tar`
+and `python_toolchain`. Either adapter emits `:app_layers_pip` with
+`LayerTarsInfo` and `SizeHintInfo`, plus unsized interpreter and source tars. A
+hand-rolled rule should return the same providers:
 
 ```starlark
-TODO
+return [
+    DefaultInfo(files = depset([tar])),
+    LayerTarsInfo(tars = [tar]),
+    SizeHintInfo(sizes = {tar: size_bytes}),
+]
 ```
 
 ### 2. Optimize the layers
 
-From the layer candidates, group and optimize them with `optimized_layers`. Note that `SizeHintInfo` is not required on all candidates, the rule will use the hints when available but they only matter for groups where in-group reordering and flattening is desired:
+From the layer candidates, group and optimize them with `optimized_layers`. Note
+that `SizeHintInfo` is not required on all candidates, the rule will use the
+hints when available but they only matter for groups where in-group reordering
+and flattening is desired:
 
 ```starlark
 load("@rules_layer_optimizer//layers:defs.bzl", "layer_group")
@@ -57,29 +98,59 @@ optimized_layers(
 
 ### 3. Build the image
 
-The output of `optimized_layers` can be fed directly into `oci_image` or `image_manifest`:
+The output of `optimized_layers` can be fed directly into `oci_image` or
+`image_manifest`:
 
 ```starlark
-TODO
+load("@rules_oci//oci:defs.bzl", "oci_image")
+
+oci_image(
+    name = "image",
+    base = "@ubuntu",
+    entrypoint = ["/app"],
+    env = ":env",
+    tars = [":layers"],
+)
+```
+
+`rules_img` takes the same archives as `layers` and the environment file as `env_file`:
+
+```starlark
+load("@rules_img//img:image.bzl", "image_manifest")
+load("@rules_img_images.bzl", "image")
+
+image_manifest(
+    name = "image",
+    base = image("ubuntu"),
+    entrypoint = ["/app"],
+    env_file = ":env",
+    layers = [":layers"],
+)
 ```
 
 ## Try it locally
 
-The [examples module](examples/README.md) demonstrates tar optimization, Python OCI images, and
-package-sharing checks. Both Python adapters are tested with `rules_oci` and
-`rules_img`.
+The [examples](examples/README.md) are two modules that wrap these pieces in a
+macro taking only `name` and a `py_binary`: one with rules_oci and
+aspect_rules_py, and one with rules_img and rules_python.
 
 ```sh
 bazel test //...
-(cd examples && bazel test //...)
-(cd e2e/minimum && bazel test //...)
-(cd e2e/smoke && bazel test //...)
-(cd e2e/inference && bazel test //...)
-(cd e2e/rules_python && bazel test //...)
+(cd examples/py_image_with_rules_oci_and_aspect_rules_py && bazel test //...)
+(cd examples/py_image_with_rules_img_and_rules_python && bazel test //...)
 ```
 
 Start with [configuration](docs/configuration.md) and the capability docs:
 [layers](layers/README.md), [inference](inference/README.md),
 [Python](python/README.md), and [APT](apt/README.md).
 See [compatibility](docs/compatibility.md) for supported dependency minimums and
 [CONTRIBUTING.md](CONTRIBUTING.md) for development checks.
+
+## Future work
+
+The optimizations provided by this ruleset are not useful only for Python. The
+same strategy can be applied to any other language whose third-party packages
+remain intact in the final runtime tree. A shared package directory, such as a
+Java archive, a `node_modules` tree, or a Ruby gem, can be its own reusable
+image layer. C++ and Rust are unsuitable: compilation folds dependencies into
+the binary, so no package boundary remains to share across images.

apt/README.md

@@ -5,9 +5,10 @@
 `tars`. Consumers select distributions and package repositories themselves.
 Use `python:inference.bzl` for the variants that match logical pip packages.
 
-`inferred_apt_deps` combines matched mappings with explicit `base_packages` and
-`packages`. Outputs carry LayerTarsInfo plus SizeHintInfo for primary package
-tars. Transitive package tars are retained and may remain unsized. Duplicate
+`inferred_apt_deps` accepts `layer_inference_bundle` targets. It combines matched
+mappings with explicit `base_packages` and `packages`. Outputs carry LayerTarsInfo
+plus SizeHintInfo for primary package tars. Transitive package tars are retained
+and may remain unsized. Duplicate
 files are removed and conflicting hints fail analysis.
 
 The adapter identifies a package's primary tar through its public `:data`
@@ -21,7 +22,7 @@ generates v2 locks or supply an existing compatible lock. Tests use synthetic
 package repositories rather than live resolution.
 
 See [configuration](../docs/configuration.md) for module setup. The
-[smoke module](../e2e/smoke/README.md) verifies the adapter contract without a
+[distroless contract](../e2e/distroless/README.md) verifies the adapter contract without a
 production APT universe. Base packages, compatibility shims, dpkg status policy,
 and distribution selection are deliberately consumer-owned.
 
@@ -33,6 +34,6 @@ registers a mapping and a private `<name>_for_deps_test` that validates trigger
 The Python variant derives `for_keys` itself from the configured hub.
 
 `inferred_apt_deps(name, deps = [], inferences = [], base_packages = [], packages = [], **kwargs)`
-collects base packages, matching mappings, then explicit packages in that order.
+takes `layer_inference_bundle` targets in `inferences`. It collects base packages, matching mappings, then explicit packages in that order.
 It exposes `DefaultInfo`, `LayerTarsInfo`, and `SizeHintInfo`.
 Both APIs accept ordinary Bazel attributes such as `visibility` and `tags`.

apt/defs.bzl

@@ -1,6 +1,7 @@
 """Public apt building blocks."""
 
-load("//apt/private:inferred_apt_deps.bzl", _apt_inference = "apt_inference", _inferred_apt_deps_rule = "inferred_apt_deps_rule")
+load("//apt/private:inferred_apt_deps.bzl", _apt_inference = "apt_inference", _inferred_apt_deps_rule = "inferred_apt_deps_rule", _make_inferred_apt_deps = "make_inferred_apt_deps")
 
 apt_inference = _apt_inference
 inferred_apt_deps = _inferred_apt_deps_rule
+make_inferred_apt_deps = _make_inferred_apt_deps

apt/private/apt_size_hints.bzl

@@ -74,3 +74,27 @@ def render_apt_size_hints_bzl(size_hints):
 
     lines.append("}")
     return "\n".join(lines) + "\n"
+
+def _apt_size_hints_repository_impl(rctx):
+    size_hints = {}
+    for index in range(len(rctx.attr.locks)):
+        lock_hints = parse_apt_lock_size_hints(
+            rctx.read(rctx.attr.locks[index]),
+            rctx.attr.dependency_sets[index],
+            rctx.attr.repositories[index].repo_name,
+        )
+        for package, size in lock_hints.items():
+            previous_size = size_hints.get(package)
+            size_hints[package] = max(previous_size or 0, size)
+
+    rctx.file("BUILD.bazel", "exports_files([\"size_hints.bzl\"])\n")
+    rctx.file("size_hints.bzl", render_apt_size_hints_bzl(size_hints))
+
+apt_size_hints_repository = repository_rule(
+    implementation = _apt_size_hints_repository_impl,
+    attrs = {
+        "dependency_sets": attr.string_list(),
+        "locks": attr.label_list(allow_files = [".json"]),
+        "repositories": attr.label_list(),
+    },
+)

apt/private/distroless.bzl

@@ -1,9 +1,7 @@
 """rules_distroless adapter for APT package layer metadata."""
 
 load("//apt/private:apt_utils.bzl", "apt_package_size_hint")
-
-# buildifier: disable=bzl-visibility
-load("//layers/private:layer_groups.bzl", "SizeHintInfo")
+load("//layers:providers.bzl", "SizeHintInfo")
 
 def _primary_files_for_package(target, ctx):
     """Extract the primary tar exposed by a rules_distroless package target."""

apt/private/inferred_apt_deps.bzl

@@ -7,15 +7,9 @@ mappings that matched.
 """
 
 load("//apt/private:distroless.bzl", "apt_size_hint_aspect")
-
-# buildifier: disable=bzl-visibility
-load("//inference/private:dependency_inference.bzl", "dependency_graph_aspect", "dependency_identities", "inference_matches", "validate_inference_triggers")
-
-# buildifier: disable=bzl-visibility
-load("//inference/private:inferred_layers.bzl", "LayerInferenceBundleInfo", "LayerInferenceInfo")
-
-# buildifier: disable=bzl-visibility
-load("//layers/private:layer_groups.bzl", "LayerTarsInfo", "SizeHintInfo", "merge_size_hints")
+load("//inference:graph.bzl", "dependency_graph_aspect", "dependency_identities", "inference_matches", "validate_inference_triggers")
+load("//inference:providers.bzl", "LayerInferenceBundleInfo", "LayerInferenceInfo")
+load("//layers:providers.bzl", "LayerTarsInfo", "SizeHintInfo", "merge_size_hints")
 
 def _apt_inference_rule_impl(ctx):
     files = [p.files for p in ctx.attr.packages + ctx.attr.tars]
@@ -149,7 +143,10 @@ def make_inferred_apt_deps(dependency_aspects = []):
         implementation = inferred_apt_deps_rule_impl,
         attrs = {
             "deps": attr.label_list(aspects = [dependency_graph_aspect] + dependency_aspects),
-            "inferences": attr.label_list(default = []),
+            "inferences": attr.label_list(
+                default = [],
+                providers = [LayerInferenceBundleInfo],
+            ),
             "base_packages": attr.label_list(default = [], aspects = [apt_size_hint_aspect]),
             "packages": attr.label_list(default = [], aspects = [apt_size_hint_aspect]),
         },
@@ -166,16 +163,5 @@ def _inference_triggers(for_deps):
 def _inference_entries(ctx):
     entries = []
     for inference in ctx.attr.inferences:
-        if LayerInferenceBundleInfo in inference:
-            entries.extend(inference[LayerInferenceBundleInfo].entries)
-        elif LayerInferenceInfo in inference:
-            sizes = inference[SizeHintInfo].sizes if SizeHintInfo in inference else {}
-            entries.append(struct(
-                for_deps = tuple(inference[LayerInferenceInfo].for_deps),
-                for_keys = tuple(inference[LayerInferenceInfo].for_keys),
-                files = inference.files,
-                sizes = sizes,
-            ))
-        else:
-            fail("{} is not a layer_inference or layer_inference_bundle target".format(inference.label))
+        entries.extend(inference[LayerInferenceBundleInfo].entries)
     return entries

apt/size_hints.bzl

@@ -0,0 +1,5 @@
+"""Public APT lock size-hint repository rule."""
+
+load("//apt/private:apt_size_hints.bzl", _apt_size_hints_repository = "apt_size_hints_repository")
+
+apt_size_hints_repository = _apt_size_hints_repository

docs/README.md

@@ -1,11 +1,11 @@
 # Documentation
 
-- [Configuration](configuration.md): module setup and package size metadata.
+- [Configuration](configuration.md): module setup, package sizes, and apt and environment mappings.
 - [Compatibility](compatibility.md): supported dependencies and current limits.
 - [Layers](../layers/README.md): providers, ordering, and budget decisions.
 - [Inference](../inference/README.md): exact-label and Python-aware matching.
 - [Python](../python/README.md): reusable wheels and source layers.
 - [APT](../apt/README.md): package metadata and inference.
-- [Inference-only consumer](../e2e/inference/README.md): package matching without size hints.
-- [Examples](../examples/README.md): executable consumer configurations.
+- [Examples](../examples/README.md): two-argument `py_image` modules.
+- [End-to-end checks](../e2e/image_backends/README.md): one directory per behavior, including the [inference-only consumer](../e2e/inference/README.md).
 - [Release preparation](../.bcr/README.md): archive verification and release metadata.

docs/compatibility.md

@@ -14,9 +14,9 @@ use Bazel 9.2.0. Other execution platforms are not yet claimed as supported.
 | `rules_python` | **1.9.0** | 1.9.0 |
 | `aspect_rules_py` | **2.0.0-alpha.4** | 2.0.0-alpha.4 |
 
-The minimum combination and current endpoints run the same
-[consumer tests](../examples/README.md), including all four Python/image-backend
-combinations in Docker. The [minimum module](../e2e/minimum/README.md) pins exact
+The current [image backend checks](../e2e/image_backends/README.md) and the
+[minimum module](../e2e/minimum/README.md) run all four Python/image-backend
+combinations in Docker. The minimum module pins exact
 versions with root-only overrides and verifies the resolved module graph. This
 prevents transitive version upgrades from invalidating minimum-version coverage.
 These are supported floors, not a claim that every future release has been tested.