Repository navigation
Integrate and qualify Inspect AI on current 0.7 (Pierre handoff) #342
Description
The user has transferred full Inspect AI integration to Pierre (@tholop). This work is removed from the current agent's active Sprinty scope. This issue owns the integration and its qualification; the partial carry is not a finished sandbox.
Base and source to reuse
Work against the shared 0.7 branch. Commit 86de47d77d5de45417b78c1955a2f056f4a4c337 merges current main b862c89d8363fb2ffa09a870378122aa3e8fb532 and all the committed OCI/client work through 224130ed1. Fetch and pull the latest shared branch before integration; the normal push to shared 0.7 has completed.
The repository already has these implementation branches. Inspect their ancestry/diffs and integrate them in this order, resolving against current 0.7 rather than blindly duplicating the old integration:
- feat(inspect-ai): add VM-mode Capsem SandboxEnvironment integration #338,
feat/inspect-capsem, tip6f9656f7b: VM SandboxEnvironment, controller/exec/files/lifecycle/transfer, tools, registration/package metadata, SDK/gate/CI/docs routing and tests. - feat(inspect-ai): add OCI container execution mode and Compose parser #339,
feat/inspect-capsem-containers, tip99a088b39: OCI execution mode, Compose parser/fields, workload controller, file/exec targeting and acceptance. - feat(inspect-ai): add host-side Dockerfile and Compose build image support #340,
feat/inspect-capsem-host-build, tipda3aea14c: host Dockerfile/Compose image builder and OCI registry handoff candidate. This source is a candidate, not authorization for implicit host execution/credentials/network access.
Original full design/source: #291, commit bb61fc82d44bc42c978c4acf5435a1975600c75e. Preserve Pierre's source provenance when adapting it. Related tracking: #310 (VM/workload), #311 (full Dockerfile/Compose), parent #289.
What is already on 0.7
integrations/inspect-ai/ currently contains pure Dockerfile and Compose components, without an installed package/Inspect registration:
- Dockerfile instruction scanning, heredoc handling/lowering, multistage CA patching, final-stage USER/WORKDIR metadata. Carry commit
1cec4f2c9. - Compose single-service field extraction, interpolation/.env behavior, build context/args/target, command/entrypoint/user/workdir, mounts/ports, healthcheck/init/memory/network intent.
ComposeInputssupplies immutable evaluator-selected text/environment/directory facts;ComposeLimitsbounds bytes/nodes/depth and YAML/merge/alias/interpolation expansion. Carry commitf7a697c26. - These overlaps must be reconciled with the newer branches above, retaining the input bounds and redaction instead of creating competing parser implementations.
The Python and TypeScript SDKs already expose Hypervisor.images.list/pull and explicit VM.exec targets. Python target commit 9c6a0b973, TS 224130ed1. Existing service APIs admit/pull immutable OCI images, create workloads and execute as the workload user/cwd/environment; explicit target=vm is for VM operations. Current file APIs are workspace-scoped with a 10 MiB request/file limit. Ordinary client VM.close() closes a connection; it does not destroy a VM.
Integration order
- Reconcile VM-mode controller, configuration, SandboxEnvironment registration and packaging from the VM branch against current generated SDK DTOs. Use the SDK/gateway, explicit IDs, and evaluator-selected inputs.
- Complete owned-session cleanup and target-correct exec/file operations. Attached/named/unrelated sessions must survive cleanup. Exercise timeout, cancelled creation/lost responses, nonzero execution and process death. Internal managed-ownership primitives exist, but no new claim/close endpoint or lease policy was approved.
- Integrate native OCI/Compose launch behavior against actual ContainerSpec and ExecRequest capabilities. Preserve real refusal/cancellation behavior; no retry into VM-root exec after workload denial.
- Complete the selected Dockerfile/Compose build path, with a reviewed execution/context/network/registry/credential authority boundary. Reuse the host-build candidate if appropriate, but do not infer permission from the presence of its source. For a guest builder, ordinary runc workloads currently deny namespace/mount delegation and do not ship Docker/BuildKit; installing an engine alone does not establish a supported builder.
- Connect the package, docs and existing CI/gate owners, then qualify the entire installed integration on both native architectures. Coordinate genuine missing native controls with the service owner and regenerate shared DTOs/SDKs together where required.
Confirmed image/cache scope from the user
images.list is the registry catalog: images potentially available from the configured registry. It may include truthful local disk status and a VM-use count. Cache verification, reuse/download and protecting data referenced by running VMs remain service-owned through existing pull/create paths.
Do not make Inspect integration depend on separate image-cache inventory/removal endpoints, cache heartbeats, or a public cache-only pull mode. The old combined four-route cache/managed-session proposal is withdrawn from this scope. A cached image never substitutes for admission of its source/resolved digest.
Required behavior and proof
Preserve the full integration rather than closing this with an image-only subset:
- VM and OCI exec, files, tools, readiness, resource settings, owned cleanup and exact error/output semantics.
- Actual Docker frontend build behavior: context/.dockerignore, build args/target, FROM/scratch/multistage/COPY --from, shell/JSON RUN and heredocs, COPY/ADD ownership/mode, USER/WORKDIR/ENV/ENTRYPOINT/CMD.
- Compose precedence/interpolation and explicit overrides; bind files/directories, named volumes, protocol/port handling, healthcheck, init, memory and network modes. Preserve documented refusals for unsupported shapes.
- CA bundle preservation, stage propagation, browser/NSS trust where applicable, CA-rotation/corruption cache invalidation, and build-result reuse.
- Evaluator controls all host inputs and credential/network grants. No ambient secret discovery, global policy edits, host Docker socket mounts, private runc/bundle/cgroup rewrites, or deletion by guessed VM name.
- Real installed Inspect task proof through the authenticated gateway and workload, with functional output, cleanup/resource evidence and ledger attribution. Parser tests and mocked controller tests prove their components only. Acceptance is hermetic, not an optional public-network/live flag.
During implementation use TDD and minimal focused local tests. No global machine lock or repeated broad workspace/VM/install/benchmark runs. Full qualification belongs at the major phase boundary and before release. Current 0.7 is development source: phase/runtime/release qualification is still outstanding.
Focused checks for the currently carried helpers:
python3 build_system/scripts/ci/run-bounded-command.py --timeout-seconds 60 -- \
env PYTHONPATH=integrations/inspect-ai uv run --project build_system --frozen \
python -m pytest -c build_system/pyproject.toml --rootdir . \
integrations/inspect-ai/tests/containers -qExpected closeout: one working installed integration on current 0.7, coherent source/provenance, native/packaged parity for the required features, and complete recorded end-to-end qualification.
Activity
docs(inspect): hand integration ownership to Pierre
feat(inspect-ai): add OCI container execution mode and Compose parser
test(inspect-ai): add hermetic host-build live VM acceptance to gate
tholop commentedon Oct 9, 2026
Update: all 6 PR branches in the stack are rebased onto current
0.7(#291 is now closed in favor of #342 and this stack). The 3 service/SDK prerequisite PRs (#341, #336, #337) are ready for review first, while the 3inspect-capsemintegration PRs (#338, #339, #340) are pushed as Draft so the prerequisites can land on0.7first:0.7(ready for review, land first):feat/shared-vm-name-rule->0.7): shared VM name validation rule.feat/sdk-vm-labels->0.7): VM labels oncreate/fork/list(used byinspect-capsemowned-session cleanup).feat/sdk-typed-helpers->feat/sdk-vm-labels):ErrorCodeenum,CreateTimeoutError/ExecTimeoutError/VmNotFoundError,discover_gateway/Hypervisor.connect.feat/inspect-capsem->feat/sdk-typed-helpers, Draft): VM-modeSandboxEnvironment, owned-session cleanup, andlinux/x86_64installed-package qualification.feat/inspect-capsem-containers->feat/inspect-capsem, Draft): native0.7OCI container execution mode and reconciled Compose parser.feat/inspect-capsem-host-build->feat/inspect-capsem-containers, Draft): default-off, operator-granted hostdocker build(HostBuildGrant) + hermetic live gate acceptance.linux/x86_64installed-package qualification passed across wheel and sdist (SDK_IMAGE_PACKAGE_ACCEPTANCE_OK,INSPECT_CAPSEM_CONTAINER_ACCEPTANCE_OK,INSPECT_CAPSEM_HOST_BUILD_ACCEPTANCE_OK,INSPECT_CAPSEM_VM_ACCEPTANCE_OK,0leaked sessions).darwin/arm64qualification is queued on themacos-15CI lane from this push.0.7:test-installfails across0.7PRs because the stable release manifest has noruntimeentry yet.test(inspect-ai): add hermetic host-build live VM acceptance to gate
test(inspect-ai): add hermetic host-build live VM acceptance to gate