Skip to content

fin(ci): Finch impage pull issue; login to Public ECR with Finch as the runner - #9366

Draft
vicheey wants to merge 3 commits into
developfrom
ci/finch-ecr-login-as-runner
Draft

vicheey wants to merge 3 commits into
developfrom
ci/finch-ecr-login-as-runner

Conversation

@vicheey

@vicheey vicheey commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Which issue(s) does this change fix?

No GitHub issue. Fixes the tier1-finch integration failures in the nightly-release run (https://github.com/aws/aws-sam-cli/actions/runs/39232160642):

  • tests/integration/local/execution/test_execution.py::TestLocalExecution::test_tier1_execution
  • tests/integration/local/invoke/test_invoke_durable.py::TestInvokeDurable::test_tier1_durable_invoke
  • tests/integration/local/callback/test_callback.py::TestLocalCallback::test_tier1_callback

Why is this change necessary?

The tier1-finch job logs in to Public ECR with sudo finch login. On Linux, finch login is a plain nerdctl login that writes $DOCKER_CONFIG/config.json or ~/.docker/config.json for the user it runs as. Under sudo that is /root/.docker/config.json.

SAM CLI runs as the runner user and talks to finch-daemon through the Docker SDK, which only reads the runner's Docker client config. finch-daemon never reads a credential file itself; it only uses credentials the client sends. So every image pull in the job went out anonymously, hit the Public ECR rate limit for anonymous pulls, and surfaced as Failed to pull emulator image ... 404 ... no such image. That broke every test that pulls the durable functions emulator image.

How does it address the issue?

The job now logs in the way a non-root Finch user on Linux does, so SAM CLI finds the credentials where it already looks. Only tests/setup_testing_resources.py changes:

  • Create the runner's own ~/.docker/config.json ({}, 0600) if it does not exist, atomically. setup_finch.sh makes finch setuid, so the login itself runs as root. docker/cli keeps the owner and mode of an existing config.json when it rewrites it, so the file stays owned by and readable to the runner. Only a first-time root login creates a root-owned file.
  • Run finch login without sudo. setup_finch.sh already makes finch usable without sudo, and this matches the no-sudo setup the job uses.
  • Fail fast if the config is not readable by the runner after the login, instead of letting the tests pull anonymously.
  • Mask the ECR token in the GitHub Actions log (defensive; it is passed on stdin and was not printed).

The Docker login path is unchanged apart from the token masking.

Integration test run on this branch: https://github.com/aws/aws-sam-cli/actions/runs/39863937466

tier1-finch (https://github.com/aws/aws-sam-cli/actions/runs/39863937466/job/120121288143): 72 passed, 1 failed, down from 3 failures. The login step writes the credentials to /home/runner/.docker/config.json, the emulator image pulls (Successfully pulled image public.ecr.aws/durable-functions/aws-durable-execution-emulator:latest), and no rate-limit or "no such image" errors remain.

Test Before After
test_tier1_durable_invoke ❌ image pull failed ✅
test_tier1_callback ❌ image pull failed ✅
test_tier1_execution ❌ image pull failed ❌ unrelated assertion, see below

Remaining failures in this run (not caused by this change):

What side effects does this change have?

  • CI only. No SAM CLI code changes.
  • On Finch runners, Public ECR credentials are now stored in the runner's ~/.docker/config.json instead of /root/.docker/config.json. The runner is ephemeral.
  • If a future runner image breaks the Finch login (for example, finch is no longer usable without sudo), the setup step now fails clearly instead of the tests failing later with misleading pull errors.

Mandatory Checklist

PRs will only be reviewed after checklist is complete

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…the credentials

`sudo finch login` on Linux writes /root/.docker/config.json, which SAM CLI
running as the runner cannot read, so tier1-finch image pulls went out
anonymously and hit Public ECR rate limits (surfacing as "no such image").

- Create the runner's own ~/.docker/config.json (atomically, 0600) if missing.
  docker/cli keeps the owner of an existing config.json when a root-privileged
  `finch login` rewrites it, so the credentials stay readable by the runner.
- Run `finch login` without sudo (setup_finch.sh makes `finch` usable without
  sudo), and fail fast if the config is not readable afterwards.
- Mask the ECR token in the GitHub Actions log.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant