Repository navigation
Let clients invalidate incremental state with an opaque key - #2263
Open
daveinglis wants to merge 1 commit into
Open
daveinglis wants to merge 1 commit into
daveinglis wants to merge 1 commit into
Conversation
The driver decides what to recompile from a build record and a priors file. A build system may have inputs the driver can't see, such as the contents of the SDK. When an SDK is updated in place the sources and the command line are unchanged, so the driver skips the compile and leaves objects built against the old SDK. Add an optional incrementalInvalidationKey to Driver.init. It's an opaque string that goes into the hash the build record stores for the options. If it differs from the previous build's, the driver compiles everything, as it does when the options changed, and records the new key. An empty or missing key leaves the hash as it was, so clients that don't pass one aren't affected and the record format doesn't change. The test harness gets a matching incrementalInvalidationKey, and the tests cover a changed key, the same key, and adding a key to a build that had none.
daveinglis
requested review from
DougGregor,
artemcm,
compnerd and
owenv
as code owners
November 11, 2026 19:18
Author
|
@swift-ci test |
Author
|
@swift-ci test windows |
Author
|
@swift-ci test |
daveinglis
added a commit
to daveinglis/swift-build
that referenced
this pull request
Nov 11, 2026
Updating an SDK in place now produces a new build description, but nothing in it reruns. The SDK's files aren't tracked as dependencies of the compile (clang's -MMD leaves out system headers) and the explicit module scan and the Swift driver's planning filter them out on purpose. The command lines of the compiles, the scan, the planning and the module builds also don't change when the SDK path stays the same. So after an SDK version bump, a build with unchanged sources compiled nothing and kept using modules built from the old SDK headers. Put a signature of the SDK's metadata files into the signatures of these tasks, so a changed SDK reruns them: - Clang: the signature of the compile and scan tasks, with or without explicit modules, and the key of the explicit module build tasks. - Swift: the signature of the driver tasks, and the signature of every driver job through SwiftDriverJob.commandLineSignature, which covers the explicit module jobs and the compile jobs. The graph-less LibSwiftDriver used for thunk info doesn't plan jobs that run, so it passes an empty signature. The signature is SDK.inputsSignature, computed once per task producer context. It covers SDKSettings.plist and, for Apple SDKs, SystemVersion.plist, which the SDK registry's signature now shares through SDK.inputSignaturePaths. SDKSettings.json is left out because the registry only reads the plist. That still leaves Swift's compile jobs, because swift-driver decides what to compile from its own build record, which doesn't track the SDK. Pass the signature to the driver that plans the build as its incrementalInvalidationKey, so the driver recompiles everything when it changed. The driver that is only created to start the dependency scanner doesn't plan jobs, so it doesn't get one. This needs a swift-driver with Driver.init(incrementalInvalidationKey:) (swiftlang/swift-driver#2263), so this doesn't build until that has landed. The tasks are only recreated when the build description is regenerated, which the existing SDK change detection decides from the SDKs in Core's registry. SDKs found elsewhere, such as in the workspace's registry, aren't covered. Checked by hand with a copy of the macOS SDK: after editing a module header and bumping SDKSettings.plist, a C target with explicit modules reran the scan, 7 module builds and the compile, a C target without explicit modules reran the compile and saw the new header, and a Swift target reran the planning, 6 explicit module builds, the module emission and SwiftCompile, and the rebuilt modules contained the new declaration. The builds after it ran nothing. Before this change only ClangStatCache ran. An SDK edited without touching the metadata files is still not detected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The driver decides what to recompile from a build record and a priors file. A build system may have inputs the driver can't see, such as the contents of the SDK. When an SDK is updated in place the sources and the command line are unchanged, so the driver skips the compile and leaves objects built against the old SDK.
Add an optional incrementalInvalidationKey to Driver.init. It's an opaque string that goes into the hash the build record stores for the options. If it differs from the previous build's, the driver compiles everything, as it does when the options changed, and records the new key. An empty or missing key leaves the hash as it was, so clients that don't pass one aren't affected and the record format doesn't change.
The test harness gets a matching incrementalInvalidationKey, and the tests cover a changed key, the same key, and adding a key to a build that had none.