Repository navigation
[Caching] Make batch partitions independent of -j under caching - #2250
Dharin-shah wants to merge 2 commits into
Conversation
Each batch's frontend command line lists its primary files, and the compilation cache key of a batch job includes them. The number of batch partitions is max(-j, ceil(files / 25)) unless -driver-batch-count is given, and build systems pass -j with the machine's core count, so two machines with different core counts split every module of more than one file into different batches, and none of those batches share a key. A cache warmed on a 6-core machine misses on a 12-core machine for every batched Swift compile. When caching is on and -j is given, use a fixed minimum partition count in place of the value of -j, so that batch composition depends on the inputs alone. -driver-batch-count still overrides it, without -j the count stays one per 25 files, and without caching nothing changes.
cachemeifyoucan
left a comment
cachemeifyoucan
left a comment
There was a problem hiding this comment.
Generally good but the reason why it wasn't done already is the worry about performance impact. It will be good to have number to justify that (from batch for modules of small number of files to large number of files and how much a difference a default number that is not core count matters).
| /// The minimum number of batch partitions when compilation caching is on | ||
| /// and -j is given, in place of its value, so that batch composition, and | ||
| /// with it every batch's cache key, is the same on every machine. | ||
| @_spi(Testing) public static let defaultCachingBatchCount = 6 |
There was a problem hiding this comment.
Any reason behind the magic number 6? Since any number will work here. Any concrete perf number?
Otherwise, we might just default to divideRoundingUp(numInputFiles, sizeLimit)? Cache replay prefers larger batches (but it might add serialization for the initial build).
There was a problem hiding this comment.
I added the numbers in the benchmark comment, but basically, any fixed number works for the cache keys, so it's just about speed. On an 18-core Mac, 6 made a 250-module app about 10% faster than following the core count, and only cost time when one mid-sized module builds on its own (about 15%, a second or two). A bigger number like 12 mostly removes that cost but also most of the app-level win, so 6 seemed like the better trade, and It's also the value Tuist picked after hitting this (tuist/tuist#15060, #15067), so the two would line up.
On divideRoundingUp alone, hmm I think it would hurt the initial build more, since every module under 25 files would compile in a single process, but I haven't measured it directly. Happy to run it if you'd like. On replay, I didn't see any difference between 6 and 18 batches.
| let sizeLimit = info.sizeLimit ?? defaultSizeLimit | ||
|
|
||
| let numTasks = numParallelJobs ?? 1 | ||
| // With compilation caching, the primaries of a batch are part of its |
There was a problem hiding this comment.
This comment is too long unless you are trying to rival the original comment. It only need one sentence to explain why caching wants a fixed partition not affected by core counts. If you want to write more, you need to justify how you pick a default number.
3d97bf1 to
ad23b5c
Compare
|
Working on getting the numbers.. |
|
Thanks for taking a look! Here are some numbers. Setup: M5 Pro (18 cores), Xcode 27.2, Debug simulator builds with caching and prefix mapping on. Since 27.2's driver doesn't have the patch, I passed How to read the tables: the driver splits a module's files into batches and compiles them in parallel. Today the number of batches follows the core count, so 18 on this Mac. The patch uses 6, or one batch per 25 files once a module is big enough (8 at 200 files, 12 at 300). "With 12" is the same idea with 12 instead of 6, for comparison. The percentages are the change in build time against today: + means slower, − means faster. One module on its own, cold cache:
A 250-module app (4.6k files), cold cache: 217.8 s today, 196.5 s with the patch, so 10% faster and 9% less CPU. With 12 it's only 3% faster. Fully cached builds take the same time whether the module is split into 6, 12 or 18 batches, at every size, so replaying fewer, bigger batches doesn't cost anything extra. Any fixed number works for the cache keys, so it really comes down to speed, and I think the worry is real but narrow. The patch only costs time when one mid-sized module (roughly 50 to 450 files) builds on its own on a machine with many more cores than batches: about 15% slower here, 1 to 3 s. In a normal multi-module project the other modules keep the cores busy and fewer batches means less duplicated work, so it ends up faster. Past 450 files the 25-file limit takes over and nothing changes. A bigger constant like 12 removes the single-module cost but gives back most of the project win, so I'd keep 6. It's also the value Tuist picked after hitting this (tuist/tuist#15060, #15067), so the two would line up. A flat pin like that skips the 25-file limit, though, and costs a 1,000-file module 62%; the patch doesn't. |
|
A 10% slow down is quite significant. Also note your number is specific to a 18 core machine, where a different core count might result in different numbers. I would be nice to do a thorough search of the space to make you pick a good number. In the meantime, you can specify |
let me test it on a 8 core one , will get back |
Was looking into cache misses across machines with different core counts. I believe the issue is that the batch count follows
-j, so a 6-core and a 12-core Mac split modules differently and never share a batch key 📉The fix uses a fixed minimum of 6 under caching instead of
-j, the value Tuist already pins.-driver-batch-countstill wins and non-caching builds don't change.Haven't built it locally, so leaning on CI 🙏
Resolves #2249.