Repository navigation
Add --DRT-testfilter and --DRT-testmode=list to the default unittest runner - #23981
atilaneves wants to merge 2 commits into
Conversation
DMD perf checkNo differences above the noise thresholds. All measurementsdc8b891 vs merge-base f10507a · about these metrics · historical dashboard |
|
I would suggest mangling and not this: Due to things like template instantiations, it's going to be useless, and all ways to get the name anyway should provide you with the mangling as well. |
|
How is this intended to be used in a real world project? |
The idea is to turn on support for running tests by name, and then to add support for actually naming them something reasonable separately. |
Like in unit-threaded: users give their tests names and use those names to run specific tests. Having testing support in the language is great, but the out-of-the-box experience isn't. There's a reason silly and unit-threaded are two of the most popular dub projects, and that a DConf 2026 talk had a Google Test inspired test runner. |
I get that, I should have been more precise in my question. I'm asking specifically about the user experience and integration with tools. Are users supposed to manually type |
Can this be done in a separate PR closing an issue? |
|
What about adding a flag that outputs execution time of individual unittests upon exit? |
e23414a to
cb1bde1
Compare
I don't know how else to do it. Suggestions welcome.
Temporarily, the idea is to merge this first then another PR would allow users to give their tests "proper" names.
Aren't those orthogonal? By which I mean: if dub is going to pass these anyway, does it matter that much how ergonomic the flags are? |
|
Key question is output formatting and whether timings of individual unittests should be reported. See master...nordlow:dmd:unit-test-duration-report. No tests added yet. |
|
Dependant PR merged. This needs a rebase now. |
cb1bde1 to
688bb8d
Compare
688bb8d to
dc8b891
Compare
There was a problem hiding this comment.
I don't know how else to do it. Suggestions welcome.
A comma separated list of mangles of unittests to run.
Aren't those orthogonal? By which I mean: if dub is going to pass these anyway, does it matter that much how ergonomic the flags are?
It matters because this PR adds 200 lines of new flag parsing code in druntime, exposed as a public API so it will have to be maintained for at least a deprecation/edition cycle. I prefer to keep druntime small, and I'm not sure we need to commit this filter logic as one of druntime's responsibilities. If experience shows this is desirable, then we can always add it. But I can already see this spiral into "oops we now have a duplicate regex engine in druntime for test filtering, if only we left filter syntax as the build system's responsibility". That's why I'd like to see rationale for specifically this PR's --DRT flag api, or alternatively, label it explicitly as experimental so we can change it later.
…runner --DRT-testfilter takes a comma-separated list of package, module or unittest names, matched on whole dot-separated components, with a `-` prefix to exclude. A name matching nothing, an empty selection, or use with a per-module --DRT-testmode is an error. --DRT-testmode=list prints the names of the (selected) unittests without running them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0042528 to
4898c6a
Compare
I simplified it. |
Summary
Adds exact unittest selection and listing to the default per-unittest runner.
Names consist of the module name followed by the compiler-generated unittest name, as printed by
list:The list accepts exact names only. Package/module selection, prefix matching, and exclusions are not supported. Names are not mangled symbols; user-defined names are intended as separate future work.
Unknown names and empty entries are errors, reported before any tests run. An empty option value means no filtering. Duplicate names run the test once, in the runner's usual order.
--DRT-testmode=listprints one name per line on stdout without running tests or main. Combining it with--DRT-testfilterlists only the selected tests. Filtering cannot be combined withrun-main,test-only, ortest-or-main.Modules without individual unittest metadata cannot be listed or selected; unfiltered execution retains its existing fallback. Custom handlers set through
Runtime.moduleUnitTesterorRuntime.extendedModuleUnitTesterare unaffected.Evidence
After: It accepts only comma-separated exact unittest names. Tests cover single and multiple names, duplicate names, failures, listing, empty values, malformed lists, unknown names, and incompatible modes.
test/unittest/.runsuite passes with release and debug runtime builds on Linux x86-64.Merge Danger
Door: Two-way for implementation changes; the new command-line interface introduces a compatibility commitment.
Blast Radius: Druntime. Selection and listing affect the default unittest runner when explicitly requested.