Skip to content

proccontrol: restore the tinfo entry that makes pc_irpc post its IRPCs - #280

Merged
bbiiggppiigg merged 2 commits into
masterfrom
bbiiggppiigg/pc-irpc-restore-tinfo
Sep 22, 2026
Merged

bbiiggppiigg merged 2 commits into
masterfrom
bbiiggppiigg/pc-irpc-restore-tinfo

Conversation

@bbiiggppiigg

@bbiiggppiigg bbiiggppiigg commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Problem

pc_irpc never completes. It hangs on the first of its 64 configurations and produces
zero result rows, so it contributes no coverage on any platform. In isolated per-test
runs of current master it is recorded as HUNG on x86_64, aarch64 and ppc64le alike — it was
the one test hung on all four of our lab machines.

Root cause

runIRPCs() posts IRPCs by iterating the global tinfo map:

for (std::map<Thread::const_ptr, thread_info_t>::iterator i = tinfo.begin();
     i != tinfo.end(); i++)
{
   ...
   post_irpc(thr);
}

tinfo is cleared at the top of every runIRPCs() call, so an entry has to be created for
each user thread before that loop runs. #258 removed the line that did it, from the
IRPC-creation loop:

          Thread::ptr thr = *j;
-         thread_info_t &t = tinfo[thr];
          if(!thr->isUser())

The variable t was genuinely unused and the warning was correct, but the initializer was
not dead: std::map::operator[] default-constructs and inserts the entry. With the line
gone tinfo stays empty, the posting loop body never executes, and no IRPC is ever posted.

runIRPCs() then cannot terminate:

  • has_pending_irpcs() is false, because it only reports RPCs with posted && !completed
    and nothing was posted;
  • all_irpcs_completed() is false, because the rpc_data_t objects were created in pinfo
    and none of them ever completes;
  • so the inner while (has_pending_irpcs()) is skipped, the break is never taken, the
    post_sequential re-post loop iterates the same empty tinfo, and the outer
    while (!myerror) spins on the CPU forever.

Observed directly under gdb: the driver's main thread sits in has_pending_irpcs()
(pc_irpc.C:117, called from runIRPCs at line 399) in state R, while the mutatee is
stopped under ptrace.

Fix

Create the entry explicitly, after the isUser() check so that system threads stay out of
tinfo — the post_sequential re-post loop iterates tinfo without an isUser() check of
its own, which is what the checks added in 76402cb were meant to prevent. Being a statement
whose only purpose is the insertion, it can no longer be mistaken for a dead local.

Verification

Isolated runs (test_driver -test pc_irpc -all) against dyninst master 7e53b6c88:

before after
result rows 0 32
variants passed 0 32 / 32
configurations exercised 1, then spun 2048
  • x86_64 (Ubuntu 20.04, gcc-11): 32/32 PASSED, and deterministic — 3/3 repetitions,
    zero bad reps.
  • aarch64 (Ubuntu 20.04, gcc-11): 32/32 PASSED.
  • ppc64le (POWER9, Ubuntu 20.04): improves from 0 rows to 11 (8 PASSED, 512
    configurations exercised), but does not finish the matrix — the multithreaded create
    variants fail in Group Setup and a later one still hangs. Details in the comment below.
    Those failures were unreachable before this change and look like the platform's known
    proccontrol thread-create weakness, not something this patch introduces.

On x86_64 and aarch64 all 32 variants pass: g++ and gcc, create and attach,
SPST/MPST/SPMT/MPMT, dynamic and static linkage. Only pc_irpc.C changes, so no other test
is affected.

🤖 Generated with Claude Code

runIRPCs() posts IRPCs by iterating the global tinfo map, and re-posts from
that same map in the post_sequential path. tinfo is cleared at the top of
every runIRPCs() call, so an entry has to be created for each user thread
before the posting loop runs.

PR #258 removed

    thread_info_t &t = tinfo[thr];

from the IRPC-creation loop as an unused variable. The variable was indeed
unused, but the initializer was not: std::map::operator[] default-constructs
and inserts the entry. Without it tinfo stays empty, the posting loop body
never executes, and no IRPC is ever posted. runIRPCs() then spins forever in
its wait loop -- has_pending_irpcs() is false because nothing was posted,
while all_irpcs_completed() is false because the rpc_data_t objects exist in
pinfo and none of them ever completes -- so while (!myerror) never exits.

pc_irpc therefore hung on the first of its 64 configurations on every
platform, producing no result rows at all. Verified on coriander (x86_64):
32/32 variants PASSED, 2048 configurations exercised, where before the test
emitted a single "Running:" line and then spun.

Create the entry explicitly with emplace(), after the isUser() check so that
system threads stay out of tinfo (the post_sequential re-post loop iterates
tinfo without an isUser() check of its own). Being a statement whose only
purpose is the insertion, it can no longer be mistaken for a dead local.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bbiiggppiigg

Copy link
Copy Markdown
Member Author

ppc64le result: big improvement, but not a clean pass — separate pre-existing problems surface

POWER9, Ubuntu 20.04, system gcc, dyninst master 7e53b6c88, this branch at 46317e92,
isolated test_driver -test pc_irpc -all under a 900 s cap:

before (master testsuite) after (this branch)
result rows 1 (just the HUNG marker) 11
variants passed 0 8
Running: configurations exercised 1, then spun 512

So the fix does what it claims here too: the test gets off the ground instead of spinning on
configuration 1, and the 8 variants that do run each complete all 64 configurations.

But ppc64le does not finish the matrix:

pc_irpc  g++  none 64  create   SPMT  dynamic nonPIC  FAILED (Group Setup)
pc_irpc  g++  none 64  create   MPMT  dynamic nonPIC  FAILED (Group Setup)
pc_irpc ALL_REMAINING_VARIANTS HUNG(timeout 900s)

Both failures are multithreaded create variants, and they fail before the test body runs:

Failed to handle events during thread create
Failed to launch mutatees
Group setup failed:
Process has unexpected error code: 0xff

There are no **Failed test # markers in the detail log, i.e. no iRPC assertion fails — the
mutatee never launches. The remaining hang is on the next multithreaded create variant.

I do not think these belong to this change. They were unreachable before it: with tinfo
empty the test spun on its first configuration and emitted no rows at all, so nothing past
that point had ever been exercised on this platform. This looks like the same ppc64le
proccontrol thread-create weakness that shows up elsewhere in our per-test baseline of
current master, where 19 tests hang — 16 of them pc_* — and pc_tls, test_thread_3 and
test_thread_8 fail outright.

Summary of verification across the three architectures:

arch before after
x86_64 0 rows, hung 32/32 PASSED, deterministic 3/3 reps
aarch64 0 rows, hung 32/32 PASSED
ppc64le 0 rows, hung 8 PASSED, 2 FAILED (Group Setup), still hangs later

Happy to open a separate issue for the ppc64le multithreaded-create failure if that is
useful — it needs its own investigation and should not gate this fix.

🤖 Generated with Claude Code

Comment thread src/proccontrol/pc_irpc.C Resolved
The rest of it described the bug being fixed rather than the code, which is
not useful once the fix is in place. That explanation lives in the commit
message of the preceding commit, where it belongs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants