Every so often a bug is interesting not because it's hard to fix, but because of what finding it teaches you about your own tools. This is one of those.
The symptom was quiet and persistent. A NullPointerException was surfacing in our error tracker, always with the same shape:
NullPointerException: No event for ID -5235165696230610229
…thrown while building the failure-grouping projection for a Build Scan
It had fired 3,145 times between April and July 2026, all grouped under a single fingerprint, and always from the same source: a background projection that runs on every Build Scan®. The builds themselves were never affected: they ran, passed, and published as normal. Everything here happened on the consuming side, after the fact, and it was contained by design. The affected projection is marked as errored, so the Build Scan still renders, just without the section that projection feeds. Every occurrence pointed at a single ID, and that would turn out to be the first real clue. We handed the investigation to Claude Code, working in our monorepo, and asked it to find the root cause before proposing any fix.
What follows is the actual path it took: a first session that mapped the entire problem but could not reproduce it, and a written handoff to a second session. That session, on a different model, found the exact, improbable set of preconditions and built a test that reproduces it on demand. Every step of it ran on data Develocity had already captured: the error tracker, the build tags, and the raw event stream behind every Build Scan.
Develocity builds every Build Scan from an event stream. When a build runs tests, the Develocity Gradle plugin emits a running commentary: a test started, a test finished, and (relevant here) the executor that ran it. Test Distribution can farm tests out to a pool of executors, and Predictive Test Selection narrows which tests need to run in the first place. So the data model tracks executors explicitly with three event types:
- TestExecutorAssigned: an executor joins the run and is given an id.
- TestStarted: a test (or suite) begins, tagged with the executorId of the executor that ran it.
- TestExecutorReleased: the executor is done and leaves.
The invariant is obvious once you say it out loud: every executorId a test refers to should have a TestExecutorAssigned before it's used and a TestExecutorReleased after. Those two events bracket the executor's lifetime; the server later resolves that id to a human-readable executor name when it renders the Build Scan.
The exception came from the server asking for the name of an executor id it had never seen assigned. So the first job wasn't to read server code. It was to look at the actual events. Every Develocity user can do this through the Build Export API, which streams the raw events behind any Build Scan:
GET /build-export/v2/build/{buildId}/events
?eventTypes=TestStarted,TestExecutorAssigned,TestExecutorReleased
Pulling those events for the affected builds and counting them gave the first hard signal: the counts didn't balance. More executors were released than were ever assigned. Call one of those released-but-never-assigned executors a phantom: the test events refer to it, but nothing ever announced it.
| Build | Assigned | Released | Phantoms (released, never assigned) |
|---|---|---|---|
| Build A | 14 | 15 | 1 |
| Build B | 33 | 57 | 24 |
Executor events from two affected builds, via the Build Export API. Released should never exceed Assigned.
Then the detail that explained the recurring id: it is a deterministic hash of an executor's descriptor. That is why the same value, -5235165696230610229, showed up across two independent builds rather than varying the way random corruption would. The same code path, on the same kind of task, produced the same phantom every time. The bug was structural.
Laid out as a lifecycle, the whole bug fits in two lines:
- Healthy: the executor id is bracketed by a TestExecutorAssigned and a TestExecutorReleased:
TestExecutorAssigned → TestStarted ×N → TestExecutorReleased - Phantom: tests ran on an executor that was released but never announced:
TestExecutorAssigned ✗ → TestStarted ×N → TestExecutorReleased → lookup fails
The tests ran, and the executor was released, but the TestExecutorAssigned event that should have introduced it was never sent, so the id resolved to nothing.
Each layer of Develocity's own telemetry cut the search by an order of magnitude, from error tracker to raw events to a repeating fingerprint:
- Error tracker, 3,145 occurrences over three months, all one fingerprint. But the message alone said nothing about why.
- One build. Tags pinned it to the CI of detekt, the open-source Kotlin static-analysis tool: one release, one recurring task. Down from thousands of events to a single build to inspect.
- Raw events: Released > Assigned, and the same phantom id in two builds. The bug is in the plugin that produces the stream, not the server that reads it.
Why this matters for the method
None of this required a debugger or a local repro yet. The Superpowers systematic-debugging skill that Claude loaded has one iron law, no fix without a root cause, and its first move is "read the evidence, don't guess." Build Scan events are the evidence, captured in production, replayable on demand. That's the unlock: across thousands of builds, on machines we will never touch, the telemetry is precise enough to debug from.
The first session ran on Claude Opus 4.8, and it did an enormous amount of correct groundwork. It traced the failure back through the server to the exact lookup that threw the NullPointerException. Reading the plugin's test-capture code turned up the shape of the defect. The code path that writes an executor id into a TestStarted event had drifted apart from the code path that announces that executor with a TestExecutorAssigned event.
Opus delegated two sub-investigations to parallel sub-agents: one on the impact of a server-side workaround, one on the capturing-side cause. Then it verified the sub-agents' key claims against the source before trusting them.
It even correctly predicted why an obvious server-side patch (substitute a placeholder name for the missing executor) was the wrong call. That patch would trade a visible, contained failure for a silent one, quietly corrupting executor attribution in the Build Scan. All of that was right, and all of it is in the record.
Then it tried to reproduce the bug in a functional test, and that's where it stalled. It built five different reproducers, each targeting a plausible trigger:
| # | Hypothesized trigger | Result |
|---|---|---|
| 1 | Test Distribution + configuration-cache hit | invariant held |
| 2 | Two plugin copies in one build + Test Distribution + config-cache hit | invariant held |
| 3 | Predictive Test Selection + non-platform fallback | inconsistent setup, inconclusive |
| 4 | Two plugin copies in one build, variant | invariant held |
| 5 | Two plugin copies in one build, messaging between them off | invariant held |
Opus's five reproduction attempts. Every clean one passed, meaning the bug did not occur.
This is the moment that usually goes wrong. The cause was understood at the code level; the temptation to ship a fix "that obviously addresses it" was high. The systematic-debugging skill forbids exactly that: a fix you can't watch fail and then pass isn't verified. So instead of guessing, Opus did the disciplined thing: it wrote up everything it knew, including every avenue that failed and why, and handed off.
"The invariant holds in every simple scenario I can construct. The real trigger is genuinely a narrow multi-path edge which I can't reproduce deterministically in a func test with reasonable effort."
— Claude Opus 4.8, session one, declining to ship an unverified fix
The second session started fresh: cleared context, a different model, Claude Fable 5. It was handed exactly two things: the handoff document, and a checkout of detekt, whose CI setup happened to hit the exact combination that tripped our plugin's bug. That was enough. Where Opus had explored breadth, Fable went straight for the one combination none of the five reproducers had tried together.
The bug needed three independent preconditions to hold in the same task at the same time, a genuine perfect storm. That is why it had survived a test suite that checked each ingredient in isolation:
- A composite build. An included build that also applies the Develocity Gradle plugin: a second, no-op application living in its own classloader.
- A configuration-cache hit. The second run, restored from the configuration cache, where the objects the plugin inspects are rehydrated, not freshly created.
- Predictive Test Selection on the task. With test acceleration enabled, its reporting infrastructure is what announces the executor a test ran on, not the plugin's ordinary test capture.
None of the three is unusual, and none is a bug. What breaks is the place they meet.
The check that decides whether a test descriptor is the "executor-aware" type now spans two classloaders, so on restore it silently fails. The plugin falls back to a synthetic executor for the plain Gradle test worker. Because test acceleration is enabled, announcing executors falls to that side of the plugin, which never saw this one. The TestExecutorAssigned event that should have announced it is never emitted.
The phantom is what comes out the other end: an executor written into TestStarted, but never announced with a TestExecutorAssigned. That is why the five reproducers never caught it: four passed cleanly, and the fifth never reached a clean run. The closest attempts put two copies of the plugin in a single build, which is a different shape from an included build applying it on its own classpath. None of them combined all three conditions in one task, and the bug only appears where all three cross.
Fable encoded that exact recipe as a functional test: a real included build that re-applies the plugin, Predictive Test Selection enabled, run twice so the second run is a configuration-cache hit. Run against the unfixed code, it failed, and not vaguely: it produced the exact phantom id seen in production, -5235165696230610229. The bug in the wild and the bug on the bench were now provably the same bug.
With a test that goes red on demand, the fix became a two-line behavioral correction, verifiable by watching the test go green:
- Emit a TestExecutorAssigned when a test runs on a default worker rather than a distributed executor, instead of assuming test acceleration already did. This closes the assign/release gap at its source.
- Stop emitting a dangling TestExecutorReleased event for an executor that was never assigned.
Then the full regression sweep: 17,242 tests across the affected modules, all green, with the new reproducer flipping cleanly from red (unfixed) to green (fixed). We proved both directions by stashing and restoring the fix.
- Traced the exception to its exact server lookup.
- Read the plugin code; named the assign/release drift.
- Ran parallel sub-investigations; verified their claims.
- Ruled out the tempting server-side placeholder patch.
- Could not reproduce: five reproducers, none with all three conditions.
- Wrote an honest handoff, failed avenues included.
- Read the handoff; skipped the dead ends.
- Combined the three untried conditions at once.
- Reproduced the exact production phantom id.
- Fixed both halves of the invariant.
- Verified red to green; 17,242 tests pass.
- Opened the PR, with a changelog entry.
The handoff document let a fresh session inherit hours of context and spend its budget only on the part still unsolved.
The telemetry was good enough to debug from. The bug lived in the Develocity Gradle plugin, on detekt's CI, triggered by a configuration-cache hit inside a composite build, about as far from a local breakpoint as you can get. What made it tractable was that Develocity had already captured the exact event stream, and the Build Export API let us replay it.
Method beat cleverness. The pivotal decision wasn't a flash of insight; it was Opus refusing to ship a plausible fix it couldn't reproduce. The systematic-debugging discipline turned a dead-end session into a precise map, and the map is what let the next session win.
A handoff is a first-class artifact. Writing down what you tried and why it failed isn't admitting defeat. It's the thing that let a different model, in a fresh session, pick up cold and go straight to the answer. The failed avenues were as valuable as the findings.
Different models, different strengths. One session was broad and careful, the other fast and decisive, and the two turned out to be complementary. The relay between them, mediated by a plain markdown document, outperformed either working alone.
The phantom executor is gone from the stream at its source, and the reproducer will catch it if it comes back. The whole investigation is a small proof of something we already believed: the best data for understanding a build is the data the build already produces.
Trying this on your own builds
Most bugs never need the raw event stream. The error tracker and the build tags did most of the narrowing here, and both are ordinary tooling. But when the failure only exists on a machine you cannot attach a debugger to, the raw events are all you have. The Build Export API streams them behind any Build Scan, the same way we used it in Act I.
