Industry

What Does a Week of Directed-Test Writing Actually Cost?

Hiroshi Watanabe · 6 min read
DV engineer analyzing coverage reports

When we started building Photoniq, we had a hypothesis: DV engineers spend a disproportionate amount of time on the analysis phase of coverage closure — figuring out what to write — compared to the actual test-writing phase. To check that assumption, we talked with DV engineers and leads at chip companies over several months, asking them specifically about time allocation during coverage closure campaigns.

The numbers we got back were more consistent — and more lopsided — than we expected.

How We Collected This Data

We talked with DV engineers and verification leads at AI chip startups and mid-size fabless semiconductor teams, asking a specific question: for a typical coverage closure task (closing 10-20 functional coverage bins during the last 4 weeks before tape-out), how did they allocate their time?

We asked them to break down a "typical directed test" into phases: reading and understanding the coverage report, tracing the RTL logic to understand what preconditions are needed, writing the actual SystemVerilog test or sequence, running it and debugging, and updating the coverage tracking spreadsheet. Not a rigorous research survey — conversations with working engineers about real projects.

We want to be clear: this is not a controlled study with statistical significance. These are directional observations from a small set of conversations. We're sharing them because the pattern was consistent enough to be useful, not because we can claim scientific precision about the numbers.

Where the Time Actually Goes

Across the engineers we spoke with, the rough breakdown for a single directed-test task looked like this:

Coverage analysis and diagnosis: 40–60% of total time. Reading the UCDB report or GUI, identifying which bins are uncovered, determining whether the miss is a constraint problem or a genuine coverage hole, and tracing the RTL to understand what logic drives the uncovered bin. For FSM transition misses with complex predecessor dependencies, this phase alone could take most of a day.

Test writing: 20–30% of total time. The actual SystemVerilog — the sequence, the constraints for the new test, the checker that verifies the expected behavior. Once an engineer knows exactly what they're writing and why, this phase goes faster than most people outside DV would expect.

Debug and iteration: 15–25% of total time. Running the test, finding out it doesn't hit the target bin (usually because the precondition analysis was incomplete), iterating on the constraints, re-running. This phase scales with how complex the precondition setup is.

Documentation and tracking: 5–10%. Updating the test plan, marking bins closed, noting which test vector covers which bins for future reference.

The ratio that stood out: 40–60% of time spent on diagnosis, 20–30% on writing. The hard part is the analysis, not the SystemVerilog.

A Concrete Example: FSM Transition Closure

One DV lead described a specific closure task from a recent SoC project. The design had a memory controller with an 11-state FSM. At week 5 of the verification campaign, 8 FSM transitions were still uncovered. The transitions were all error recovery paths — transitions from busy or error states back to idle, triggered by timeout or abort conditions that required specific bus sequences to set up.

His estimate for closing those 8 transitions: "About 3 days of work for one engineer, if they know the FSM well. Maybe 5 days if they're coming in cold."

Breaking that down: roughly half the time was spent understanding which bus transaction sequences were required to reach each source state, and what the exact timing conditions were for the transition trigger. The other half was writing the 8 directed test sequences, running them, and fixing the cases where the testbench setup was off by a cycle.

Three to five engineer-days for 8 FSM transitions. At a fully-loaded cost of $1,500–$2,000 per engineer-day for a senior DV engineer in the Bay Area, that's $4,500–$10,000 for 8 coverage bins. If a design has 40 hard-to-reach bins across functional coverage, FSM transitions, and toggle misses, the labor cost of closing them manually is $20,000–$50,000 — on top of the schedule impact of doing that work in the last few weeks before tape-out, when the design is still changing.

The Schedule Risk Is the Bigger Number

The labor cost matters, but the schedule risk is harder to quantify and usually larger in impact. Directed-test writing during the last 3–4 weeks before tape-out competes with every other verification activity: regression triage, signoff criteria review, late-breaking RTL changes, DRC/LVS iterations. Engineers doing coverage closure aren't doing those other things.

In several of the conversations, the real pain wasn't the cost of the directed-test writing itself — it was the coverage closure work crowding out final regression analysis, which is what actually catches late-breaking bugs. "I spent two weeks writing directed tests and not enough time triaging the failing regressions" was a pattern we heard more than once.

The compounding effect: late coverage closure delays signoff, which compresses the regression window, which increases the probability of a bug escape. The cost of a tape-out re-spin — $1–5M for a leading-edge process node, depending on design complexity — dwarfs the engineering labor cost of the coverage closure work itself.

What AI Analysis Changes About This Breakdown

The diagnosis phase — the 40–60% that's pure analysis work — is where we focused Photoniq's model. If we can hand an engineer an explained recommendation ("this FSM transition requires: transaction type A, followed by B, with the timeout counter at ≥ N — here's the RTL path") instead of a UCDB report and an RTL file, the diagnosis time drops to near zero.

What remains: the actual test writing (20–30% of original time), the debug iteration (15–25%), and the tracking overhead. The total time to close a hard coverage bin drops by 50–70% in the cases we've measured internally — not because the writing got faster, but because the analysis phase was replaced by reading a ranked, explained recommendation list.

We're not claiming that coverage analysis is the only thing consuming DV engineer time during closure. Testbench maintenance, regression triage, and design questions are all real overhead. And we're not saying every coverage bin is closeable with a single directed test — some holes reveal actual testbench deficiencies that require more fundamental work. But the analysis phase of coverage closure is the most compressible part of the pipeline, and compressing it is what creates schedule margin for the verification work that matters more in the final weeks.

The Metric That Should Drive Coverage Closure Investment

The question chip teams should be asking isn't "how many coverage bins are unclosed?" It's "what is the expected schedule impact of closing them the current way, and does that schedule impact create re-spin risk?"

If you have 30 uncovered functional coverage bins at 5 weeks before tape-out, and your current closure rate is 4 bins per engineer-day, that's 7.5 engineer-days of work. If you have two DV engineers on it, that's ~4 days of elapsed time — achievable if nothing else is competing for those engineers. If you have 60 bins and one engineer, you're already in trouble on schedule alone, before accounting for the likelihood that some of those bins reveal deeper testbench issues.

This is the calculation DV leads run implicitly when they estimate tape-out readiness. Making it explicit, with better input data on which bins are actually hard to close and which can be closed with constraint modifications, is what makes coverage closure tractable under real schedule pressure.

The data we collected didn't surprise us because the numbers were large. It surprised us because the pattern was so consistent: in nearly every conversation, engineers described doing their most expensive work — the diagnosis — without any tooling support beyond the coverage report itself. That's the gap we're building to close.