Source-linked DAST is dynamic application security testing that ties every runtime finding back to the exact source file and line that produced it. Instead of a probabilistic confidence score, you get deterministic evidence: the endpoint, the request, the response, and the code location, reproducible the same way on every run.
That last word is where AI-native security testing is quietly struggling. A finding that appears on one scan and vanishes on the next cannot be triaged, cannot be assigned, and cannot be fixed with confidence. The industry has spent two years asking whether AI can find vulnerabilities. The harder question is whether it can find the same one twice.
Key takeaways
- Large language models are non-deterministic by design. Even at temperature zero, the model vendor's own documentation says identical inputs can produce different outputs.
- Run the same AI security review over unchanged code twice and the set of findings can shift; independent research has measured accuracy swings of up to 15% across identical runs, with a best-to-worst gap as large as 70%.
- A finding you cannot reproduce is a notification, not a result. Deterministic, source-linked evidence is what makes it actionable.
- AI is generating more code, and more attack surface, than teams can verify by hand: 45% of AI-generated code samples in Veracode's 2025 research introduced an OWASP Top 10 vulnerability.
- The fix is not "trust the AI less." It is to anchor probabilistic analysis to deterministic evidence: determinism on the finding, agents on the fix, humans on the merge.
The reproducibility problem: why AI findings change between runs
Run the same static or dynamic scanner over unchanged code twice and you expect the same output. Run an AI security reviewer over the same code twice and you often do not get it.
This is not a rumor, it is a documented property of the technology. Anthropic's own documentation states that "even with temperature set to 0, the results will not be fully deterministic and identical inputs may produce different outputs across API calls." Independent academic work quantifies the effect: one study measuring repeated runs under supposedly deterministic settings found accuracy swings of up to 15%, with a gap between best and worst runs as large as 70%. Non-determinism is not a bug in one product. It is how these systems sample tokens, compounded by hardware-level floating-point behavior.
Point that same non-deterministic engine at your source code as a security reviewer and the consequence follows directly: run the review twice over unchanged code and the set of findings can differ. A vulnerability the model raises on one pass may be gone on the next. The code did not change; the reviewer did.
For a marketing chatbot, run-to-run variance is a curiosity. For a security program, it is a workflow failure. If a critical finding surfaces on Monday's pipeline run and disappears on Tuesday's, which one do you staff, patch, and report?
What source-to-finding linkage actually means
The answer is not to distrust AI. It is to give every finding an anchor that does not move.
A source-linked finding is a runtime result carrying four things that reproduce identically every time:
- the endpoint that was exercised,
- the request that triggered the behavior,
- the response that proves it, and
- the exact source file and line that owns the vulnerable code.
The first three are what good dynamic testing has always produced: proof at runtime, an attacker's-eye view that no static model of the code can fully predict. The fourth is what closes the loop. A runtime finding that says "this endpoint is vulnerable" tells you what. A source-linked finding that says "this endpoint is vulnerable, and here is the exact line of the handler that makes it so" tells you what to change. That linkage is deterministic: it is derived from reading the application's own routes and code, not inferred probabilistically, so it points at the same line on every run.
This is the same principle behind fixing API security problems at their source rather than at the perimeter. Runtime tells you a door is unlocked. Source linkage tells you which door, in which wall, and who framed it.
Why deterministic evidence beats a confidence score
Confidence scores are an attempt to make probabilistic output feel decisive. They rarely survive contact with a triage queue.
A confidence score answers "how sure is the model?" It does not answer "will I see this again?" or "where do I go to fix it?" When the underlying finding is unreproducible, a high confidence score is worse than useless, because it invites you to act on something that may not exist on the next run. That is the trap: a model can report high confidence in a finding that will not survive the next scan.
Deterministic, source-linked evidence answers the questions a confidence score cannot. It reproduces, so it can be assigned and tracked. It carries a file and line, so remediation has a destination. It survives being handed to a second engineer, a ticketing system, or an automated agent without degrading into "well, it flagged something." This is why dynamic testing has always been a necessary complement to static analysis rather than a replacement, a point we made in The Essential Role of DAST in Complementing SAST: runtime proves exploitability, source linkage makes that proof fixable.
There is a coverage dimension too. A crawler only tests the endpoints it can reach, and modern applications hide routes behind single-page front ends, undocumented APIs, and generated code. Deriving the API surface from source, deterministically and locally, means the scanner exercises endpoints a crawl-only tool would never see, and every one of those findings arrives already linked back to its origin. AI is generating more of this surface, and more code, than teams can manually verify: Veracode's 2025 GenAI Code Security Report found that 45% of AI-generated code samples introduced an OWASP Top 10 vulnerability. The volume is going up. The only way to keep pace is evidence you do not have to re-verify by hand.
A finding you cannot reproduce is a notification, not a result.
What a source-linked finding looks like in the workflow
Here is where the two halves of the argument meet: the same determinism that makes a finding trustworthy for a human is what makes it safe to hand to an agent.
| Property | LLM-only review | Crawl-only DAST | Source-linked DAST |
|---|---|---|---|
| Reproducibility | Varies run to run; identical inputs can produce different findings | Deterministic on what it reaches | Deterministic; same finding, same file and line every run |
| Evidence delivered | Natural-language explanation plus a confidence score | Endpoint, request, response | Endpoint, request, response, plus the exact source line |
| Endpoint coverage | Depends on prompt and context window | Only what the crawler can reach | Crawler plus a source-derived API spec, reaching unlinked and undocumented routes |
| Triage | Re-verify each finding by hand | Reproducible proof of concept | Reproducible proof mapped to the owning code |
| Remediation handoff | A suggestion to re-check | Points at the endpoint | Points at the line, ready for a human-approved pull request |
| Fit for AI agents | The agent is the source of truth, and it wavers | The agent gets a finding to re-triage | The agent gets fixed evidence; a human approves the merge |
That last column is the model we build toward, and it has a name: agent-ready, not autonomous. NightVision is the deterministic, source-linked evidence layer that AI coding agents consume through the NightVision MCP server and our skills for Claude Code. The agent supplies the remediation logic. NightVision supplies evidence tied to the source line, so the agent is reasoning from a fact, not from a finding that might evaporate. And the pull request it drafts still lands in front of a human, the same detection-to-pull-request flow we built for authentication issues. Determinism on the finding, agents on the fix, humans on the merge.
This is not an argument against AI in security. It is an argument for giving it a deterministic floor to stand on. The probabilistic layer proposes; deterministic, source-linked evidence is what confirms it, assigns it, and fixes it. Agents move fast on top of facts, not on top of findings that might evaporate.
Related readingPart of our series on modern application security testing. Start with the pillar, What You Should Know About Application Security Testing, and see the wider trust argument in Autonomous Penetration Testing: A CISO Reality Check.
Frequently asked questions
What is source-linked DAST?
Source-linked DAST is dynamic application security testing that connects each runtime finding to the exact source file and line responsible for it. You get the endpoint, request, and response as runtime proof, plus a deterministic pointer to the code that needs to change.
Why do AI security findings change between runs?
Large language models sample their output probabilistically, and even with temperature set to zero they are not fully deterministic, a limitation the model vendors document themselves. Hardware-level floating-point behavior adds further variance. Run the same review twice and the set of findings can differ.
Does this mean AI is useless for security?
No. AI is genuinely useful for surfacing candidate issues and reasoning about code at scale. The problem is not that it finds nothing; it is that its raw output is not reproducible on its own. The lesson is not to stop using AI, it is to anchor its findings to deterministic, reproducible evidence so the ones worth acting on hold still from one run to the next.
How is source-linked DAST different from SAST?
SAST reads code statically and can point at a line, but it cannot prove a vulnerability is reachable and exploitable at runtime. DAST proves exploitability by exercising the running application. Source-linked DAST adds the line-level pointer back onto that runtime proof, giving you both the exploit and the address.
Can a DAST tool really map a runtime finding to a source line?
Yes, when it derives the application's routes and API surface from source rather than guessing from the outside. That source-derived map lets the scanner reach endpoints a crawler would miss and lets each finding carry a deterministic link back to the owning code.
How does this help AI coding agents fix vulnerabilities?
An agent is only as reliable as the evidence it reasons from. Handed a finding that may not reproduce, it makes a guess. Handed a reproducible finding tied to a specific line, it can draft a precise fix, which a human then reviews and merges.
Is this the same as autonomous pentesting?
No. NightVision is agent-ready, not autonomous. It supplies deterministic, source-linked evidence that agents and humans act on. For the fuller picture, see Autonomous Penetration Testing: A CISO Reality Check.
The bottom line
The last two years of AI in security were a story about generation. The story that matters now is reproducibility. A finding you cannot reproduce is a notification, not a result, and no confidence score fixes that. Give every finding an address, the endpoint, the proof, and the line of code behind it, the same way on every run, and it becomes something a human can approve and an agent can act on. Determinism on the finding, agents on the fix, humans on the merge.