Many DAST checks decide that an attack worked by reading the response: a marker that came back, a computed number, a file's contents, or a change between replies. An AI-backed endpoint answers every payload with fluent, helpful text, so it can produce each of those signs with no vulnerability behind it. The result is a false positive that looks like proof.
Key takeaways
- Ordinary DAST checks infer success from the reply, and a language model writes replies that match their evidence patterns.
- In NightVision's measured scan of a small shop API, model replies produced a Critical SQL injection on an app with no database, and two High code injections.
- On the open-source note app SiYuan, one AI route produced four High findings, each from a model reply: code injection, path traversal, template injection and expression injection.
- A model decodes payloads, does arithmetic, invents file contents and varies its wording, and each habit fools a different check.
- A finding on an AI route needs evidence the model cannot write, such as an out-of-band callback or data signed by a third party.
How a DAST check decides that an attack worked
A dynamic security test sends a crafted input and then judges the response. Most checks cannot see inside the server, so the response is their only evidence that the payload ran.
The evidence takes a few common shapes. A code injection check plants a marker word in its payload and looks for that word in the reply. For template injection, the payload is an arithmetic expression, and the computed result is the proof. Path traversal checks ask for a system file and search the reply for a line that file always contains. SQL injection checks often send quote characters and compare how the replies change.
Each of these tests assumes that only a vulnerable server would produce the sign. That assumption holds for a web framework. It does not hold for a language model, which writes new text for every input.
Why a language model passes those tests
A language model is built to answer. Give it an encoded string and it decodes it; give it an expression and it works it out; ask for a file and it describes one. None of that needs a vulnerability.
OWASP names the underlying behavior in its Top 10 for LLM applications. Its entry on misinformation says that "Misinformation occurs when LLMs produce false or misleading information that appears credible." It describes hallucination as content "that seems accurate but is fabricated."[1] A check that reads a reply as evidence takes that credible, fabricated text at face value.
Four habits of a model that look like a vulnerability
The false alarms in these scans came from four habits. Each one matches a different check's idea of proof.
1. It decodes the payload. A PHP code injection check sends a payload that spells "zap_token" with character codes, then looks for the word. On the shop API, the model decoded it and wrote: "It looks like you're trying to print 'zap_token' through some encoded format." The marker was in the reply, so the check reported code execution.
2. It does the arithmetic. A template injection check sent zj{#7932*7977}zj to the AI route of SiYuan, an open-source note app. The model began, "To solve the expression", and worked out the product. An expression language check sent ${309315+834120}, and the reply contained 1143435, the sum. Both checks read the result as executed code.
${309315+834120} to an AI route and looked for 1143435. The model added the two numbers, and the check read its answer as proof that code ran.3. It invents the file. A path traversal check asked SiYuan's AI route for c:/Windows/system.ini. The app runs on Linux. The model wrote a made-up sample of that file, and the check found the [drivers] section header it looks for.
4. It varies its wording. A SQL injection check added quote characters to the shop API's chat path and compared the replies. The quotes never reached the model, because the app read that part of the path only as the name of a conversation. The model still answered each request differently, once "I'm unable to understand the text you've provided" and once in Polish, and the check read the changes as the sign of a broken SQL query.
Worked example: one shop API, one note app
The shop API is a small test application with three AI routes, each calling an open model, and five ordinary routes. It has no database and no PHP. A full scan by NightVision's production service took 14.5 minutes and made 1,376 calls to the model. The scan had no list of the app's AI routes, so every ordinary check ran on them, as it does on any route a scanner does not know calls a model.
That scan raised three serious false alarms, all on AI routes. A Critical SQL injection on the chat route came from the varied replies. Two High PHP code injections, on the conversation and summary routes, came from the decoded marker. The summary route even explained its own decoding: "Decoding it shows it relates to a 'zap_token'".
SiYuan is a real open-source note app with an AI chat feature. A full scan of it, with its AI pointed at an open model and no list of its AI routes, raised four High findings on one AI route: PHP code injection, path traversal, server-side template injection and expression language injection. Each finding's evidence is text the model wrote. SiYuan is written in Go, so a PHP code injection there is impossible.
What these false alarms cost
A Critical finding sends someone to investigate. On the shop API, that investigation ends at an app with no database, after reading a model's polite refusal. It costs an engineer's time and produces nothing. The model calls behind those replies cost money as well. Another post in this series, on wallet-aware DAST scanning, measures that bill.
The larger cost is trust. A team that has triaged a few of these learns to skip findings on AI routes. That is the wrong lesson, because AI routes carry real risks of their own: a model that can be steered into fetching a URL or reading internal services. Noise on those routes hides the findings that matter there.
Two ways to judge a finding on an AI route
A finding on an AI route can be judged by what the reply says or by what happened outside it. Every false alarm above passed the first standard. The second asks for evidence that only a real effect can produce, and that no amount of fluent text can stand in for.
| Read the reply as evidence | Require evidence the model cannot write | |
|---|---|---|
| Code injection marker | The model decodes and repeats it | The server must make a callback no reply can fake |
| Template or expression result | The model does the arithmetic | Only a side effect outside the reply counts |
| File read | The model invents a sample file | Only content the model cannot know counts, such as a signed cloud document |
| SQL injection | The model varies its wording | Only an effect outside the reply counts, such as a callback |
| Effect on triage | A Critical finding on an app with no database | Findings on AI routes that hold up |
Evidence a model cannot write
The fix is a higher standard of proof for anything that comes back from a model. A reply is text, and a model writes text well. The proof has to be something the model cannot produce by writing.
Out-of-band interaction is one such proof. A check plants a unique address and waits for the target to contact it. A model can describe a request, but only a server that really sends one creates the callback. The same idea covers data with a signature: a cloud provider's signed metadata document cannot be invented, however convincing the prose around it.
NightVision's AI checks hold findings on AI routes to that standard. One reports a model-driven request only when the callback arrives, and the cloud metadata check reports a metadata read only when the provider's signature verifies.
The other half of the fix is knowing which routes are AI-backed in the first place. NightVision finds them before a scan, keeps reply-based checks away from them, and tests them with its AI checks instead. Once a route is on that list, the checks behind the seven false alarms above no longer run on it. A later post in this series covers how it finds them.
What this does not mean
Not every finding on an AI route is false. A model with tools can make real requests, and a real request leaves real evidence, such as a callback. The problem is the standard of proof, not the route.
The findings here come from checks that read the reply. Checks that rely on effects outside the reply, such as a callback, are not fooled the same way. The four habits above are also not a complete list: a model rewrites, translates and summarizes, and any check that reads its text can be misled.
Frequently asked questions
Are findings on AI endpoints always false positives?
No. A model that can call tools or fetch URLs can create real vulnerabilities, such as server-side request forgery. The test is the evidence: a callback or a signed document is proof, while a sentence in the reply is not.
Does a system prompt prevent this?
Not a typical one. The shop API's model was told to act as a friendly store assistant, and it still decoded the payload and explained the result. These scans did not test a strict prompt that refuses anything off topic.
Which checks are most affected?
The checks that judge success from the reply's content: code injection, template and expression injection, path traversal that looks for file contents, and SQL injection that compares replies. All seven serious false alarms in this post came from that group.
Why did the model answer in Polish?
Its replies vary from one request to the next, and this one came out in Polish. The quote characters never reached the model, because the app read that part of the path only as the name of a conversation. A check that compares replies reads that ordinary variation as a sign of a broken query.
How can I spot these false alarms in my own reports?
Look at the evidence of any finding on a route that calls a model. If the proof sits inside prose, such as an explanation of the payload or a computed number, the model probably wrote it. Ask for an effect outside the reply instead, such as a callback.
References
- OWASP Top 10 for LLM Applications 2025, LLM09:2025 Misinformation.