An out-of-band check proves a vulnerability when the target contacts a server only the scanner controls. The target decides when: in one test, callbacks arrived 30, 120 and 600 seconds after the request. A scanner that stops listening when its check ends marks the check passed, a false negative on a bug the target proved.
Key takeaways
- Out-of-band checks, SSRF being the best-known, do not prove the bug from the response. They wait for the target to reach out.
- The target decides when that happens. A queue, a retry, a scheduled job that fetches the URL later, or an AI model behind the endpoint can push the callback minutes past the check.
- Any scanner misses the callback when the process that registered the payload is gone before it arrives, whether a cloud worker was torn down or a desktop window was closed.
- A longer wait is a bet on a deadline, paid by every scan, and there is no right number.
- NightVision keeps that memory on a collector it owns. Each payload is stored with the finding it would prove, and every callback is kept. A callback that lands after the scan, even days after, becomes a finding on that scan within minutes of arriving, with its evidence.
How an out-of-band check proves a bug
Most active checks send a request and prove the bug from the response. Out-of-band checks send the request too, but they do not rely on the response, because the bug they look for leaves nothing visible in it. They use a server outside the target that only the scanner controls, and the proof is the target reaching that server on its own.
The mechanism is simple. The check makes a unique address on a server it controls, remembers which request that address belongs to, and puts the address into the payload it sends. If the target reaches that address, the server records the visit, and the scanner matches it back to the request and raises a finding. Figure 1 shows the three moves.
The checks that work this way are the ones whose bug happens on the server, out of sight. Blind SSRF is the best-known. OWASP's definition gives the shape: "In a Server-Side Request Forgery (SSRF) attack, the attacker can abuse functionality on the server to read or update internal resources." The server does the fetching, and when it never echoes what it fetched, the response the scanner sees is whatever the page normally returns; only the fetch itself can prove the bug. The class that asks an AI model behind an endpoint to fetch a URL works the same way, and the FAQ at the end names the rest.
Three places a callback can arrive
The target decides when it calls back, not the scanner. There are three windows: during the check, after the check while the scan is still running, and after the scan has closed. Only the first window is short. The other two last as long as the target's own delays.
1. During the check. The target fetches the URL as part of handling the request. This is the case scanners are built around, and the case demos show.
2. After the check, while the scan is still running. The fetch went into a queue, a worker pool, a retry loop, or an AI model that is answering other prompts first. The check has moved on to the next rule.
3. After the scan. The target reaches the address hours or even days later: a scheduled job that fetches stored URLs, a nightly process that follows every link in the database, a queue that only drains off-peak, or a person opening the stored request from a log. The scan finished long before that, and its report has already been read.
Targets built around AI models are full of such delays. An endpoint with an AI model between the request and the fetch can answer in tens of seconds once it is queueing. NightVision measured one such target during a scan: 1,354 responses took longer than five seconds, and the slowest took 82.7 seconds.
Nobody is listening when the callback arrives
Any scanner misses a callback when the process that registered the payload has exited before the callback arrives. A scanner that runs each check in a short-lived cloud worker loses it when the worker is torn down; a desktop scanner loses it when the window is closed. The last poll happens moments after the rule ends, and whatever arrives later lands on a server that records it for nobody. Figure 2 shows the timing on one check.
The result is a false negative that the report shows as a passed check. A passed check means only that the rule raised nothing, so a real vulnerability, proven by the target itself, goes into the report as absent.
NightVision runs each check, or a small batch of checks, in its own worker, so that a slow rule cannot hold the rest of the scan hostage. The worker starts, runs the rule, writes its alerts, and exits. For NightVision, then, the process that sent the payload is gone by design, and the question is where the memory of what each payload means lives. The rest of this post answers it.
What a longer wait buys
The first fix everyone reaches for is a grace period: after the rule ends, sleep and poll once more. It is a bet on a deadline, and there is no right number. Thirty seconds catches a slow worker pool. Two minutes catches a retry loop. Several minutes catches an AI model under load. An hour catches a batch job. Every second is paid by every out-of-band rule on every scan, including the great majority of scans where nothing was ever going to call back.
Owning the callback server is the easy half, and open-source servers such as Interactsh already handle it well. The hard half is remembering what each payload meant after the process that sent it is gone, and having a finished scan to attach the finding to.
Three ways to handle a late callback
There are three ways to handle a late callback: wait longer after each rule, keep the scanner process alive, or store the meaning of each payload on a server outside the scanner. The table below compares what each approach catches, what it costs, and what it still misses.
| Approach | Catches | Costs | Still misses |
|---|---|---|---|
| Wait longer after each rule | Callbacks inside the chosen window | Compute on every scan, whether or not a callback was ever coming | Everything past the window, which for a batch job is everything |
| Keep the scanner process alive | Callbacks while the process lives | A process that outlives the scan, which a per-check worker does not have | Callbacks after the process finally stops |
| Store the meaning on the server | Every callback, whenever it lands | One small server and a scheduled check | Weaker evidence, not nothing: a callback after the payload's active period is marked as such. After the retention period there is no record left to attach to |
The third row is the shape of the fix. Waiting costs compute on every scan and grows with the window. Listening costs one small server whose bill is the same whether a callback comes in a second or in a week.
Remember on the server, not in the scanner
NightVision registers each out-of-band payload with a collector it runs itself, together with the finding the payload would prove. The collector keeps every callback it receives for as long as the payload is kept. Figure 3 shows the exchange.
A callback that lands while the scanner is alive is raised as a finding the normal way. A callback that lands after the scanner is gone waits on the collector. A late-findings step runs when each scan closes and again on a schedule, minutes apart, and turns what is waiting into findings on the scan they belong to.
There is no deadline in this design. A payload stays active for a fixed period, and a callback after that is still recorded and flagged as arriving after expiry. The horizon is a confidence axis, not a cliff.
A worked example
Two targets show the two windows. The first is a deliberately slow SSRF sink with four vulnerable paths: one fetches the URL at once, the others fetch it 30, 120 and 600 seconds after the request. A scanner that keeps the meaning only in its worker reports one of the four: the worker has exited before the three later callbacks arrive, and they land on a server that no longer knows what they mean.
With the collector holding the meaning, a production scan of the same target reported all four. The immediate one was raised during the scan. The 30-second and 120-second ones were posted by the late-findings step six minutes after the scan closed. The 600-second one followed sixteen minutes after. Each landed on the scan it belonged to, with the callback's protocol, source address, arrival time and the raw request as evidence.
The second target is a chat endpoint backed by a local AI model that answers minutes after the scan is over. A scanner that keeps the meaning only in its worker would report none of its callbacks. A full production scan of the same target with the collector gained twelve late findings ten minutes after it closed, one per payload that called back. A second full scan the same afternoon gained 17 within seconds of closing and 10 more six minutes later.
The late findings can even outlive the rule that sent them. Of the ten that arrived six minutes after the second scan closed, eight came from a rule the scanner had already stopped running: its payloads were out, and the target answered them anyway.
What a late finding carries
A finding that arrives after the scan needs to explain itself more than one raised in the moment, because the reader was not watching. It has to say what was sent, what came back, and when.
Each late finding carries the rule and the request that planted the payload, the callback's protocol (DNS, HTTP, HTTPS or LDAP), the address it came from, and when the callback arrived next to when the payload was planted. The address places the fetch on a network rather than on one host: for a DNS callback it is the resolver the target used, and for an HTTP callback it is the address the fetch left from.
It also carries the raw interaction and a flag that says it was posted after the scan closed. A callback that arrives after the payload's active period is posted with lower confidence. A payload that old is as likely to be a stored value being reprocessed, or a person following a link from a log, as a live vulnerability.
That is enough for a reviewer, or a coding agent, to decide whether the bug is real and where to fix it. It is the same bar every other NightVision finding has to meet.
Related readingThis is the out-of-band sibling of the failure in Did Your DAST Scan Actually Test the App? There, a passed check hid a rule that sent nothing. Here, a passed check hid a proof that arrived after the scanner stopped listening.
Frequently asked questions
Which checks does this affect?
Any check that proves itself through an external interaction. Blind SSRF is the common case, and the check for an AI model that fetches an external URL works the same way. In-band checks that read the response are unaffected.
How late is too late?
There is no cutoff for correctness, only for confidence. A callback within the scan or minutes after it is a confirmed finding. A callback after the payload's active period is posted with lower confidence and says so. After the retention period ends, the payload is purged and there is nothing left to match.
Does the scan status change?
The scan stays completed. The finding is added to that scan with a flag that says it was posted after the scan closed, and the check it belongs to is set to failed.
Where to read next
For the sink these checks exercise most, read Server-Side Request Forgery (SSRF), Explained. For what a finding has to carry once it lands, whoever posts it, read Evidence an Agent Can Act On. The open-source server many scanners use for the method is Interactsh.