A 93% confidence score from an algorithm was enough to arrest a man 300 miles from the crime scene — and now the legal system has to explain why software testimony outweighed basic geography.
The Summary
- Robert Dillon is suing Florida law enforcement agencies after being arrested for allegedly attempting to lure a child from a McDonald's, despite living 300 miles from Jacksonville Beach where the crime occurred
- Jacksonville Beach police relied on AI facial recognition software that returned a 93% probability match from security camera footage of the suspect
- Charges were later dropped, but not before Dillon was arrested at his home and prosecuted
- The case exposes how probabilistic AI outputs are being treated as actionable intelligence by law enforcement without human verification of basic facts
The Signal
Jacksonville Beach police arrested Robert Dillon based primarily on an algorithm's 93% confidence score. The software analyzed security footage from a McDonald's showing a man allegedly trying to persuade a girl under 12 to leave with him. The arrest happened at Dillon's home, 300 miles away from where the incident occurred.
This wasn't a case where facial recognition provided a lead that detectives then corroborated. The 93% match appears to have been sufficient grounds for arrest, prosecution, and the disruption of Dillon's life. The charges were eventually dropped, but only after he'd been processed through the system.
"A 93% probability from software was apparently more convincing than the fact that a suspect lived 300 miles from the crime scene."
The case reveals three compounding failures:
- AI facial recognition systems produce probabilistic outputs, not certainties, yet they're being operationalized as definitive identification
- Law enforcement treated the algorithm's confidence score as evidence without basic investigative follow-up (like checking if the suspect could have plausibly been at the location)
- The threshold for arrest shifted from "beyond reasonable doubt" to "software says 93%"
The lawsuit targets multiple law enforcement agencies, suggesting the problem extends beyond one department's misuse of technology. This is systemic automation replacing judgment. When an algorithm becomes the primary investigator, basic questions don't get asked. Geography doesn't matter. Alibi doesn't matter. The machine said 93%.
We've seen wrongful arrests from facial recognition before, but those cases typically involved misidentification where the suspect and perpetrator were in the same city. This case is different because of the geographic impossibility ignored in favor of the algorithm's output. It suggests police are over-indexing on AI confidence scores without understanding what those numbers actually mean or building in human checks.
The Implication
If you're building agent systems that interact with law enforcement, compliance, or any high-stakes decision-making, this case is your canary. Probabilistic outputs need guardrails, not blind trust. A 93% match should trigger investigation, not arrest. The gap between "this looks similar" and "this is the person" requires human judgment, context, and corroboration.
For law enforcement agencies, this lawsuit will likely force policy changes around how AI-generated leads are used. Expect more jurisdictions to mandate that facial recognition can only be used as an investigative tool, not as primary evidence. For the companies selling these systems, expect liability questions about how confidence scores are communicated and whether their software is being deployed beyond its intended use case. The algorithm didn't fail here. The humans who trusted it without question did.