When AI gets it wrong: A man’s fight against a faulty facial recognition system

In late 2023, a Florida man named Richard Dillon found himself in a nightmare no one should experience. Police arrested him for allegedly trying to lure a child away from a McDonald’s, all because an AI system claimed his face matched surveillance footage. The problem? Dillon was hundreds of miles away at the time. His alibi—scars from skin cancer surgery—wasn’t even close to the suspect’s features, yet the flawed system still pointed to him. Dillon’s ordeal didn’t end with the arrest. He spent a night in jail, borrowed money to pay bail, and even faced lingering suspicion long after charges were dropped. His story is now part of a lawsuit arguing that police relied too heavily on AI without proper checks. The technology in question, FACESNXT, is supposed to help identify suspects, but critics say it’s far from reliable. Studies show facial recognition often makes mistakes, especially when photos are blurry or poorly lit—conditions that seem to apply in Dillon’s case.What makes this situation worse is how AI tools are used in policing. Instead of confirming guilt, these systems should provide leads for further investigation. Yet officers sometimes treat a "match" as proof, skipping proper verification. Dillon’s lawsuit highlights this dangerous shortcut, one that has led to over a dozen known false arrests. The bigger question is whether police should trust AI at all when human oversight is skipped. Even after his name was cleared, Dillon says the damage lingers. He avoids interactions with kids now, fearing the stigma of being falsely accused. His case isn’t just about one man’s suffering—it’s a warning about the risks of unchecked technology in law enforcement.