Cops’ favorite AI tool automatically deletes evidence of when AI was used

On Thursday, a digital rights group, the Electronic Frontier Foundation, published an expansive investigation into AI-generated police reports that the group alleged are, by design, nearly impossible to audit and could make it easier for cops to lie under oath.

Axon's Draft One debuted last summer at a police department in Colorado, instantly raising questions about the feared negative impacts of AI-written police reports on the criminal justice system. The tool relies on a ChatGPT variant to generate police reports based on body camera audio, which cops are then supposed to edit to correct any mistakes, assess the AI outputs for biases, or add key context.

But the EFF found that the tech "seems designed to stymie any attempts at auditing, transparency, and accountability." Cops don't have to disclose when AI is used in every department, and Draft One does not save drafts or retain a record showing which parts of reports are AI-generated. Departments also don't retain different versions of drafts, making it difficult to assess how one version of an AI report might compare to another to help the public determine if the technology is "junk," the EFF said. That raises the question, the EFF suggested, "Why wouldn't an agency want to maintain a record that can establish the technology’s accuracy?"

It's currently hard to know if cops are editing the reports or "reflexively rubber-stamping the drafts to move on as quickly as possible," the EFF said. That's particularly troubling, the EFF noted, since Axon disclosed to at least one police department that "there has already been an occasion when engineers discovered a bug that allowed officers on at least three occasions to circumvent the 'guardrails' that supposedly deter officers from submitting AI-generated reports without reading them first."