AI security company made false claims about weapons-detection system used at Colorado STEM school, FTC says
The Federal Trade Commission has taken action against a technology company, alleging it deceived users about its artificial intelligence-powered security screening systems installed in a Colorado school after a deadly shooting.
In 2023, Douglas County commissioners approved nearly $1 million for a one-year trial of Evolv Technologies at STEM School Highlands Ranch, the site of a shooting in 2019. The AI-based security system had previously claimed to detect weapons using cameras, sensors, and artificial intelligence, but those claims are now being called into question.
"I hope that Kendrick is the poster child for school safety," said John Castillo.
Castillo lost his son, Kendrick, in that 2019 shooting at the school. The tragedy motivated the county to increase investment in school safety.
"The feeling of being safe and actual security are two different things," Castillo said.
He didn't feel Evolv Technologies was the right use of the nearly $1 million.
"I wasn't for it from the beginning, just because I don't believe that physical deterrents like metal detectors are proven enough right now," Castillo said.
Neither did former county commissioner Lora Thomas, who resigned from her position in December after years of infighting with fellow board members.
"I had real concerns about the technology," Thomas said. She says her former colleagues, Abe Laydon and George Teal, supported spending the money on the Evolv trial and "kept it very hush hush."
Evolv claims to create a "safer, faster and better" security experience through AI-powered screening and analytics.
"They made a number of different claims over time on their website and in other marketing materials, including claims that their Evolv Express product would create weapons-free zones or safe zones," said Michael Atleson, senior staff attorney for the Federal Trade Commission.
"They made claims about the accuracy of the product and its ability both to detect all weapons and ignore all personal items and to allow people to walk into venues without having to remove anything from their bags," he continued. "They also made claims about labor cost savings and the speed at which people would be able to go through the system."
But the FTC says Evolv was unable to support those claims.
An FTC complaint alleges Evolv "deceptively advertised that its Evolv Express scanners would detect all weapons and made misleading claims that its use of artificial intelligence makes its screening systems more accurate, efficient, and cost-effective than traditional metal detectors."
It details one instance where a knife got through Evolv's technology at a New York school and a child was stabbed. The event occurred before county commissioners funded the trial at STEM.
"They [Evolv] swindled them," Castillo said. "They made those schools vulnerable by not doing what they said that they can do."
"I'm not surprised at all," said Thomas. "All along, Olivia, I have had this fear that kids and parents will have this false sense of security that their kids are safe when they are not."
Evolv announced a resolution of the FTC inquiry in late November, where the company did not admit wrongdoing.
"The order against Evolv includes an injunction that prohibits the company from making all of the claims that we challenged as deceptive, unless and until it has sufficient evidence to support those claims," said Atleson.
He says a judge signed the order in December.
In a press release, Evolv assured users that "the FTC did not challenge the core efficacy of Evolv's products, including the use of artificial intelligence in its technology. Instead, the focus of the inquiry was related to how the technology was described for a period of time in historical marketing materials."
The company says it's taking steps to "develop a new Brand Integrity Program centered on transparency and education."
Atleson says the FTC wants to send a message that AI companies need to keep their claims in check.
"AI is this shiny, mysterious thing," he said. "Especially with the rise of generative AI and companies are trying to stuff it into everything to sell more products, and a lot of the claims that they're making are hyped up and are untrue or unsupported. And so we think it's important to take action against companies that are making those claims and remind them that the law applies to them, and to advise consumers and business customers that they need to ensure that they're not just accepting marketing pitches and are instead focused on what is really true and what these products really can and can't do."
He recommends schools consult independent school safety experts when making security decisions.
"If you're a school system or nonprofit or small company that is interested in security products, especially if you're considering steps to take after a tragedy or another incident, do not rely on what marketers are telling you you need, and you should be talking to independent and unbiased experts," Atleson said.
Douglas County Commissioner Abe Laydon shared the following statement with CBS Colorado:
"When Douglas County chose to ramp up its investment in keeping kids safe following the STEM shooting, we had a lot of folks concerned about any technology that wasn't 100% effective at stopping the threat of all school violence. Working with experts, you quickly realize that nothing is 100% effective. What we can – and should – do is layer together deterrence efforts to get as close to perfect as we can for our community's children. We will continue to provide funding for the best strategies and technologies available to keep children safe."
"We need to do more for school safety and security," Castillo said.
He believes funds should go to other security measures, such as hiring law enforcement school resource officers, funding mental health efforts, or implementing physical screenings done by staff.
"Always somebody's going to be able to go through and check a backpack better than a metal detector. Why is it needed? I think that there are different avenues, and I wish that they would explore those, especially here in Douglas County," Castillo said.
Neither Douglas County nor Douglas County School District would confirm if Evolv is still being used at STEM School Highlands Ranch.
STEM School did not respond to numerous requests for comment in the last week, but the Evolv metal detectors are still present at the school's entrance.
As part of the resolution, Evolv must notify certain schools of the FTC's action surrounding the alleged false claims and provide them with an opportunity to exit their contracts. It's not clear if STEM is one of those schools, or if they'll be taking Evolv up on that offer.
Update: After the publishing of this story, STEM sent a letter to parents addressing the FTC's actions and the efficacy of the system. They reassured parents that, although the system does not have all of the features that were promised, the system is still effective at detecting weapons.
Dear STEM Community,
On January 2nd, CBS News published an article about the Evolv weapon detection system used at STEM. The article references a recent finding by the Federal Trade Commission (FTC) that Evolv misrepresented the artificial intelligence (AI) capabilities of its system. This report raises questions about the overall safety and functionality of the Evolv technology.
We acknowledge that Evolv overpromised its AI capabilities, and we appreciate the FTC for bringing this issue to light. Over the past year, we have expressed similar concerns to Evolv, particularly noting that the system does not "self-learn" as initially claimed.
Despite these shortcomings, we have thoroughly evaluated the Evolv system and unanimously agree that it effectively detects firearms. This core functionality significantly enhances the safety of our students and staff, which remains our highest priority. Therefore, we have decided to continue using Evolv's technology for firearm detection, even though its AI features do not meet original expectations.
We are actively engaged in discussions with Evolv as they work to address these concerns and improve their product. Our commitment to the safety and well-being of our community drives all of our decisions. Based on our experience and ongoing dialogue with Evolv, we are confident in the system's ability to provide effective firearm detection. This alone is a compelling reason to maintain our partnership as we strive to ensure the safest possible environment for everyone at STEM.
Thank you for your understanding and continued support.
Sincerely,
Matt Cartier
Chief Innovation Officer
STEM School Highlands Ranch
