Good Question: How Does Facial Recognition Technology Work?
MINNEAPOLIS (WCCO) -- A man was shot in New York City Thursday after he attacked an NYPD officer and four others with a hammer.
Investigators had been able to identify David Baril by linking a subway surveillance photo with an old mug shot through facial recognition software.
So, how does the technology work? WCCO talked with Marty Kenner, the global lab manager at 3M's public security business.
"The idea, basically, is to identify some key landscape features," Kenner said.
3M is one of three major companies in the U.S. that make facial recognition software.
Each person's face is unique, so the software maps dozens of positions on a person's face. It includes characteristics like where the ear lobe falls, the width of a nose or the mouth, the distance between the eyes and the position of the corner of the eyes.
"It's essentially like a mapping system for mountains or anything else, the relationship between all of those points," Kenner said.
The system can then compare that facial map to one other image or to a database of many images.
Law enforcement often uses the technology to compare the image to thousands of mugshots. No local law enforcement in Minnesota regularly use this technology yet, but it's a quickly-growing industry.
At least 26 states allow state, local or federal law enforcement agencies to search, or request searches, of photo databases during investigations, according to a 2013 analysis by the Washington Post.
"A lot of people don't know that Facebook is actually tracking what you look like and they are using that to tag pictures of you that are uploaded," CNET's editor-at-large Tim Stevens said in an interview with CBS This Morning in 2014.
According to Anil Jain, a professor of computer science and engineering at Michigan State University, facial recognition can be 99 percent accurate when comparing two images under controlled conditions -- people are looking straight at the camera, they are not covered in accessories and the age difference between the images is less than ten years.
This can generally happen with mug shots, driver's licenses and passport photos. The accuracy level drops when the face is partially covered, the person is turned from the camera or the pictures are several years apart. Facial characteristics change with age.
Jain and his colleagues have done research on why facial recognition didn't catch the Tsarnaev brothers right after the Boston Marathon bombings. The researchers simulated a facial recognition search after the fact.
"The younger brother was easily identified, but with the hat and sunglasses, the older brother couldn't be identified on facial recognition," Jain said.
It is also much easier to match images one to one, rather than one to several thousand or a million.
"In that case, it's not nearly as accurate as fingerprints, for instance, because some of these things can change over time," Kenner said.
Accuracy is also seriously affected when a person's head is turned, but Kenner said 3M has technology to build faces of models.
"We can rotate it and tip it, and we can affect lighting angles and things like that and try to do a match," he said. "Accuracy will go down a bit because if I turn my face too much, I'm only going to have one eye, but I can get enough information to classify to a small group of people."
This technology has been somewhat controversial for privacy reasons, but most in the field think it will continue to have huge growth, not only in law enforcement, but as a replacement for our passwords.
Kenner also believes an ATM might recognize your face rather than your PIN in the future.
