Artificial intelligence is preserving our ability to converse with Holocaust survivors even after they die
Tonight, as the world struggles to contain and recover from the novel coronavirus, we offer a story we completed just before life changed so dramatically. It is a story of history, hope, survival and resilience, which has its roots in another time when the world was convulsed by crisis – World War II.
- How a father's final message motivated a Holocaust survivor to record his memories
- From the 60 Minutes archives: Survivors of Josef Mengele's twin experiments
This year marks the 75th anniversary of the end of that war and of the liberation of concentration camps across Europe. Most of the survivors who remain are now in their 80s and 90s. Soon there will be no one left who experienced the horrors of the Holocaust firsthand, no one to answer questions or bear witness to future generations. But a new and dramatic effort is underway to change that. Harnessing the technologies of the present and the future, it keeps alive the ability to talk to, and get answers from, the past.
Correspondent Lesley Stahl's interview with Holocaust survivor Aaron Elster, who spent two years of his childhood hidden in a neighbor's attic, was unlike any interview she had ever done.
"Aaron, tell us what your parents did before the war," Stahl asked Elster.
"They owned and operated a butcher shop," Elster said.
It wasn't the content of the interview that was so unusual.
"Where did you live?" Stahl asked.
"I was born in a small town in Poland called Sokolów Podlaski," Elster said.
It's the fact that this interview was with a man who was no longer alive. Aaron Elster died two years ago.
"What's the weather like today?" Stahl asked.
"I'm actually a recording," Elster said. "I cannot answer that question."
Heather Maio came up with the idea for this project. She had worked on exhibits featuring Holocaust survivors for years and wanted future generations to have the same opportunity to interact with them as she'd had.
"I wanted to talk to a Holocaust survivor like I would today," Maio said. "With that person sitting right in front of me and we were having a conversation."
She knew that back in the 90s, after making the film "Schindler's List," Steven Spielberg created a foundation named for the Hebrew word for the Holocaust – Shoah – to film and collect testimonies from as many survivors as possible. They have interviewed nearly 55,000 of them so far and have stored them at the University of Southern California. But Maio dreamed of something more dynamic, being able to actively converse with survivors after they're gone. And she figured, in the age of artificial intelligence tools like Siri and Alexa, the technology had to be creatable.
She brought the idea to Stephen Smith, executive director of the USC Shoah Foundation, and now her husband. He loved it, but some of his colleagues weren't so sure.
"One of them looked at me," Maio said. "She was, like, 'You wanna talk to dead people?'"
"And you said, '"Yes, because that's the point,'" Stahl said.
"That's the point," Maio said.
"Well maybe people thought you're turning the Holocaust into something maybe hokey?" Stahl asked.
"Yeah," Maio said. "They said that, 'You're gonna Disney-fy the Holocaust.'"
"We had a lot of pushback on this project," Smith said. "'Is it the right thing to do? What about the wellbeing of the survivors? Are we trying to keep them alive beyond their deaths?' Everyone had questions except for one group of people, the survivors themselves, who said, 'Where do I sign up? I would like to participate in this project.' No barriers to entry."
The first survivor they signed up to do a trial run was a man named Pinchas Gutter, who was born in Poland and deported to the Majdanek concentration camp with his parents and twin sister Sabina at the age of 11. He is the only one who survived. They flew Gutter from his home in Toronto to Los Angeles, and asked him to sit inside a giant lattice-like dome.
"Yeah, I call it a sphere," Gutter said. "They call it a dome. And then eventually, it was called a bubble."
A bubble surrounding him with lights and more than 20 cameras. The goal was to future-proof the interviews so that as technology advances and 3D, hologram-like projection becomes the norm, they'll have all the necessary angles.
"So the very first day we went to film Pinchas, we had these ultra high speed cameras," Smith said. "They were all linked together and synced together to make this video of him. So we sit down and they press record. Nothing happens. So Pinchas is sitting there with 6,000 LED lights on him and cameras that don't work."
Sunglasses shielded his eyes.
"I was bored sitting in that chair, So I started singing to myself," Gutter said. "So suddenly, Steven had this idea, 'Oh, he's singing. We're gonna record some songs of his.'"
Both Smith and Maio said Gutter was a good sport. Eventually the cameras rolled and Gutter was asked to come back to the bubble for the real thing.
"How long were you in that chair?" Stahl asked him.
"A whole week from 9:00 to 5:00," Gutter said. "We were there with breaks for lunch. And-- but I was there from 9:00 to 5:00 answering questions."
It took so long because they asked him nearly 2000 questions. The idea was to cover every conceivable question anyone might ever want to ask him.
"Did you have to look exactly the same?" Stahl asked.
"I had to wear the same clothes and I had three pairs of the same jackets, the same shirts, the same trousers, the same shoes," Gutter said.
Gutter can now be seen -- in those shirts, trousers, and shoes -- at Holocaust museums in Dallas, Indiana, and at the Illinois Holocaust Museum in Skokie, outside Chicago, where visitors can ask him their own questions.
"What kept you going," one girl asked, "or what gave you hope while you were experiencing hardship in the camps?"
"We did hope that the Nazis would lose the war," Gutter's digital image responded.
Gutter's image is projected onto an 11-foot high screen. Smith explained how the technology works.
"So what's happening is all of the answers to the questions that Pinchas gave go into a database," Smith said. "And when you ask a question, the algorithm is looking through all of the database, 'Do I have an answer to that.' And then it'll bring back what it thinks is the closest answer to your question."
Stahl then asked Gutter's digital image a question.
"Did you have a happy childhood?" Stahl asked.
"I had a very happy childhood," Gutter's digital image said. "My parents were winemakers. My father started teaching me to become a winemaker when I was 3-and-a-half years old. By the age of 5, I could already read and I could already write."
"Wow," Stahl said. "You're very smart."
"Thank you," Gutter's digital image said with a laugh.
"I've noticed there's a little jiggle right before Pinchas starts to talk," Stahl said. "What is that?"
"What you're seeing here isn't a human being," Smith said. "It's video clips that are-- that are being butted up to each other and played. And as it searches and brings the clip in, you just-- you're seeing a little bit of a jump cut."
The jump cuts stopped being distracting once Stahl asked about the fate of Gutter's family.
"Tell us what happened when you got to the camp," Stahl said.
"As soon as we arrived there, we were being separated into different groups," Gutter's digital image said. "And my sister was somehow pushed towards the children. And I saw her, she must have spotted my mother. So she ran towards my mother. I saw my mother. And she hugged her. And since that time, all I can remember whenever I think of my sister is her long-- big, long, blonde braid."
That was the last time he saw his twin sister, Sabina. He learned later that day that she and both his parents had been killed in the gas chambers. Pinchas Gutter was alone at age 11, put to work as a slave laborer.
"Did you ever see anybody killed?" Stahl asked.
"Unfortunately, I saw many people die in front of my eyes," Gutter's digital image said.
Stahl wasn't sure how a recording would handle what she wanted to ask him next.
"How can you still have faith in God?" Stahl asked.
"How can you possibly not believe in God?" Gutter's digital image said.
"Well," Stahl said, "how did he let this happen?"
"God gave human beings the knowledge of right and wrong and he allowed them to do what they wished on this earth, to find their own way," Gutter's digital image said. "To my mind, when God sees what human beings are up to, especially things like genocide, he weeps."
"Wow. Stephen, I could ask him questions for ten hours," Stahl said.
Since Pinchas Gutter was filmed, the Shoah Foundation has recorded interviews with 21 more Holocaust survivors, each for a full week. And they've shrunk the set-up required, so they can take a mobile rig on the road to record survivors close to where they live. They've deliberately chosen interview subjects with all different wartime experiences. Survivors of Auschwitz, hidden children, and as we saw last fall in New Jersey, 93-year-old Alan Moskin, who isn't a holocaust survivor. He was a liberator.
"Entering that camp was the most horrific sight I've ever seen or ever hope to see the rest of my life," Moskin said.
Moskin was an 18-year-old private when his Army unit liberated a little-known concentration camp called Gunskirchen.
"There was a pile of skeleton-like bodies on the left," Moskin said. "There was another pile of skeleton-like bodies on the right. 'Those poor souls.' That's the term my lieutenant kept screaming, 'Oh my God, look at these poor souls.'"
"I remember the expression and the attitude of all of us," Moskin continued. "'What in the freak? What is this? God almighty'"
Each of Alan Moskin's answers is then isolated by a team of researchers at the Shoah Foundation Office. They add into the system a variety of questions people might ask to trigger that response.
"For every question that we asked, there are 15 different ways of asking the same question," Maio said. "And that's all manual."
Editors rotate the image, turn the green screen background into black and then a long process of testing begins, some of it in schools.
Students are asked to try it out. Ask whatever questions they want and see if the system calls up the correct answer.
"How did you find out that your city was getting invaded by Germany?" One student asked.
"How did you feel about your family?" Another asked.
Pinchas Gutter's digital image responded to one student by asking, "Can you rephrase that, please?"
Every question and response is then reviewed.
"We log every single question that's asked of the system," Maio said. "And see if there is a better response that addresses that question more directly."
As Stahl's crew discovered, it's still a work in progress.
"Tell us about your family when you were a little boy," Stahl asked Gutter's digital image.
"How about you ask me about life after the war?" The digital image answered back.
"So, couple of things about artificial intelligence," Smith said. "It is mainly artificial and not so intelligent."
"Just yet, for now," Maio said.
"But the beauty of artificial intelligence is it develops over time," Smith said. "So we aren't changing the content. All the answers remain the same. But over time, the range of questions that you can ask will be enhanced considerably."
Questions to draw out what it was like for Aaron Elster hiding in that attic 75 years ago.
"I used to pray to God to let me live 'til I was 25," Elster's digital image said. "I wanted to taste what adulthood would be like. So, am I a lucky guy? Yes I am."
Of more than 20 men and women who have participated so far in the project, three have already passed away. Stahl had conversations with two of them, conversations that at times felt so normal, she said she could almost forget she was talking with the digital image of someone no longer living.
First, a spunky 4'9" woman named Eva Kor, an identical twin who, together with her sister, survived Auschwitz and the notorious experiments of Dr. Josef Mengele. Kor spent her life after the war in Terre Haute, Indiana. She died last summer at the age of 85.
"Hi, Eva. How are you today?" Stahl asked.
"I'm fine, and how are you?" Kor's digital image said back.
"I'm good," Stahl asked.
Stahl said it felt natural to answer Kor's question before posing her own.
"So how old were you when you went to Auschwitz?" Stahl asked.
"When I arrived in Auschwitz, I was ten years old," Kor's digital image said. "And I stayed in Auschwitz until liberation, which was about nine months later when we were liberated."
"So we made a little announcement about the fact we were starting this project," Smith said. "I get a call the next day from a lady called Eva Kor. I didn't know her at that point in time. And she says, 'I want to be one of those 3D interviews.'"
"'I wanna be a hologram,'" Maio recalled Kor saying.
"I said, 'Well, I'm traveling, I'm very sorry,'" Smith said. "'Where're you going?' 'Oh, well, I've got to go to New York. I'm going to D.C.' 'When are you gonna go to D.C.? I'm going to D.C.' Turns out we were going to the same event in D.C. I arrive at my hotel, she's sitting in the lobby, waiting for me."
When Eva, on the right, and her twin sister, Miriam arrived at Auschwitz, they were pulled away from their parents and older sisters and taken to a barrack full of twins. They never saw their family again.
60 Minutes reported on Mengele's twin experiments in a story back in 1992, and Stahl actually interviewed the living Eva Kor at her home in Terre Haute. Eva told Stahl then about becoming extremely sick after an injection.
"Mengele came in every morning and every evening, with four other doctors," Kor said in 1992. "And he declared, very sarcastically, laughing, 'Too bad. She's so young. She has only 2 weeks to live.'"
"When I heard that, I knew he was right and I immediately made a silent pledge that I would prove you, Dr. Mengele, wrong," Kor's digital image said in the present.
Imagine, picking up a conversation almost 30 years later -- and after Eva Kor's death.
"Eva, tell us about Dr. Mengele," Stahl asked. "What was he like?"
"He had a gorgeous face, a movie star face, and very pleasant, actually. Dark hair, dark eyes," Kor's digital image said. "When I looked into his eyes, I could see nothing but evil. People say that the eyes are the center of the soul, and in Mengele's case, that was correct."
Eva and Miriam are visible in footage taken by the Soviet forces that liberated Auschwitz 75 years ago.
They went back to the camp many times, Eva continuing to go even after Miriam's death in 1993. It was on one of those visits that Eva made a stunning announcement that she had decided to forgive her Nazi captors.
"I, Eva Moses Kor, hereby give amnesty to all Nazis who participated," Kor said at the time.
She came under blistering attack from other survivors.
"How can you forgive? How is that possible?" Stahl asked Kor's digital image.
"My forgiveness does not mean that I forget what happened, which is impossible," Kor's digital image said. "My forgiveness is an act of self-healing, self-liberation, and self-empowerment."
Aaron Elster disagrees.
"I cannot forgive," Elster's digital image said. "For them to get forgiveness, they have to ask my little sister, Sarah, whom they brutally murdered. I have no right to forgive, and I will not forgive."
"What's important for me in this project is that we have Holocaust survivors who have different points of view about God and religion and faith and forgiveness," Stephen Smith said. "And that's what this project will allow us to do."
Aaron Elster, unlike many Holocaust survivors, never spent time in a concentration camp. As Jews were being rounded up in his town's marketplace and sent to Treblinka, his father told him to run. He was 9 years old.
"And I managed to crawl into the sewer that ran along the marketplace, the street," Elster's digital image said. "And kept crawling 'til I felt I was outta sight, stood up and started running."
He made it to the building of an older Polish couple named the Gurskis who'd been customers at his family's butcher shop.
"He shows up. And she didn't wanna take him," Heather Maio said. "He started crying. And then she led him upstairs."
"Aaron, how long did you stay in the attic?" Stahl asked.
"I lived in that attic for close to two years," Elster's digital image said.
Two years with just one visit a day to bring food and water.
"What was it like in the attic?" Stahl asked.
"Oh, there's so many things that I remember, the hunger, the fear, the absolute, total loneliness," Elster's digital image said. "What do you do all day? You're sitting there. I used to catch flies, outta desperation, and tear their wings off, so they wouldn't fly away, so I had them there."
"How did you survive?" Stahl asked. "How did you survive in that attic?"
"I had the ability to daydream," Elster's digital image said. "I used to write novels in my head. I was the hero all the time. And we have that ability to either give in to our misery and our pain and die or absorb the physical pain by keep your mentality, keep your soul, keep your mind. So was I bored? Was I scared? Was I in need of somebody to accept me or to tell me that I'm okay, that I'm a nice kid? Sure. But that was not part of my life."
"We got a phone call to say that Aaron Elster had suddenly passed away," Smith said. "I was at a conference at that time. So the next morning, I went into the-- little room that we had, and I turned on Aaron Elster's testimony. And I realized I was gonna be the first person ever to click that little button and ask a question of somebody who was no longer alive. And for the next six hours, people came in and outta that room. His funeral had not yet taken place, and yet the legacy was already continuing. And it was a very powerful and touching moment."
A touching moment that may soon be available to others beyond the community of Holocaust survivors. Heather Maio says in the process of developing and testing this technology, she was barraged with inquiries.
"There wasn't one person, literally not one, that didn't ask me if they could do a similar interview with either a loved one, for themselves," Maio said.
"Unrelated to the Holocaust?" Stahl asked.
"Unrelated, completely unrelated," Maio said. "'Can I do this with someone that I know?'"
"What's the answer?" Stahl asked.
"Yes," Maio said.
She has started an independent company that is trying to expand the use of this technology. Recording interviews with other historical figures like astronauts, and eventually with anyone at all.
"So do you think that this is just gonna be a tool that people will use?" Stahl asked. "Everybody will be recording their histories?"
"Interactively," Maio said.
"And other people can interview them?" Stahl asked. "It'll just be life?"
"Yeah," Maio said.
For now, though, the race is on to capture interviews with as many Holocaust survivors as possible while there's still time, so the conversations can continue, always, with people like Aaron Elster.
"Do you want revenge?" Stahl asked Elster's digital image.
"When I was a youngster I wanted revenge very very, very much, and I hated. I hated," Elster's digital image said. "But most of the perpetrators, most of the killers are dead. So who am I going to hate? The grandchildren that had nothing to do with? It's not right. Revenge is not part of my life, not part of my thinking."
"You know, here you have these people who were basically destined to be annihilated," Stahl said to Stephen Smith. "That they survived is the miracle. But they were supposed to be murdered, killed. And now they have immortality."
"They were not supposed to have a name," Smith said. "They were supposed to be destroyed for all time. And now, through this program, they will be able to continue to answer questions hundreds of years after the Nazis have gone."
"It's that 'never forget,'" Stahl said.
"We've had a lot of clichés around the Holocaust. You know, 'Never again, never forget, we must remember,' all this sort of thing," Smith said. "What this does, it makes sure that there isn't closure because it's not about a statement. It's not about a particular thing that's being instructed of you. The onus is on you to ask the questions. The onus is on you to be curious, and to want to know. And so in a sense, it turns the learning on its head and says, "'I'm not gonna tell you what the lessons of the Holocaust are. I'm not gonna tell you what the Holocaust means. But if you want to find out, then you can ask.'"
So there we were, at a special moment in time when the living Pinchas Gutter could talk to the one who will live forever.
"Would you ask you a question for us?" Stahl asked Gutter.
"I will ask the one which is my favorite," Gutter said. "Can you sing me a song from your youth?"
"You want me to sing it for you?" Gutter's digital image responded.
"Yes, please," Gutter said.
With that, Gutter's digital image began to sing in Polish, with the live Gutter joining in.
"What does that mean?" Stahl asked. "What is the song? Is it a happy song?"
"Yeah, well, it's a happy song," Gutter said. "It's like a brother and a sister, which of course, my twin sister-- are traveling in the woods or on the-- on the road, and they can't get over how beautiful the world is."
Produced by Shari Finkelstein. Associate producer, Jaime Woods. Broadcast associate, Maria Rutan. Edited by Daniel J. Glucksman.