PHILADELPHIA (CBS News) — Rashon Nelson initially brushed it off when the Starbucks manager told him he couldn't use the restroom because he wasn't a paying customer. He thought nothing of it when he and his business partner, Donte Robinson, were approached at their table and were asked if they needed help. The 23-year-old entrepreneurs declined, explaining they were just waiting for a business meeting.
A few minutes later, they hardly noticed when the police walked into the coffee shop — until officers started walking in their direction.
"That's when we knew she called the police on us," Nelson told The Associated Press in the men's first interview since video of their April 12 arrests went viral.
Nelson and Robinson, black men who became best friends in the fourth grade, were taken in handcuffs from the Starbucks in Philadelphia's tony Rittenhouse Square neighborhood, where Robinson has been a customer since he was 15.
The video, recorded on a white customer's cellphone video, galvanized people around the country who saw the exchange as modern-day racism. In the week since, the men have met with Starbucks' CEO and have started pushing for lasting changes to ensure what happened to them doesn't happen to anyone else.
"We were there for a real reason, a real deal that we were working on," Robinson explained. "We put in a lot of time, energy, effort. ... We were at a moment that could have a positive impact on a whole ladder of people, lives, families. So I was like, 'No, you're not stopping that right now.'"
Robinson said he thought about his loved ones and how the afternoon had taken such a turn as he was taken to jail. Nelson wondered if he'd make it home alive.
"Anytime I'm encountered by cops, I can honestly say it's a thought that runs through my mind," Nelson said. "You never know what's going to happen."
"CBS This Morning" co-host Gayle King sat down with Starbucks' executive chairman Howard Schultz on Wednesday to discuss how the company is handling the incident.
"I'm embarrassed, ashamed. I think what occurred was reprehensible at every single level. I think I take it very personally as everyone in our company does and we're committed to making it right. The announcement we made yesterday about closing our stores, 8,000 stores closed, to do significant training with our people is just the beginning of what we will do to transform the way we do business and educate our people on unconscious bias," Schultz said.
Democratic Mayor Jim Kenney, who is white, said what happened at the Starbucks "appears to exemplify what racial discrimination looks like in 2018." Police Commissioner Richard Ross, who's black, said in a Facebook post that arresting officers "did absolutely nothing wrong," and added that Nelson and Robinson were disrespectful to officers.
Ross said officers did what they were supposed to do and were professional in their dealings with the men, "and instead they got the opposite back."
Nelson and Robinson originally were supposed to meet Andrew Yaffe, a white local businessman, at a Starbucks across town. But the plan changed, and they agreed to meet at the Rittenhouse Square location, where they'd met several times before on a potential real estate opportunity.
The black men arrived a few minutes early. Three police officers showed up not long after.
Nelson said they weren't questioned but were told to leave immediately.
Yaffe showed up as the men were being handcuffed. He can be seen in the video demanding an explanation for the officers' actions. Nelson and Robinson did not resist arrest, confused and unsure of what to think or what might happen next.
"When you know that you did nothing wrong, how do you really react to it?" Nelson said. "You can either be ignorant or you can show some type of sophistication and act like you have class. That was the choice we had."
It was hardly their first encounter with police, a rite of passage that becomes a regular occurrence for many black men their age. But neither had been arrested before, setting them apart from many of their peers in the gritty southwest Philadelphia neighborhood where they grew up.
Robinson briefly wondered what he might've done to bring the moment on himself.
"I feel like I fell short," he explained. "I'm trying to think of something I did wrong, to put not just me but my brother, my lifelong friend ... in this situation."
Attorney Stewart Cohen, representing Nelson and Robinson, said the men were illegally profiled. He pointed to Title II of the Civil Rights Act of 1964, which prohibits discrimination on the basis of race in hotels, restaurants, theaters and other public accommodations.
Seattle-based Starbucks Corp. has said the location where the arrests occurred has a policy that restrooms are for paying customers only.
In audio recordings released by police Wednesday, the employee who called 911 can be heard telling the operator, "I have two gentlemen in my cafe that are refusing to make a purchase or leave." In a subsequent communication between police and dispatch, someone says "we have a disturbance at the Starbucks" and refers to "a group of males inside causing a disturbance," and additional officers are sent. The Starbucks employee never called it a disturbance.
Nelson and Robinson spent hours in a jail cell with no outside contact and no sense of what would happen next. They were released after midnight, when the district attorney declined to prosecute them for trespassing. They had no idea the video of their arrests was making the rounds on the internet.
The day after their arrests, they thought about what to do next.
"You go from being someone who's just trying to be an entrepreneur, having your own dreams and aspirations, and then this happens," Nelson said. "How do you handle it? Do you stand up? Do you fight? Do you sit down and just watch everyone else fight for you? Do you let it slide, like we let everything else slide with injustice?"
Robinson, still focused on the previous day's business deal, called Yaffe to reschedule. Yaffe told him about the video and the traction it had gotten.
Over the weekend, attention and outrage over the video grew, prompting a protest at the local Starbucks restaurant and a national boycott. By Monday, the men were set to meet with Starbucks CEO Kevin Johnson to discuss what happened.
Johnson has responded quickly to public outcry around the arrests, calling them "reprehensible," apologizing and ordering stores closed for mandatory training to tackle unconscious bias.
Nelson and Robinson said they're looking for more lasting results and are in mediation proceedings with Starbucks to implement changes, including the posting in stores of a customer bill of rights; the adoption of new policies regarding customer ejections, racial profiling and racial discrimination; and independent investigations of complaints of profiling or discrimination from customers and employees.
Robinson said he appreciates the public support the men have received but anger and boycotting Starbucks are not the solution.
"We need a different type of action ... not words," he said. "It's a time to pay attention and understand what's really going on. We do want a seat at the table."
(© 2018 CBS Interactive Inc. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed. The Associated Press contributed to this report.)