Mobile shopping and banking? Many consumers say, "Never!"

A majority of American adults own smartphones, but security concerns are scaring many people away from using them for functions like mobile banking, payments and shopping, a new survey finds.

In a poll conducted for the ID security firm Intercede, almost half of U.S. consumers surveyed, 48 percent, said they would "never" use bill payment apps, and 44 percent said they would never use mobile banking services.

One in five said they did not feel safe shopping on their smartphone. And 63 percent said they worry about security on their mobile device.

The survey of about 2,000 people also highlighted some other common security concerns. People's number one worry was safeguarding their data against identity theft should their phone get stolen.

With so many recent security breaches in the news -- from Target to the Heartbleed bug to Russian hackers stealing passwords --Intercede CEO Richard Parris says it's not surprising that consumers just don't trust mobile security.

Parris noted that most modern smartphones have multiple onboard security features -- secure microSD, biometric fingerprint readers, virtual smart cards -- already available, but it's up to app developers and service providers to make use of them.

"If this baseline security technology was more widely embraced, the cellphone has the potential to be the personal security enabler of the future and not the security bleed it is viewed as being today," Parris told CBS News in an email.

Are some smartphones more secure than others? When comparing the security of mobile operating systems, Parris singled out iOS due to Apple's vertical integration approach to manufacturing iPhones and iPads. However, other companies can utilize some of the onboard security features available on most mobile devices. TrustZone, embedded in the ARM processor, is one such security feature that is available on a variety of tablets and smartphones running Android.

"Google needs to encourage the app community to start writing secure apps using the TrustZone technology that is already embedded in the latest core processors to secure the customer experience," Parris explained to CBS News. "They also need to work with service providers, such as banks and retailers, to ensure their services can only be accessed from secure apps."

Of course, smartphone users need to do their part by following recommended security practices. Many admit they fail to take simple steps like locking their phones and using secure, complex passwords for all their online accounts and services.

Half of those surveyed said they rely on their memory to remember all their passwords, suggesting they're choosing weak combinations or using the same one on multiple sites, which the experts warn is an invitation to hacking. A majority of users left their social media and email automatically logged in on their phones, and one out of five did so for online shopping sites.

Half of all users also admitted they knew the login details for another person's mobile device.