A large number of people have missed the six-year bull market.
More than half of Americans, 52 percent, are currently not investing in the stock market—either by buying individual stocks or mutual funds, or through a retirement account such as a 401(k) or IRA, according to a new Bankrate.com survey. Among the non-investors surveyed, 53 percent said they don't have the money to invest and 21 percent said they don't trust stock brokers or financial advisors.
"It's a definite negative, especially for younger people who can afford to take some risk right now because they have time to recover from potential losses," said Claes Bell, a Bankrate.com analyst. "For most people, stocks should be some part of the mix in their overall retirement portfolio because of that ability to earn higher returns over a long period of time."
Read MoreRetirement savings last on most Americans' to-do lists
What's worse is that Bankrate's survey of more than 1,000 adults may be overestimating the percentage of Americans investing in the stock market. A study from the Federal Reserve found that only 13.8 percent of U.S. households owned stocks in 2013, down from 15.1 percent in 2010.
Financial advisors say that a big part of their job is educating new clients on benefits of owning stock.
"Once we explain that owning stocks in their accounts helps their portfolio stay ahead of inflation...most become more open to try. No one wants to run out of money," said Anna Sergunina, a certified financial planner in Odenton, Md. "We have to start with addressing their fears first. The fear is not wanting to invest in stocks, the fear is they don't understand why and how it can benefit them."
Even if the benefits to stock investing are clear, having enough money to invest remains hurdle for many Americans.
Read More7 habits of highly effective retirement savers
"When the median U.S. household income hovers around $52,000, it is not surprising that more people do not have money to invest in the market," said Wan McCormick, a certified financial advisor in Fairfax, Va.