And that fear is being seen in the charts. Roughly half of the stocks in the S&P 500 are trading below their 50-day moving average and about a quarter of the index is trading below their 200-day moving average. The S&P 500 index itself closed below its 100-day moving average on Friday.
All of this is happening as the market is heading into the busiest time for corporate earnings.
So, what does this mean for investors?
"The key is to let the market tell you what it wants to do," said CNBC contributor Andrew Busch, editor and publisher of The Busch Update. "And, right now, it's starting to tell us that it's turning over a little bit."
Busch notes that over the past year, each successive pullback's low was higher than the previous one. Thus, even though the S&P 500 is now below its 100-day moving average, he sees the uptrend still intact. "So far, this is not telling you necessarily that we're going to break the trend," said Busch.
But, there's one level that Busch thinks could change everything: the lows from Feb. 3.
"1,737 is really where you should get quite nervous if we break that level," said Busch. "It changes the pattern of the S&P 500 that we've been in for at least a year. We would have put in a lower low and that's really what I'm looking for to change my portfolio mix of what I have."
Though he thinks the market will be volatile for the a while, Busch is holding off on taking any action.
"Until we break that level," said Busch, "I'm not really going to do a whole lot."
CNBC contributor Gina Sanchez, founder of Chantico Global, said that based on the market's recent performance, the fundamentals are telling a slightly different story than the technicals.
"If you look at the sectors that have performed best over the last few months, it's been sectors like utilities [and] health care," said Sanchez. "Those are pretty defensive sectors."
Though the health-care sector of the S&P 500 was down 4 percent last week, it's still 0.4 percent ahead for the year. The only other S&P 500 sector up so far in 2014 is utilities, which have gained 9.6 percent year to date.
"A lot of this is the market stepping back and looking at valuations," said Sanchez. "We could see some continued selling and that pressure is going to be in the highest [price-to-earnings] names and the highest beta [market sensitive] names. … That's what's in trouble."
To see the full discussion on the S&P 500, with Busch on the technicals and Sanchez on the fundamentals, watch the video above.