Last year, Zelman famously declared the market was in "housing nirvana." Then last month, she admitted nirvana had taken a pause but was "around the corner" yet again. She said fundamentals are still good, and she is surprised at the bearishness among traders.
"We think housing starts will be definitely up at least double digits. We know the builders are bringing a significant amount of new product to market," Zelman said. But, she added, "I guess the expectations have just gotten so negative, and I think that it's piling on bearish sentiment with [Federal Reserve Chairman Janet] Yellen also indicating that the Fed could tighten sooner."
Indeed, in the past month, builder stocks have been hit hard: D.R. Horton is down 13 percent; Lennar down 11 percent; PulteGroup down 11 percent and Toll Brothers is off more than 9 percent.
Read MoreLooking to buy? Best cities for first-time home buyers
Some analysts continue to point to the weather impact as well as a pullback in investors' overall tolerance for risk.
"The spring selling season has been effectively cut in half in many parts of the country. That has increased the short-term risks to the equities significantly," said analyst Stephen East of the ISI Group. "We are moving away from the traditional seasonal trade of the equities, and we have investors who just do not want to take that elevated risk."
East says the market is mixed at best with lower-priced homes struggling under the weight of tighter credit and weak job growth amid younger, first-time buyers. Bright spots are in higher-priced homes and the active adult sector. In its first quarter earnings, Miami-based Lennar reported more sales of bigger, higher-priced homes. Executives at Pulte last fall said they were concentrating on selling move-up homes, as the entry-level market was rough.
Read MoreA spring thaw for mortgages?
The investor pull back may also be a function of basic geography. All real estate, and all real estate investors, are local.
"This correction in the stocks seems to be driven by an overly pessimistic view by New Yorkers and Chicago buy-siders that have been drilled with one winter storm after another," said Buck Horne, an analyst at Raymond James. "In Florida—we have two weeks of chilly temperatures and a lot of 70 degree days. I wouldn't count out this spring selling season just yet. And baby boomers—man, are they back!"
Read More Housing's mixed signals: Are we in a bubble or not?