If building activity returned merely to its postwar average proportion of the economy, growth would jump this year to a booming, 1990s-like level of 4 percent, from today's mediocre 2-plus percent. The additional building, renovating and selling of homes would add about 1.5 million jobs and knock about a percentage point off the unemployment rate, now 6.7 percent. That activity would close nearly 40 percent of the gap between America's current weak economic state and full economic health.
So what is holding housing back?
Sure, a glut of housing was built during the last great mania, and in some markets buyers are still working through those supplies. Bank lending is only now thawing, both for homebuilders and buyers. But those restraining factors have eased a lot in the last few years. The bigger thing holding back housing is simply demand. Fewer people can or want to fulfill the American dream of starting a household of their own.
It may yet prove to be temporary, but for now at least, millions more people are doubling up with roommates, living at home with parents and otherwise finding ways to avoid doing the one thing that would get the housing economy back to normal: buying a home.
Read More A one-year plan for new homebuyers
The housing market in Roanoke, Va., is a good place to see what's happening. A city of just under 100,000 people in the foothills of the Blue Ridge Mountains, Roanoke has a history as a rail and manufacturing hub and nowadays has a strong health care industry. It also has this distinction: Its housing market—based on measures like housing prices and building activity—closely matches the nationwide data.
The number of new houses and apartments that are needed in the United States is determined over the long term largely by demographics—immigrants arriving and young people moving away from home. From 2000 to 2007, the number of households rose 1.24 million a year on average—about what economists would expect, given those demographic trends.
Add in the 300,000 or so homes that fall into disrepair each year and need to be replaced, and builders would have to construct around 1.5 million homes a year to keep up with the longer-term demand.
During the boom, builders were much busier than that, putting up 2.1 million more houses from 2000 to 2006 than if they had stuck to that 1.5 million trend rate.
Read MoreNew home sales plunge in March setback
But the correction underway since the housing bubble burst has been far more severe than the overbuilding that preceded it. From 2007 to 2013, builders constructed 4.8 million fewer homes than they would have had they kept to the trend rate. In fact, if the challenge was solely to work through the 2.1 million "extra" homes created during the boom, that job would have been finished around the middle of 2010.
"We built a heck of a lot of homes during the last housing bubble," said Stan Humphries, chief economist of the real estate company Zillow. "And it's taken some time to work off that glut. But we have largely worked off that glut at this point. Now our main challenge is housing demand, and that means we need more people forming households."
Roanoke never experienced the sort of extreme bubble in prices and construction of a Phoenix or a Miami, or the structural economic decline of the industrial Midwest, or the kind of humming regional economy that spurred a speedy rebound in the likes of San Francisco and Washington. In that sense, Roanoke is like hundreds of midsize cities nationwide that are trying to climb out of a long, deep downturn.
Back in 2005, builders in the Roanoke metropolitan area took out permits for more than 1,600 new housing units. At the low point, in 2009, they took out just 449. The rebound since then has been modest, with only 656 permits issued in 2013.
Read More 5 housing trends for spring
Peter Fields, the president of Fields Construction, was responsible for more than a few of those 1,600 homes started in 2005. That year, he began work on a 100-house subdivision called North Oaks, in neighboring Salem, Va. Just as the empty streets of North Oaks started filling in with new houses that were to be priced at approximately $300,000, the housing bust hit and demand collapsed.
"A lot of builders just quit building," Mr. Fields said. "They parked their trucks and decided to do something else." Banks stopped lending for new development. Nearby subdivisions that had been in the planning stages remained only blueprints. Houses that had been started before the bust sat empty, waiting for buyers.
Now, though, those houses have owners, and new houses are rising. But things aren't good enough to lead a builder like Mr. Fields to crank up the pace of production. Whereas three years ago a builder would offer a $20,000 discount to a willing buyer, now the discounts run only $5,000 or so. Thanks to a run-up in prices for supplies like lumber and wallboard, his profit margins remain squeezed.