"We know on the defense side there are massive cuts which I believe are largely a function of the sequester. In the last two quarters, we cut defense spending at a 17 percent annualized rate. That is the second-largest two-quarter decline in the post-World War II period, trumped only by what took place in the middle of 1954. It hit in the fourth quarter last year just when people were getting ready for the cutbacks," LaVorgna said.
LaVorgna said the economy has adjusted to the higher taxes and spending cuts, and he believes there will be "a watering down" of the sequester by Congress when it works on the 2014 budget. "That fiscal drag, I don't believe will be as potent come August/September."
Peter D'Antonio, head of U.S. economic forecasting at Citigroup, said the tax hikes did hit the economy. "Early on there were people that were saying we're not going to get a hit because we didn't see it," he said. "By our measures, we did see it. It has happened. We had a breakdown of consumer spending. This was a tax event. You just can't look at overall consumer spending. What you really need to do is focus on the stuff that's discretionary. You want to look at necessities. It turned out in the first quarter, [the percent of] necessities shot up."
D'Antonio pointed out the quarter started out after the warmest December on record and ended with an unusually cold and stormy March, "in the process, we had a run up in utility bills. That was a part of it, but not all of it. But necessities did well and discretionary flattened out completely. We saw a hit but it was less and that has given us more confidence that our underlying story is coming through."
When revised first-quarter GDP was released this past week, first-quarter consumer spending was knocked back to a gain of just 2.6 percent, from an earlier estimate of 3.4 percent. Consumer spending accounts for two-thirds of economic output. The Commerce Department released data for May Thursday that showed consumer spending rose 0.3 percent in May, erasing a loss of the same size in April.
In May, consumers went for big-ticket items like cars and spent more at retailers for home improvements and sporting goods. Consumer sentiment, reported Friday, rose to its strongest level in six years. The Thomson Reuters/University of Michigan's final reading was 84.1, just below the six-year high reached in May of 84.5.
Economists had expected a lower reading, but consumers believe the economic recovery has upward momentum, according to the surveyors.
"I think just in general the economy has surprised us by being a little more resilient than I thought," said D'Antonio. "Home prices have increased at a faster pace than I thought and frankly the financial conditions are better."
The S&P/Case Shiller home price index this past week showed a 12 percent increase in 20 cities in April, from a year earlier, the largest gain since 2006.
"What we're seeing is each one of those factors that has been holding us down is fading, and as those drags dissipate we could find ourselves in a better position. I would expect growth to accelerate as we go through the second half," D'Antonio said. He expects 2.5 percent growth in the third quarter and about 3 percent in the fourth.
D'Antonio said the housing recovery can withstand the increase in rates. "Of course affordability comes down, but it's still way above where it was in the housing boom. The thing that was really holding them back was the lack of price appreciation for homes," he said, noting that has clearly changed.
He said the improvement in the economy will be in large part because of the consumer. "I think the business sector will do well also. We saw some pretty nice numbers on durable goods," he said. "They fit with my story that we're going to get not great, but OK business investment."
Durable goods orders rose 3.6 percent in May, the same as April's gains. Much of the increase came from an increase in aircraft orders but there were also increased orders for computers, communications equipment, machinery and metal.
LaVorgna said he's optimistic the economy could take off, though job growth is still sluggish. "It's almost like the GDP numbers aren't accurately reflective of how things feel … if you look at housing construction and home prices, you would say the economy looks pretty strong, all things being equal. I think it feels a lot different, and I think attitudes are much better now than they were a year ago, when we were worried about the Greek exit and what the ECB would do, which is what led us up to QE3 in the first place."
Maki said he expects the Fed to start to pare back its bond purchases in September, and he expects the pace of unemployment to begin to fall more rapidly than the Fed predicts because of the drop off in work force participation, the result of baby boomers retiring from the work force.
He expects the unemployment rate to be 7 percent early next year, a level the Fed expects to see when it is finishing its asset purchases.
"We think the Fed will be tapering despite the sluggish growth," Maki said. "We view the default now as tapering and data would have to deteriorate for them not to do so."
How the markets react to that has yet to be seen. "It's an odd time for the Fed to be talking about tapering, when GDP growth is slowing, job growth is slow and inflation is about half the rate they expect it to be," Maki said.
Bernstein expects the stock market to move higher in the second half. "We still think things are going to improve but the trajectory may not be as strong as it was, and that's why we had this little correction here. People are trying to figure out what's going on. This is a normal discussion for a midcycle environment. In a midcycle environment, there's always a tug-of-war between the negative effects of a tightening of monetary policy versus improvement in the fundamentals," he said. "And in the normal midcycle environment, the fundamentals win, and the Fed is late."
—By CNBC's Patti Domm.
