"The service side follows the manufacturing side. I don't just look at the fact that manufacturing is a small part of the economy. It doesn't matter. If there's anything we learned in the last business cycle, it's size doesn't matter," LaVorgna said.
LaVorgna's recession odds are higher than the Wall Street average. The CNBC Fed Survey last week showed an average expectation of 24 percent among the respondents.
Read MorePrivate payrolls rise 205,000
"Let me be clear — it is not my baseline. But if we have a recession, it would likely be a very shallow recession, more akin to what we saw back in 2001. However, risky assets under such an environment go lower in price if the downside risks were to materialize," he said.
LaVorgna said the odds of recession have been rising because of the worsening financial conditions and stock market sell-off.
"I would say we entered the year with a 33 percent chance of recession in my opinion, and for every 1 percentage point the stock market falls I would increase my recession risk," he said. "Now with the stock market down 7 percent, I put the recession risk at 40."
LaVorgna said there are other warnings. "You have to be worried about recession. Profit margins have peaked. The yield curves 2s/10s are the narrowest since 2010. Credit spreads have blown out to recession levels. The dollar has exploded to the upside. Materials prices have weakened. The markets are all telling you the same thing," he said.
Stocks immediately sold off when the ISM report was released at 10 a.m. ET on Wednesday, and bond yields fell as buyers rushed to Treasurys. The 10-year dropped to 1.82 percent, a one-year low. Stocks later came off their lows.
U.S. growth has been slowing, with fourth-quarter GDP up just 0.7 percent, but economists do expect the first quarter to have growth of more than 2 percent.
The market initially reacted to but later ignored a positive report on January jobs. ADP showed private payrolls increased by 205,000, a positive sign for Friday's government jobs report.
ISM showed a slowdown in most components, with one of the most worrying the decline in business activity, down 5.6 points to 53.9. That was the largest drop since the recession, according to JPMorgan economists.
Read More Banks report drop in demand for loans