The terrible start to the new year for stocks and oil and concerns about whether China's economic slowdown might sink the U.S. recovery have been overblown, Legg Mason Chairman and CEO Joe Sullivan said Wednesday.

Excluding the oil patch, "you don't get the sense we're going into a recession here. Restaurants are busy. Travel is busy. Everything feels anything but a recession, anything but negative," he told CNBC's "Squawk Box."