The survey, conducted in December, was based on 5,000 U.S. consumers ages 18 and older.
Read MoreThree wearables you would actually want to wear
"I think a lot of people have been saying that they expect smartwatch sales to start stealing from fitness trackers," said Weston Henderek, director of connected intelligence at the NPD Group. "We're not convinced of that."
One reason behind the demographic divide is that fitness trackers are more mainstream than smartwatches, Henderek said. Whereas one in 10 U.S. adults owns a fitness tracker, smartwatches only have about 2 percent penetration, according to NPD data.
Over the holidays, defined by NPD as the five-week period from Thanksgiving to Christmas, more than 2.5 million digital fitness devices were sold, according to the firm. That compares with only 250,000 smartwatches.
Henderek added that many early adopters of fitness trackers were workout buffs who have since abandoned the technology, after finding they could monitor many of its capabilities on their own. As a result, the typical buyer has shifted more to those who are just starting to exercise.
Read MoreSears, mobile and Apple Pay: What to watch in 2015
Smartwatch users, on the other hand, tend to be more tech- than fitness-oriented, he said. That's because fitness activities often take a back seat to the technology's other features.
"There's a pretty big difference in terms of the reasons why people purchase [these] things," he said. "Sometimes when you get a younger demographic, even if their income is a bit lower, that doesn't mean they're not going to shell out the money for the latest and greatest device."