A Chinese ministry spokesperson said claims are "unfounded accusations based on preliminary results," and that "China resolutely opposes hacking actions and has established relevant laws and regulations, and taken strict law enforcement measures to defend against online hacking activities.''
But China's denial of not knowing about the attacks is weak, Mandia said.
(Read More: Security Firm Discovers Cyber-Spy Campaign )
"China has a controlled Internet access, everything people do on the Internet is monitored there. We have seen thousands of attacks just in the last two years alone," Mandia said. So it's hard to believe ... that the Chinese government does not notice thousands of attacks coming from a neighborhood that happens to be co-located with units 61398, it's hard to believe they don't notice."
The cyberattacks, though not destructive, targeted valuable intellectual property.
(Read More: Is Your Business Ready for Cyber Warfare? )
"One of the things we've learned, is it's the theft of intellectual property they are after. It's not credit card data. ... It appears to be copying sensitive documents," he said.
If the U.S. wants to prevent future attacks, like the string of recent attacks on U.S. companies and government organizations, it's going to have to intervene somehow, Mandia said.
"The response can't just be technical. That's just a fact. You're not going to be able to prevent all these intrusions, there's going to have to be technical and diplomatic-based counter measures to these attacks," he said. "It's time to say that the attacks attacks weren't just coming out of China, but in fact, there are ties to the Chinese government in these attacks. But there is no clear single answer, there's a lot of complexities here. We'll be trying to solve this problem for a long time."
—By CNBC's Cadie Thompson