The newly disclosed hedge fund attack began in late 2013, when hackers sent a so-called "spear phishing" email—a seemingly innocuous message that, when opened, inserted the malware onto the hedge fund's servers. The spear phishing emails appeared as if they were about the capital markets industry, in order to make it more likely that the hedge fund employees would click on them.
The details of the attack were provided by BAE Systems and were not independently verifiable by CNBC.
Over the months after the email was sent, financial analysts and IT managers at the firm noticed two problems that they did not initially realize were connected. At first, the firm noticed that its algorithmic trading strategy—a computer-based trading system that depended on high-speed trades—had suddenly become ineffective. Upon investigation, the traders discovered an unexpected lag time between when they were issuing trade orders and when those orders were executed. The delays the attackers added to the trading software ranged from hundreds of microseconds to the low-single-digit milliseconds. BAE's analysts concluded the attackers were trying to create tiny delays in the hundreds of microsecond range.
Read MoreUS companies seek cyber experts for top jobs, board seats
At the same time, the firm's IT staff observed suspicious behavior on their computer network—files being moved on the system in ways that couldn't be explained by normal business operations. At that point, the firm brought in BAE Systems to analyze the IT problem, Henninger said.
Over subsequent weeks, the team found that the malware had been programmed to insert a random lag into the firm's order entry system of just a few milliseconds. The malware also recorded the details of those orders. "That piece of malware was undermining the effectiveness of that trading strategy and it was exposing the details of that trading strategy to someone who could easily copy that information out of the network and replicate it, trade ahead of it, trade around it, et cetera," said Henninger.
He said BAE systems does not know what happened to the trading data after it left the hedge fund's computers, but that the most likely explanation is that the intruders were able to reap significant profits from trades of their own in financial markets.
Henninger said the malware represented a multimillion dollar problem for the hedge fund. "This was not something that was a minor issue for them," he said. "This was something that was getting reviewed at the board level of this hedge fund precisely because it was having a material impact on performance across the portfolio."
Public disclosure of illicit trading based on hacked information is exceedingly rare.
In 2010, however, the SEC obtained summary judgment against a computer hacker for insider trading. In that case, the hacker, a Ukrainian citizen, penetrated the servers of an investor relations service that was preparing a press release for IMS Health Inc. According to the SEC, the hacker discovered that IMS Health was preparing to announce negative earnings—and executed several options trades that ultimately generated $287,346 in profits. The defendant was ordered to pay a penalty of about $580,000.