In the internal memo, Goldman said it immediately reported the breach to its compliance unit and opened an investigation. The bank also contacted the New York Fed and fired Bansal's manager, Joseph Jiampietro, for failing to recognize the breach. Law enforcement officials were notified by the New York Fed and they too are conducting an investigation, though Barket maintains there are still questions about the kind of information that was shared.
Read MoreNext for banks: Layoffs, chat smackdown, more regs
"It's not entirely clear if the information was confidential," Barket said, who said he is working with the U.S. attorney's office to make that determination.
A person familiar with the situation said the information pertained to exam information related to an as-yet unidentified midsize New York bank with which Goldman was working.
The FBI and New York's Department of Financial Services are also looking into the matter.
According to a person familiar with the situation, Goldman contacted the bank that was the subject of Bansal's report, telling it a Goldman employee had illegally obtained confidential supervisory information about the midsize bank. The source would not say what the bank's reaction was to the news. Both Goldman and the New York Fed declined to say if they had reached out to the bank.
As a regulator, the New York Fed is allowed certain exemptions from making information public about the entities it regulates. Some of this information is known only to the Fed; some of it is shared only with the banks. This information can provide those who see it with valuable insights into a bank's financial, compliance and legal issues.
Goldman did not say how many people at the bank saw the information, though a source said the initial investigation suggests it was limited to a small number of people.
Read MoreGoldman's elitegroup gets 78 new members
As for the New York Fed, it said in a statement it has "detailed rules and controls to protect confidential information." It said all employees are trained in how to protect it. Still, The New York Times, which broke the story, reported Bansel reached out to the Fed for clarity over what work he could and could not do in his new role at Goldman Sachs, but the guidance he received from the Fed was unclear.
Bansal's lawyer did not immediately respond to a request for comment from CNBC.
This is the latest scandal to tie Goldman to the New York Fed. On the day Bansal was fired, former New York Fed employee Carmen Segarra released audio tapes of her former supervisors talking about Goldman. Segarra alleges the tapes show her superiors at the N.Y. Fed "were soft" when it came to overseeing the investment bank.
It is a view held by some other more vocal critics like Massachusetts Sen. Elizabeth Warren. She has called out regulators and banks for being too cozy, a sentiment reinforced by what is referred to as the "revolving door" between Washington and Wall Street.
Many regulators get their start at the banks, including the current president of the New York Fed, William Dudley, who is a former partner at Goldman, and former Treasury Secretary Hank Paulson, who once served as Goldman's CEO.
Read MoreUnder fire incommodity probe, Goldman makes move
The tide flows the other way as well. Goldman declined to say how many of its 33,500 employees used to work for regulators, but one of them is Charles Himmelberg, a newly minted partner and head of global credit at Goldman. He joined Goldman in 2005 and prior to that worked for the New York Fed.
One lawyer who spoke to CNBC on condition of anonymity said it is almost impossible not to have cross-pollination between regulators and the banks, though clearer guidelines and regulations over what constitutes confidential information, and a person's role at their place of employment, could help prevent wrongdoing.