Goldman Sachs confirms that Lee is an employee at the firm and is on leave following the charges. He is the head of a group within the equity capital markets division that handles convertible securities. Last year the group was named the top "structured equity house" by International Financing Review.
Lee is also a former chairman of the board of Safe Space New York, a nonprofit group that helps at-risk youth and their families, according to a former associate.
Lee's criminal defense lawyer Edward Burke Jr.—a well-known fixture of the Hamptons legal community who has represented a slew of high-profile clients including Brooklyn Nets coach Jason Kidd—strongly disputed the allegations against the financier when contacted by CNBC.com.
"He adamantly denies this happened," Burke said. "He is looking forward to clearing his name in a court of law, with the support of his family."
"He's in the fight of his life right now, and the truth will come out," Burke said.
A source familiar with the case told CNBC.com that Lee—who married a fellow Goldman exec named Alicia a few years ago—met the 20-year-old accuser for the first time when he was out with friends at the hot spot Georgica in East Hampton. Alicia was not with the group.
Lee, his friends, and the woman later returned to Lee's house, where they continued drinking and carousing, according to the source.
In the early morning, one of the people at the party said that their car had been stolen, which led to police being called, the source said.
The accuser works in the Hamptons, the source said.
—By CNBC's John Carney. Follow me on Twitter