The first African American to hold the attorney general position, Holder is one of the last few officials who have stayed in their posts since the beginning of Obama's presidency. He is the fourth-longest serving attorney general in U.S. history.
Obama highlighted some of Holder's achievements during his tenure, including overseeing hundreds of terror cases, the largest mafia take down in American history, and helping to "safeguard our markets from manipulation and consumers from financial fraud."
"We could not be more grateful for everything you've done," Obama said to Holder at the press conference announcing the resignation.
Read MoreHolder must send message to Wall Street: Attorney
For his part, Holder thanked the president, saying "we have been great colleagues, but the bonds between us are much deeper than that—you have been there for me."
Last week, Holder said in a speech that he promised to bring charges against individuals on Wall Street with evidence of criminal conduct, especially relating to the financial crisis.