The two had already won a rare court order to temporarily block the deal, which they said undervalued Xerox. A New York judge last week agreed that CEO Jeff Jacobson had been "hopelessly conflicted" in negotiating a deal that would put him in charge, since he knew the board was looking to replace him.
"We believe Friday's decision and this agreement mark a watershed moment for corporate governance generally and for Xerox specifically," Icahn said in a statement.
In addition to Jacobson, six others of the 10-member board will be replaced by members backed by Icahn and Deason, while the two shareholders will withdraw their proxy fight as part of the settlement.
Fujifilm, which saw its shares drop 5.5 percent in Tokyo after news of the ouster, said it planned to file an objection with a court over the settlement.
It will also appeal last week's court ruling, it said in a statement, adding that Xerox's new board had an obligation to comply with agreements in January.
In a complex deal, Fuji Xerox would buy back Fujifilm's 75 percent stake in their joint venture before Fujifilm would then purchase 50.1 percent of new Xerox shares. Ultimately, Fujifilm would take control of Xerox, whose brand became synonymous with photocopying.