Central banks should keep interest rates low in the medium term, to help bolster the economic recovery, Gurria said. His words come as the European Central Bank and Bank of England's rate-setting committees meet to debate whether to change monetary policy.
The OECD's report found that, worryingly, 16.3 million people - or more than a third of those unemployed in the OECD - had been out of work more than a year by the end of March 2014. This is almost twice the number in 2007, suggesting that long-term unemployment has become more entrenched.
"The growing risk (is) that for the many who have accumulated long jobless spells, discouragement and loss of human capital make their reintegration more difficult, that is, their unemployment risks becoming structural in nature," Stefano Scarpetta, OECD director for employment, labor and social affairs, wrote in a statement.
Read MoreThis is how we should fix inequality: OECD chief
There are also significant disparities in unemployment across the group of nations. While the U.S.'s unemployment fell to 6.2 percent in July 2014 and Japan's hit 3.7 percent in June 2014, the rate hit 27.1 percent in troubled euro zone country Greece.
Nonetheless, the OECD saw some reasons for optimism in early signs of a more broad-based pick-up in jobs, noting that the euro area -- particularly countries like Ireland and Spain -- was turning the corner and that net job creation in the United States was strengthening.
Yet the U.S. is "just one bright spot," according to Gurria.
- By CNBC's Catherine Boyle. Twitter: @cboylecnbc