The news sent the euro to a one-month high against the dollar.
"This is a very strong number, it was surprisingly strong across the board," Chris Williamson, chief economist at Markit told CNBC. "Leading the growth is Germany, with strong growth in manufacturing and services and in France, which has been a weak spot in these surveys, the manufacturing there even returned to growth."
(Read more: Euro hits one month high after euro zone PMI surprise)
France's composite PMI was at a 17-month high with a reading of 48.8 up from 47.4 in June. Meanwhile, Germany's composite figure came in at 52.8 from 50.4 in June.
While the readings were better than expected, one economist warned that the outlook for the euro zone economy remained weak.
"It's encouraging to see the improvement but even my very downbeat forecast on Europe really requires the PMI to go up by one point this year and by another two points next year and that's just to be able to clock in a little bit of growth," Michala Marcussen, head of global economics at Societe Generale told CNBC.
Click here for the latest on the European markets.
Correction: This article has been updated to reflect that euro zone composite PMI rose above 50.0 for the first time since January 2012.