"The old game was very simple. If you worked for the government, you had job security, you got less but you got a longer pension," he said. But over the past 10 years, a job in public sector is paying significantly more and the benefits "are literally bankrupting the states."
New York Governor David Paterson told CNBC the state needed to balance its budget by cutting spending and reducing the size of the public sector.
Austerity measures have started in Europe, with protesters across the continent taking to the streets. On Tuesday, Greek protests turned violent, with police firing tear gas at demonstrators who were throwing sticks.
Critics have slammed Greek public sector workers for not accepting austerity measures to help their country go through the difficult times, but Zell said the Greeks were not the only ones against austerity.
"How are they any different from the public employees in New Jersey?" he said.
Decline and Fall?
The US could have the same fate as ancient Rome if there is no reform in the public sector, Zell warned.
"How did Rome fall?" Zell asked. Rome fell because people agitated for "more entitlements, more entitlements, more entitlements, until there was no more Rome," he said.
The government should stand back, reduce regulation and reduce impairments faced by people trying to better themselves, according to Zell.
"I think the most significant issue of America is that this country is built on aspiration. The greatest loss we can have is loss of aspiration," he said. "Effectively, ambition, aspiration, these are what create opportunity, not government programs."
The business sector needs to have confidence that the administration is taking the right measures to create jobs, Zell said. The key to US economic growth has been the predictability of its business climate and this is something that must not be destroyed, he said.
Zell also added that European officials have no authority to tell the US to spend cautiously, because, despite recent austerity measures, Europe has still not solved its debt crisis.
"I don't think they have any basis … to lecture us," he said. "We need to be careful that we don't follow them … in their inability to control government costs."
Economist Nouriel Roubini warned that the US economy could be affected if Europe goes through a double dip recession.
Investors should look for assets of value or for countries where demand has not been affected by the crisis, according to Zell.
"First of all, at least at the moment, our emphasis in the US is in buying distressed debt. At the moment at least, that represents the most value out there," he said.
"Offshore we're much more of an equity investor," he said, naming Brazil and China as his preferred markets.