"It's not going to be a set of austerity measures that have an immediate impact on the economy, they're going to map out a path that sees a gradual consolidation of the budget, bottom line," he added.
Australia's economy has enjoyed over 20 years of annual growth, weathering the global financial crisis thanks to a mining boom fueled by strong demand from China – the country's biggest trading partner.
But a slump in mining investment and a slow response from other parts of the economy to record low interest rates has hit tax revenue at a time when expenditure has grown.
A report by the National Commission of Audit last week called for hefty cuts to family benefits and new charges for health care and medicines.
Read MoreAustralia's central bank leaves rates steady at 2.5%
According to local media reports, the budget will unveil tougher means testing for welfare benefits is expected and rise in the retirement age to 70. The previous government had already unveiled plans to lift the retirement age to 67 by 2023 from 65 today.
In addition, Australians earning more than A$80,000 per year may have to pay an additional 1 percent in income tax. Those earning over A$180,000 could pay an extra 2 percent tax for a temporary period.
"The fiscal deficit tax – that's really been what has worried households," said Katrina Ell, associate economist at Moody's Analytics. "They don't want to pay a deficit tax to get the budget back into surplus when they don't believe that the budget being in a deficit is such a big issue. It is likely that they [the government] probably won't go through with it or it will be scaled back."
Hold on a minute
Analysts say that measured by international standards, Australia's fiscal position is in good shape.
Its budget deficit as a percentage of gross domestic product (GDP) is forecast at 2.5 percent in 2014 by the Organization for Economic Cooperation and Development (OECD). That compares with a budget deficit forecast for OECD peers Japan and the U.S. at 8.5 percent and 5.8 percent respectively.
Read More OEC cuts global growth forecast, warns of complacency
"The fiscal drag will grow a little bit with the Liberal government but it's not the scenario we've seen in Europe where austerity measures really cut growth and have been dramatic," said Ell. "If you look at global comparisons you can see that Australia has very healthy government finances."
Economic data also paints a brighter picture with positive signs from the real estate and labor markets. Employment in Australia rose by 14,200 in April to top forecasts for a third month running, data on Thursday showed.
"An extended period of low interest rates is finally having its wicked way with retail and with housing construction," analysts at Deloitte Access Economics said in a note Thursday. "And as spending lifts, so too does the take from spending taxes such as the GST [General Sales Tax]."