While India's fiscal deficit has narrowed over the past two years it remains troublingly high.
The deficit is forecast to come in at around 4.6 percent in the 2013-2014 financial year ended in March 2014, down from 4.9 percent in 2012-2013 and 5.8 percent in 2011-2012. This is significantly higher than China, for example, which recorded a fiscal deficit of 2.1 percent of gross domestic product (GDP) last year.
"A sobering fiscal reality awaits the new Finance Minister; stagnant revenues and sticky expenditures will make the job of further consolidation difficult," said Taimur Baig, chief economist, Asia at Deutsche Bank.
Read MoreAre Indian stocks due for a post-election correction?
"If the authorities choose to expedite capital spending to orchestrate a cyclical turn in the investment cycle and boost long-term growth, the short-term casualty will be the debt/gross domestic product ratio," he said.
Improving the business climate should be the anchor of the new government's agenda, said Radhika Rao, economist at DBS.
The investment environment in the past two years has been dogged by high borrowing costs, delays in project approvals and rise in corporate debt levels.
"Hopes are that the reform push in the home-state [Gujarat] of the BJP's prime ministerial candidate [Modi], will be replicated at the center. This implies that infrastructure availability and resource-based industries will be given precedence," Rao said.
Read MoreIndia elections: The most likely business winners
"But the government will need to approach the need for higher public spending prudently, given the already adverse fiscal position," she added.
On inflation, the government must complement the central bank's efforts to contain inflation. Low and stable inflation is a necessary precursor for boosting investments by providing good visibility and costs and profitability.
One way the government can do this is by addressing high levels of food inflation.
"The government holds above the required level of rice, wheat and cereal in its stocks – they should release more into market to temper prices," Rao noted.
Finally, improvement in the current account deficit - which has narrowed to less than 2 percent of GDP from over 4.8 percent in 2012-2013 – must be sustained.
"Risks are that the current account deficit could-widen back towards 3 percent, if gold imports rebound sharply when trade restrictions are lifted, along with an improvement in the domestic investment cycle," she said.
The government has put in place several measures – including a series of import duty hikes – over the past to curb demand for the yellow metal, which is one of the biggest contributors to the country's import bill. The BJP has promised to review gold import duties within three months of coming to power.