"It's always been a challenge, particularly in the rural areas where the population is not literate. They are scared actually to go to the bank," said Ram Sangapure, an executive director at number four state-run lender Punjab National Bank.
Lambodar Mishra, 37, who runs a small grocery shop in the eastern state of Odisha, was one of many villagers the government targeted.
He and his two young sons opened accounts last year but have never gone back to the bank 5 km away from their village.
"Why do I need a bank? Whatever I earn, I put in my business, I buy things to sell in my shop. I don't earn a lot."
Many in rural India have been left behind by modernization.
World Bank figures show a majority of people in India live on less than $2 a day. Low rates of literacy have also deepened suspicion of paperwork and plastic and popularized cash.
Read MoreIndian billionaire worries about disruptive technology
Industry experts say payment mechanisms could be improved while the government could provide more subsidies directly - a process which could take several years to complete - to encourage the use of bank accounts.
Banking on it
State-run banks dominate India's banking landscape and have driven the government's inclusion campaign. But they are struggling with bad debt and slow lending growth, and getting Indians to use the new accounts will make the difference between a long-term cost and a long-term opportunity.
"These accounts which today we think are a liability and an expense to the bank, maybe down the line, in a year or so, these accounts will start yielding some returns," Sangapure said.
Bankers and industry experts estimate the cost of opening and maintaining a bank account at between 200 and 250 rupees ($3-$4), which means the annual cost of the empty accounts for Indian banks could add up to some 21 billion rupees ($328.3 million).
That is excluding the lost opportunity had cash been in the account - deposits are a key source of funding for Indian banks, which pay roughly 4 percent interest on savings but lend that money at rates of 10 percent and higher.
The financial inclusion campaign has seen progress. Fallow accounts made up 77 percent of the total last September, a month after Modi launched the scheme. That has dropped to just over half.
Read MoreEl Niño cloud hangs over Asian stimulus
But some say banks sorely need more direct transfer of subsidies, as well as banking products catering for the needs of the urban and rural poor, to get more people to use their accounts.
"With the full scale launch of direct benefit transfer to these accounts, we understand the usage of account would improve substantially," said Vikaas Goel, who works on financial inclusion at remittance provider FINO PayTech.
Indian federal and provincial governments subsidize a variety of commodities and services at an annual cost of nearly $59 billion, or about 3 percent of the nation's GDP.
Subsidies that started flowing into the accounts this year have helped lower the number of fallow accounts. The government plans to transfer all subsidies including food, fertilizer and kerosene, potentially taking direct payments to more than $60 billion.
In comparison, money in active accounts run under the scheme totaled about $2.8 billion as of June 3.