India and Japan are clearly at different stages of development. One-third of the world's 1.2 billion extreme poor, who are living on less than $1.25 a day, are in India, and less than a third of the developing country's 1.24 billion people live cities and towns. Its gross domestic product (GDP) per capita is around $4,000 on a purchasing power parity (PPP) basis.
Read More A fifth of developing world lives on below $1.25 a day
By contrast, Japan enjoys a relatively high GDP per capita of around $37,000 on a PPP basis, and more than 90 percent of its population lives in cities.
"Japan, being a wealthy net creditor nation can be an important investor in India's much-needed infrastructure. Japan, by outsourcing production, can also play an instrumental role in bringing India into Asia's elaborate cross-country manufacturing supply chain, as it has done for so many other Asian countries over the decades," Nomura said.
The proportion of overseas production by Japanese manufacturers grew to more than 20 percent in 2012, and it's expected to rise to more than 25 percent by 2018, Nomura noted. But while many Japanese companies have been expanding in China, they've recently shifted focus as costs there have risen, it added.
Nomura is particularly optimistic on the infrastructure segment as part of Japan's growth strategy includes measures to step up infrastructure-system export orders.
Read More Is Carrefour's India exit a setback for Modi?
"India is a natural fit for this strategy and would be a willing recipient of such expertise as infrastructure development is so high on the government's agenda," it said.
Others also expect a pickup in Japanese investments onto the subcontinent.
"Bilateral economic relations between Japan and India are set to blossom," said Rajiv Biswas, chief economist for Asia-Pacific at IHS. Not only have Modi and Abe established a good rapport on a personal level, "Japan-China relations have soured over some disputed islands, which triggered anti-Japanese riots in China in 2012," Biswas said. Over the last 18 months, Japan's foreign direct investment (FDI) into China plunged, he noted.
Read More Wal-Mart cozies up to China as it shelves India
"India, as Asia's other BRIC economy with a large consumer market of 1.2 billion people, has therefore become more attractive to Japanese companies both as a market and as a location for low-cost manufacturing production," he said, noting Japanese FDI into India already exceeds $16 billion.
To be sure, there are significant headwinds to increased investment in India.
The World Bank rates the country at 134th on the list of best places to do business, out of 189 countries.