Ask any investor in India what their biggest concern is, and there's a good chance they'll give two words: "black money."
Black money, the moniker for income that is shipped out of India in order to evade taxes, is estimated to have grown to a total of $500 billion—a significant sum in a country with a gross domestic product of $1.87 trillion. Now, new Prime Minister Narendra Modi is moving to reassure investors both foreign and domestic by making black money a thing of the past.
A crackdown on the rampant practice may start imminently. The Indian government this week handed over to the country's Supreme Court a list of more than 600 people with unauthorized foreign bank accounts.
Read MoreModi promises US CEOs a return to market reforms
But the question is, will the moves help raise investor confidence?
"We will need demonstrable punishment: send people to jail and keep them there," recommended Suhel Seth, managing partner at Counselage India, a brand marketing consultancy. Without retribution, he said, individuals who evade taxes will likely commit the same acts again.
Others say India's government needs to go beyond making a list of alleged perpetrators and focus on how to improve the country's transparency, which has scared many foreign fund managers from investing in India.
Ron Shah, head of investment-focused fund Jina Ventures, said that "under-the-table" recycling of capital through black money channels has impeded the development of credit and accountability for India.
Boost for the little guy?
Tackling the black money problem could prove a boon for the little guy in India, sand Manu Bhagavan, professor of history at Hunter College.
"There is widespread feeling that 'hidden wealth,' if brought back to the country, could significantly help with its development and with the upliftment of the poor," Bhagavan said.
Regardless, experts agree that India has a long way to go before it's able to boost sentiment and ultimately cure its black money issue.
Read More India's delivery men offer prize investment
"Black Money in India has been around for as long as one can remember," said Seth of Counselage India.
Modi has designated a team that's compiling a list of corporate executives and politicians who are stashing cash overseas—specifically in tax-friendly nations like Switzerland, the British Virgin Islands and Mauritius.
Rand Corp. senior political analyst Jonah Blank pointed out that "over one-third of India's FDI comes from the tiny island-nation of Mauritius—almost always as a way for businesses to avoid the tax collector."