The scant training available means that India only has 3.5 million workers undergoing skills courses a year, compared with 90 million in China, according to Indian government data.
The lack of proper training is compounded by prejudice against manual labour under the Hindu caste system, which has traditionally left jobs that might get your hands dirty to the lowest of the low.
As a result, only one in 10 workers in India's construction industry are skilled, according to government data.
Private companies fill the gap
The government has a goal to provide at least some skills to 500 million people by 2022. But private companies such as the Godrej Group are taking matters into their own hands, recruiting and training workers themselves to be ready with skilled labor when an economic recovery comes.
"I always say that there is no unemployment in India. It's only unemployability," said Adi Godrej, whose businesses range from consumer goods to real estate and infrastructure.
Larsen & Toubro (L&T), the country's biggest construction company, says it could face a labor shortage next year, just when it plans to ramp up investment after two years of slow economic growth.
It has gone out to rural areas to find recruits and bring them to sprawling training centers - such as the one in the outskirts of Mumbai, where young men practice bricklaying and putting up scaffolding - from which up to 20,000 students graduate a year, many of them joining L&T.
Read MoreOne MIT graduate's quest to turn stigma to empowerment
Yogesh Devdas Dudhpachari, 24, is one of L&T's recruits. An unskilled motorcycle mechanic, he attended an ITI to learn carpentry but ended up back at his village without a job before being taken on by L&T.
"Skill and time is valued here," he said during a break in his training. "We were not doing anything in our villages."
Even then, companies struggle to find volunteers.
Most young people eschew building and manufacturing jobs in favour of less physically strenuous work, despite data showing that wages for professions with acute shortages such as plumbers and electricians are higher than even low-level IT engineers.
At L&T's training centre in Mumbai, 25 percent of people who joined this year have left the three-month training course.
"There is a bit of reluctance ... to join this kind of trade," said Ajit Singh, executive vice president of L&T's Corporate Infrastructure and Services. "The rural youth are used to staying in their comfort zones in the villages and don't want to move out to the project sites."
Analysts say the biggest push needs to come from the government, which has already started easing decades-old labour rules and is trying to centralise a hodgepodge of bodies that over-regulate employment.
Modi has also vowed to make skills training a major plank of his "Make in India" initiative, passing through parliament a programme that will make it easier for employers to hire apprentices for two years.
Economists say India needs to go beyond numerical targets for skills training by lifting the quality of ITIs and working with the private sector to improve apprenticeship programs.
"The initiatives are in the right direction," said Chetan Ahya, Morgan Stanley's Asia-Pacific chief economist.
"However, alongside focusing on the quantitative aspect of skilled labor force the policy makers will also need to focus on the qualitative aspect of the skill programs."