The American Federation of State, County and Municipal Employees, the largest national union for public service providers, with 1.6 million nurses, corrections officers and sanitation workers among others as members, passed a resolution in 2014 against social impact bonds.
According to the resolution, "Tying high stakes to outcome-based performance measures may motivate investors to game the system to avoid losing money, including not serving the neediest populations, focusing on selected outcomes at the expense of other aspects of a program, or rigging the rules."
In 2013, a study by the Department of Legislative Services in Maryland looked at the state's social impact bond program for reducing recidivism, and concluded that the complexity of arranging all of the moving parts was more costly to the state than what it saved.
"The additional costs of a SIB program cannot be justified by offsetting savings. Other potential benefits do not justify the cost or complexity of a SIB program either," the study's summary said.
Read MoreSocial impact investing: Making money by making a difference
Emily Gustafsson-Wright, a fellow at the Brookings Institution, who has researched social impact bonds extensively, said the benefits are more long term.
In the case of pre-kindergarten education, she said the savings for the government are probably much bigger than what they're calculating now.
"It not only reduces remedial education, it reduces crime, it reduces unemployment down the road," she said.
Social impact bonds have significant potential in the right circumstances, Gustafsson-Wright said.
"What this really is about is getting governments and service providers to focus on outcomes, and getting governments to focus on preventative services," she said. "It's a lot less risky than business as usual for governments. Billions are spent on social services that we don't know what the outcomes are."
Pennsylvania state Rep. Mike Schlossberg, who worked closely on the state proposal, said he is hoping to fund early education through social impact bonds in his home district.
He said he sees it as a chance to improve future outcomes for kids and save money.
"They're going to wind up being much more productive citizens over the course of their career," he said. "We invest a little now, save a ton of money down the line."
Schlossberg said there are challenges in the coordination, but that he supports pay-for-success contracts.
"It is a highly complex process," he said, adding that "a pay-for-success program works when you have enough data to unquestionably prove that throwing money at a problem will improve that problem."
Schlossberg, a Democrat, said both parties can get behind using social impact bonds to fund projects.
"Democrats sometimes have a tendency to talk about spending, and Republicans have a tendency talk about cutting, but this is a way."