It sounds like the perfect tax.
The pied-a-terre tax proposed for New York City would raise hundreds of millions of dollars, presumably from rich foreigners who buy multimillion-dollar, part-time apartments in the city. Overseas residents don't vote, so politicians like it—and voters may like it because rich New Yorkers hate getting out bid by rich foreigners.
In short, if any new tax on the wealthy in New York has a chance of passing, it's this one.
Read MoreNewYork City's most expensive listing: $130 million
But a close reading of the proposed tax reveals that it's not so simple. Some New York residents, it appears, could also fall under the new tax.
The proposal, first floated by the left-leaning Fiscal Policy Institute and being crafted into legislation by State Sen. Brad Hoylman, would tax the 1,556 non-primary residences in New York City worth more than $5 million.
It would start with a 0.5 percent surcharge on the amount over $5 million and rise gradually to 4 percent for home values above $25 million. It's expected to raise about $665 million a year, according to the Institute. (Even though the tax is on New York City residents, it has to be passed by the state).
Read MoreAnd the priciest city to live in is...
Yet people who have a primary residence in New York would also have to pay the tax if they have a second home in the city valued at more than $5 million.
"If someone in New York owns more than one condo and both are valued at more than $5 million, they would pay the tax on one of the properties," said an aide to Hoylman, a Democrat.
Of course, no one's going to feel sorry for New Yorkers who own two units valued at more than $5 million, but many wealthy New Yorkers own other properties in the city as investments.
And what if an affluent family is moving within New York City? If they've purchased a new home for $5 million or more but haven't yet sold their old one, would they also face the tax? Probably.
Such taxes have become common around the world as governments seek to cash in on the rush of rich foreigners buying real estate. Voters are also frustrated by wealthy buyers from overseas bidding up local property values.
The U.K. is proposing a new "mansion tax" on properties valued at £2 million or more. And Singapore and Hong Kong have recently passed measures adding special taxes and fees on foreign buyers. But all three markets have slowed recently.
Read MoreBillionaires are hoarding mountains of cash
New York's real estate market weathered the housing crisis much better than other areas of the U.S., and the proposed tax would only hit the very top of the market, which has been on fire.
Real estate lobbyists and luxury developers in New York City are already opposing the possible New York tax, fearing it could reverse this trend.
"The last thing we want to do is lower the value of real estate, which would then lower the taxes for the City of New York," Steven Spinola, president of the Real Estate Board of New York, told the website Capital.