The IMF's warning follows comments from Japanese finance minister Taro Aso on Friday, who sounded fears over how the political impasse in the U.S. could lead to further U.S. dollar selling and yen buying, which would be bad news for Japanese exporters and the success of Abenomics overall.
Japan's domestic currency has weakened around 20 percent since Prime Minister first started talking up his Abenomics policies in mid-November last year. However, over the past week its strengthened around 1.5 percent to a multi-month high of around 96.66 to the dollar amid the shutdown fears.
(Read more: US growth in danger as shutdown heads into second week)
Schiff said uncertainty over the shutdown could lead to a surge into safe-haven assets, like the yen, which would not be desirable for Japanese policy makers.
"[This] could reverse some of the yen depreciation we've seen over the past six months or so, and that would be very unhelpful at this time," he said, adding that the recent strength was "not necessarily problematic" but a much larger appreciation would be.
On a more positive note, Schiff told CNBC the IMF was pleased with Japan's recent decision to push ahead with the scheduled sales tax hike from 5 to 8 percent next April and from 8 to 10 percent in October 2015.
The measure had stirred much debate in recent months; although it is seen as a vital step in helping Japan address its huge fiscal debt pile, others were concerned it would damage economic growth.
"The confirmation that [Japanese policy makers] are going ahead with the consumption tax next April is really a major step forward," said Schiff.
(Read More: Is consumption tax hike the 'wrong policy' for Japan?)
However, the IMF Mission Chief warned that on its own, the measure would not be enough to help Japan substantially cut its debt pile. The sales tax hike would generate roughly 2 to 3 percent of gross domestic product (GDP), he said, while recent stimulus measures would add a similar amount.
"That only gets you half way to the amount of the adjustment required to begin to bring the debt to GDP ratio to a declining path... there is going to be further adjustment needed," he added.
Japan's current account surplus fell 63.7 percent in August from a year earlier, Ministry of Finance data showed on Tuesday, surprising analysts who had forecast a 23.4 percent annual increase.
(Read More: Will Japan's elderly get burned by 'Abenomics'?)
Schiff also pointed out that recent lows in Japanese Government Bond (JGBs) yields, which are trading at lows not seen since May, were a testament to the success of the Bank of Japan's aggressive stimulus program, but warned that support was needed on the fiscal side. The yield on a 10-year JGB was trading at 0.648 percent on Tuesday.
— By CNBC's Katie Holliday: Follow her on Twitter @hollidaykatie