In early Asian trading on Monday, the yen was hovering just below 98 to the dollar. The yen has seen some strength in recent weeks as a partial U.S. government shutdown weighed on the U.S. dollar.
Nishioka said there were several other factors weighing on exports as well, including the fact ongoing nuclear power plant closures following the March 2011 tsunami and Fukushima power plant disaster, were still bumping up Japan's fuel import bill.
(Read more: Why Japan should fear US shutdown: IMF)
"In the beginning of this year, we had expected that some parts of the nuclear power plants would resume in a few months. But at the moment most of the nuclear power plants are now under inspection. It means that... the Japanese economy will have to pay further import costs to the energy consuming countries," she added.
However, IHS' Nash pointed out that one reason why exports had not picked up as much as hoped could be because many Japanese goods are increasingly being manufactured overseas in South East Asia.
"The Japanese are becoming increasingly confident with branded goods manufactured in other places, whereas previously they were only comfortable with branded goods if they were made in Japan," he said.
(Read more: Will Japan's elderly get burned by 'Abenomics'?)
"Although the money still comes to Japanese companies the volume of those exports is actually coming out of other countries out of the name of Japanese companies," he added.
—By CNBC's Katie Holliday: Follow her on Twitter