In addition, while a weaker yen is generally considered good for the country's exporters as it makes their products more attractive overseas, "exporters also buy raw materials, and that could mean higher prices," he said, noting that Japan's biggest import is energy, which is priced in U.S. dollars.
Merner believes that if the yen continues to weaken, it may affect demand for Japanese government bonds (JGBs).
"If you have inflation, then people want higher interest rates," he said. "Even though you've seen rather bad returns on government bonds, the fact that until recently there was no inflation [meant] you were getting real returns."
Read More Japan inflation rises in July but remains short of BOJ target
He expects that if the yen continues to weaken, authorities may intervene, but he isn't sure what level would spur action.
The speed of the yen's rise is another concern, with Reuters reporting that Japan's economy minister Akira Amari said rapid moves in currencies are undesirable.
"It's very fast," said Greg Gibbs, senior foreign exchange strategist at RBS. "It's caught a lot of global investors on the hop," he added, citing wide expectations that the pair would remain range bound.
Read More Japan stocks set to ride new wave of yen weakness
But Gibbs believes the yen is catching up to its fundamentals, including a deteriorating external balance, rising inflation, falling yields and rapid policy easing as well as increasing expectations that GPIF, the government pension investment fund, will decrease its allocations to JGBs as it increases allocations to risky assets.