Valuations do appear attractive at 12.9 times 12-month forward earnings, a nearly 27 percent discount to the long-term average, according to data from Nomura.
But it's clear some of the optimism has been shaken. Allocations to Japan have dropped to a 12-month low, with only a net 16 percent of fund managers overweight on the market, according to Bank of America-Merrill Lynch's fund manager survey for March. So far this year, foreign investors have pulled around $17.75 billion out of Japan equity funds, according to data from Jefferies.
(Read more: Why you should be patient with Abenomics)
Last week, Goldman Sachs cut its three-month and six-month targets for the Topix index, citing the market's "unanticipated weakness" so far this year and limited near-term catalysts.
It now expects the Topix at 1200 in three months and 1300 in six months, down from 1350 and 1375 respectively. The Topix closed Wednesday at 1172.07.
(Read More: Why Japan shares can't get fresh traction)
Goldman attributed the weakness so far this year to a combination of domestic and external factors. In addition to headwinds from weaker-than-expected U.S. economic data, concerns over Chinese growth and the political tension in the Ukraine, Japan stocks face concerns over the upcoming consumption tax increase and the lack of progress on structural reforms as well as foreign selling and limited domestic buying of shares, Goldman said.
Analysts have been warily counting down to the planned consumption tax increase to eight percent from five percent in April, as it is expected to dent corporate earnings and consumer spending. Many expect the Bank of Japan will step in with further easing measures afterward.