"Japan's economy never really recovered for 20 years. It continued to stumble along," he said. "The U.S. economy is recovering. It's patently obvious from the data," he added, citing last week's employment data.
Nonfarm payrolls data Friday showed the U.S. added 217,000 jobs in May, in line with expectations, with the unemployment rate at 6.3 percent.
Read More Whatever happened to the great rotation?
O'Friel, however, doesn't believe the data is still a bit of a drag, with the economy only now getting back to break even on an absolute basis with the pre-financial crisis levels and it still lags when population growth of around 3 percent is included.
"We're still 7-9 million jobs short of full recovery," he said.
At the same time, on the Japan side, O'Friel isn't as convinced that its economy is necessarily an underperformer.
"We hold Japan to an impossible standard," O'Friel said. Japan's economy grew to become No.2 globally, before slipping back to No.3, he noted.
"When people talk about recovery, they talk in league table terms of how do they get back to where they once were. They won't," he said, citing headwinds from a declining population and slower productivity growth.
But O'Friel noted the U.K. was once the biggest economy globally, but now ranks fifth or sixth. "To not get back to number one or number two isn't necessarily a failure."
Read More Recent bond-yield creep is start of something: Pros
But there may be another obstacle to U.S. Treasury yields dropping around 2 percentage points to match JGBs: they've simply never been that low before.
U.S. Treasurys' absolute lowest yield in the midst of the Federal Reserve's latest incarnation of quantitative easing --which it is currently in the process of tapering -- was around 1.5 percent, Matthews noted.
—By CNBC's Leslie Shaffer. Follow her on Twitter @LeslieShaffer1