Ten-year Treasury yields meanwhile fell to a one-week low of about 2.69 percent Wednesday.
"The curve has been flattening and we've seen two-year yields really move up, especially after the Fed last week because of shifting expectations that maybe rates could move higher sooner than expected," Eric Viloria, currency strategist at Wells Fargo Securities, told CNBC.
"The attraction with [long-dated] U.S. Treasuries also has to do with the risk-on, risk-off dynamic and when you have geopolitical tensions in Eastern Europe or concerns about growth because we've had disappointing data in China," he added.
Read MoreHas Yellen just unleashed the dollar bulls?
Ray Attrill, co-head of currency strategy at National Australia Bank, said that it was the way in which the U.S. yield curve has flattened that was impacting dollar/yen.
"Since last week, 10-year yields have come down and this is much less positive for the dollar," he said.
"The underlying message in the data is that the economy is still not gaining momentum, so it may be a case that this is reflected at the long-end of the curve," he added.
Data Wednesday showed orders for U.S. durable goods rose 2.2 percent in February from a month earlier after two straight months of falls. But the data also showed a surprise drop in core capital goods, a gauge of planned spending on capital goods.
—By CNBC's Dhara Ranasinghe. Follow her on Twitter