The greenback gained after Friday's jobs report for July showed better-than-expected employment gains, raising expectations that a further rate increase this year is likely. It has since given back those gains.
"It's been a fairly broad trend of dollar selling this week," said Vassili Serebriakov, FX strategist at Credit Agricole in New York.
Data on Tuesday showing that U.S. nonfarm productivity fell in the second quarter helped weigh on the .
Other markets including Treasuries are also seen influencing price moves.
At the same time, investors expecting that Yellen may strike a more hawkish tone at the central bank's Aug. 26 symposium in Jackson Hole, Wyoming, may be unlikely to place bets until closer to the event.
"There is a window of opportunity to be short dollars without the threat of a hawkish Fed message for a couple of weeks," said Serebriakov. "That is probably partly driving the market."
The dollar index against a basket of currencies fell 0.48 percent to 95.70, after hitting the lowest since last Thursday. The greenback was 0.63 percent weaker against the at 101.29. It was 0.56 percent lower against the euro at $1.1175.
The next major U.S. economic indicator will be Friday's retail sales report for July.
Reduced liquidity as investors and traders take summer vacations is also seen as exacerbating price moves.
The hit $0.7713 after it advanced to a more than three-month peak of $0.7756, buoyed this week by Australia's relatively high yields and stronger investor appetite for risk.
"Part of the Australian dollar's resilience is the lack of follow-through in pricing for a Fed hike in September, limiting the U.S. dollar's gains," analysts at Westpac said in a note.
The British pound recovered some of its recent weakness, rising 0.05 percent to $1.3007, recovering from $1.2956 on Tuesday, its lowest since July 11.
The Reserve Bank of New Zealand is expected to be the next central bank to ease conditions, by cutting rates on Thursday by 25 basis points to 2.00 percent.
The kiwi gained 0.78 percent against the U.S. dollar to $0.72.