On Thursday, Atlanta Fed President Dennis Lockhart said the U.S. central bank's last policy statement was deliberately trying to convince investors of a possible December hike and was successful in doing so.
"All of this pressure has been due to the increasing expectations for the Fed to move in December," said David Meger, director of metals trading for Vision Financial Markets in Chicago.
"That has strengthened the dollar against a basket of currencies and pressured the precious metals complex across the board."
Spot gold fell to $1,103.90 an ounce, its weakest since Sept. 16 earlier and was down 0.2 percent at $1,104.86. U.S. December futures settled down 0.2 percent at $1,104.20.
Bullion, which has lost around $60 during the past six trading sessions, is about $30 shy of a 5-1/2-year trough of $1,077 hit in July.
As gold pays no interest, the rise in returns on U.S. bonds and other markets is seen as negative for the metal.
Read MoreYou shouldn't own any gold now: Portfolio strategist
"Bullion weakened after perceivably hawkish monetary policy comments by... Yellen," said HSBC analyst James Steel. "The next focus for the gold market may shift to the upcoming release of nonfarm payrolls data on Nov. 6."
A robust U.S. jobs report on Friday could trigger another sell-off in gold, which is already facing weak technicals and investor outflows.
Assets in SPDR Gold Trust, the top gold-backed exchange-traded fund, fell to 680.11 tons on Wednesday, the lowest in six weeks.
Palladium dropped as much as 4.2 percent to a six-week low of $600 an ounce and was on track for its biggest weekly loss in more than four years. The metal suffered after breaking technical support levels and as investment in exchange-traded products continued to fall.
Platinum fell 0.7 percent to $945.50 an ounce and silver dipped to a one-month low of $14.90 an ounce.