Gold prices whipsawed after policy-makers decided to keep its wording on the timing of any interest rate hikes out of concern that a change could be misinterpreted by financial markets.
Spot gold was last down 0.6 percent at $1,191 an ounce, after hitting a high of $1,202.30 earlier. U.S. gold futures for December delivery settled $3.20 lower at $1,193.90 an ounce.
Minutes of the Fed's Oct. 28-29 meeting show that Fed officials renewed a debate from their September meeting about whether they should alter language they have used since the spring that they expected to keep a key short-term interest rate low for a "considerable time" after halting monthly bond purchases.
The Fed did decide at the October meeting that the economy had improved enough to halt the bond purchases but they kept the "considerable time" language because of worries that removing it would be misinterpreted.
The Fed's first rate hike is currently not expected to occur until mid-2105.
Speculation is also building over Switzerland's gold holdings ahead of the Nov. 30 vote on a proposal that would force the Swiss National Bank (SNB) to hold at least 20 percent of its assets in gold.
Initially flat, prices plunged more than 1 percent after an opinion poll released on Wednesday showed support for the measure declined to 38 percent from 44 percent last month.
The chairman of the Swiss central bank said in an interview earlier this month that a vote in favor of the motion, which would also oblige the SNB to repatriate gold holdings and end all sales, would be disastrous for the country.
Read MoreCentral banks: The new gold bugs?