"Given (Fed Chair) Janet Yellen's last comments highlighted 'noise' in the inflation reading, these comments (from Bullard) have some significance," said Stan Shamu, market strategist at IG, in a note.
"In fact, most of the Fed comments this week have been quite hawkish and perhaps this resulted in the subdued performance in equities."
Also weighing on sentiment was data showing that U.S. consumer spending rose less than expected in May, which led many economists to pare back growth forecasts for the second quarter.
Friday will be a quiet day for data, with just the final reading of the University of Michigan's June sentiment survey due.
Few companies are due to post quarterly earnings, with Commercial Metals and KB Home reporting before Wall Street opens.
In corporate news, major brokers like Deutsche Bank, RBC and ITG are severing ties to the dark pool operated by Barclays, wires reported on Thursday. It follows fraud charges by the New York authorities.
In addition, the New York Times reported that BNP Paribas is set to plead guilty to criminal charges and pay an $8.9 billion fine to the U.S. government. This comes after a widespread investigation into the French bank for transferring billions of dollars to countries blacklisted by the U.S.