Most major European bourses all closed lower on Monday. Germany's DAX ended down provisionally by 0.4 percent, while the French CAC closed down around 0.3 percent. London's FTSE 100 was shut because of a U.K. public holiday.
Yellen said the case for a rate hike had "strengthened in recent months" pointing towards the "solid performance" of the labor market in a speech at the annual Jackson Hole gathering of central bankers on Friday.
U.S. stock indexes rose on Monday.
Fed Vice Chairman Stanley Fischer backed up Yellen's view in an interview with CNBC. Asked whether there could be a rate hike next month and more than one this year, Fischer said that Yellen's comments were "consistent with answering yes to both" of the questions.
Asian markets were broadly lower with the negativity spilling into Europe. The stronger U.S. dollar, which was bolstered by Yellen's comments, also weighed on oil prices.
In European individual stock news, shares of France's Alstom closed around 2.9 percent higher after the engineering firm was awarded a $2 billion contract by Amtrak to supply new high-speed trains.