U.S. crude futures fell from an 18-month intraday peak to settle at a two-week low, as the U.S. dollar rallied to its highest since 2002.
Traders said crude prices were buoyed earlier in the day by hopes that a deal between OPEC and other big oil exporters to cut production, which kicked in on Sunday, will drain a global supply glut.
Earlier in the session, both oil contracts hit their highest levels since July 2015 with Brent reaching $58.37 and U.S. $55.24, before paring gains on the strong U.S. dollar.
"The dollar strength is certainly weighing on oil prices," said Andrew Lipow, president of energy consulting firm Lipow Oil Associates in Houston, noting U.S. stock markets also pared their gains from earlier in the day with the dollar rally.
The dollar hit a 14-year high against a basket of other currencies after data showed U.S. manufacturing activity grew more than expected in November.
A stronger dollar makes greenback-denominated commodities like crude oil more expensive to holders of other currencies.
Traders were also concerned about reports that Libya will further increase production, John Kilduff, founding partner at hedge fund Again Capital, told CNBC.
Libya plans to ship from a port near Tripoli nearly 1.9 million barrels of oil this month from the recently reopened Sharara oil deposit, according to a loading document obtained by Bloomberg.
Libya, one of two OPEC countries exempt from output cuts, has increased its production to 685,000 bpd, from around 600,000 bpd in December, an official at the National Oil Corporation said on Sunday.
Also on the supply front, U.S. oil production increased in October to 8.8 million barrels a day, the highest level since May 2016.
"More than half of this growth can be attributed to Alaska and [Gulf of Mexico], but [Lower 48] onshore production is finally showing signs of life," Barclays said in a note.
"Output in several states, including Texas, Oklahoma, and North Dakota, increased in October and is perhaps on the verge of returning to growth."
Jan. 1 marked the official start of a deal agreed by the Organization of the Petroleum Exporting Countries and other exporters such as Russia to reduce output by almost 1.8 million barrels per day (bpd).
"The most likely scenario is OPEC and non-OPEC member countries will be committed to the deal, especially in early stages," said Ric Spooner, chief market analyst at CMC Markets.
Investors will be watching OPEC very closely to see whether the group's members keep their promises to reduce production.
"If 2016 was the year of words, 2017 must be the year of actions," said Tamas Varga, senior oil analyst at London brokerage PVM Oil Associates.
Non-OPEC Middle Eastern oil producer Oman told customers last week that it would cut its crude oil term allocation volumes by 5 percent in March.
Non-OPEC Russia's oil production in December remained unchanged at 11.21 million bpd, near a 30-year high, but it was preparing to cut output by 300,000 bpd in the first half of 2017 in its contribution to the accord.
— CNBC's Tom DiChristopher and Patti Domm contributed to this story.