Investor Kirk Kerkorian's Tracinda said it was withdrawing its offer to buy 16 percent of Tesorobecause of the refiner's recent adoption of a "poison pill" shareholder rights plan.
Shares of Tesoro fell more than 6 percent.
Tracinda said in a statement that the plan "inhibits value for all Tesoro shareholders by, among other things, restricting the ability of shareholders to vote, sell or acquire Tesoro shares freely without fear of triggering the draconian provisions of the rights plan."
Tracinda launched an attempt last month to buy up to 21.9 million shares at $64 each, about one-third more than where the stock currently trades. If completed, the tender would have brought Tracinda's total stake in the company to 19.98 percent.
Any shares tendered to Tracinda will be returned, the company said.
Tesoro's board has said it was neutral on whether shareholders should accept Tracinda's tender offer, but also adopted the poison pill, which would make it more costly and complicated for Kerkorian to launch a hostile bid.
The poison pill prevents any entity from acquiring more than 20 percent of Tesoro unless it negotiated a friendly deal with Tesoro's board.
Tracinda had said it was investing in Tesoro because it believed in the fundamentals of the industry and the management of the company. It said it had made clear to Tesoro that it planned to cooperate with management "to identify ways to enhance the value of the company for all shareholders."
Kerkorian built his fortune in Las Vegas hotels, but has more recently made waves in the automotive industry, acquiring and then selling off a large stake in General Motors and bidding for Chrysler.
Tesoro shares had been trading at about $57 before Tracinda made its offer and surged as high as $65.50 after it was announced.
The stock has since fallen off sharply and was trading down $3.16, or 6.1 percent, at $48.53 Tuesday morning.