The 80-year-old Cochran, who is chairman of the Senate's Appropriations Committee, has been ill since at least last year with urological problems.
Mississippi Gov. Phil Bryant, a Republican, will appoint someone to replace Cochran temporarily.
Bryant's pick would maintain the GOP's narrow, 51-seat majority in the 100-seat Senate — at least for now.
But a special election will be held in November to select a permanent replacement for the remainder of Cochran's term.
The state's other U.S. senator, Roger Wicker, is running for re-election this year. Wicker, who is also a Republican, faces a primary challenge from Chris McDaniel.
Mississippi is a solidly Republican state, giving the party a decided edge in holding on to both Senate seats. But the GOP's loss of a Senate seat from Alabama in a special election in December won by Democrat Doug Jones underscores the fact that nothing is certain in politics.