In the recording CNN referenced, Burr said that while it "didn't have a bullseye on it," "nothing made [him] feel better" than seeing the former secretary of state on the cover of a rifle magazine in a gun shop.
"Let me assure you, there's an army of support out there right now for our candidates," Burr, a Republican from North Carolina locked in a tight race, said in the recording.
He apologized for the remarks to NBC News.
"The comment I made was inappropriate, and I apologize for it," Burr said in a statement.
Burr narrowly leads his opponent Democrat Deborah Ross by one point in his fight for re-election, according to a RealClearPolitics average of recent polls.
His comments bear some similarities to those made by Donald Trump in Burr's state, who previously suggested that violence could stop Clinton from nominating judges.
"If she gets to pick her judges, nothing you can do folks," Trump told a crowd in Wilmington, North Carolina in August. "Although the Second Amendment people, maybe there is I don't know."
— NBC News contributed to this report.