Greek finance minister: 'End the vicious cycle'

Greek FM: Hope to find common ground

Greece's newly elected government will look to "end the vicious cycle" of bailout and borrowing that has persisted through years of financial crisis, Finance Minister Yanis Varoufakis told CNBC on Tuesday.

Varoufakis is a member of the Cabinet of Alexis Tsipras, who was elected prime minister on Sunday. Tsipras leads the leftist Syriza party, which has formed a coalition with the right-wing Independent Greeks party.

Read MoreECB now faces another battle: Dealing with Greece

The new government has made renegotiating Greek debt to the European Central Bank a priority. It wants European leaders, the European Central Bank and the International Monetary Fund to "table [its] comprehensive proposal for ending this never-ending Greek crisis," Varoufakis said in an interview on CNBC's "Fast Money: Halftime Report."

Tsipras' party has promised to repeal austerity policy and seek to shave off some of Greece's debt. The country has imposed stiff austerity measures in the years following a 240 billion euro ($271 billion) bailout package from the "troika" of the European Commission, ECB and IMF.

Read MoreCan Europe resist Greek demands for a debt haircut?

Varoufakis stressed "finding common ground for Europeans." He argued that Greece has been put in a tough situation where it is being asked to borrow money to pay back debts for which it already borrowed.

"You know that I can't really repay you the money I already borrowed and now you're asking me to borrow more," Varoufakis said.