Although very few investors are buying long-dated Greek sovereign debt at the moment, the yield on the 10-year bond is still seen as an indicator of investor sentiment in the country.
Sarah Pemberton, European economist at Capital Economics, said Greece's government was under pressure to exit its bailout early given the growing popularity of Syriza, the leftwing anti-bailout political party.
Read MoreGreece rejects bailout talk, eyes growth
"But the economic case is very unconvincing that Greece is able to support itself," she told CNBC, highlighting concerns that it would not be able to return to market without increasing its debt financing costs.
"Economic indicators show that the third quarter might be Greece's first quarter of positive growth, but it's in a very vulnerable positon and any negative market reaction could prolong its time in recession," Pemberton added.
Read MoreGreek PM wins confidence vote in parliament
Samaras' government has also been plagued by the prospect of snap elections early next year if the prime minister fails to gain the support of opposition lawmakers for his candidate for president. A promise to exit the painful program early was key in securing that backing.
"We don't have buyers in the Greek bond market because of the political risk, the possibility of snap elections in early 2015," a bond trader at a major Greek bank told Reuters.