Although very few investors are buying long-dated Greek sovereign debt at the moment, the yield on the 10-year note is still seen as an indicator of investor sentiment in the country.
Read MoreGreece plans new bond sales, confirms 2015 growth target
Greece is hoping to leave its bailout program early and meet its funding needs through the debt markets, rather than call on the "Troika" organizations running Greece's bailout program– the European Commission, European Central Bank (ECB) and International Monetary Fund (IMF)—for more assistance.
The country was one of the first in Europe to be hit by the global financial crisis of 2008 and the full scope of its problems helped spark the euro zone sovereign debt crisis. Greece needed to be bailed out by the IMF and EU to the tune of 240 billion euro ($304 billion) and was required to impose a tough—and hugely unpopular—austerity program.
It now hopes to leave the program before its scheduled end in 2016 without further assistance, but analysts at Capital Economics warned the recent spike in bond yields could hit these plans.
"(The) market reaction itself has made it much less likely that Greece will be able to exit its bail-out early. The current 7 percent yield on 10-year government bond yields is well above the rates it pays on its IMF loans and the rates faced by Ireland and Portugal when they exited their bail-outs," Capital Economics' Sarah Pemberton said in a note.
In more bad news for Greece, ratings agency Fitch warned on the country's banks on Wednesday, ahead of the results of euro zone-wide stress test, due October 26.
According to Fitch Ratings, Greek banks remain burdened by "large problem loan portfolios, despite boosting capital and progress with their integration and restructuring plans".
"This means that EU-wide stress tests could reveal additional capital shortfalls, especially if on-going restructuring measures are not fully incorporated," Josep Colomer, director at Fitch Ratings, said in a release.
He identified the four large Greek banks - National Bank of Greece (NBG), Piraeus, Eurobank and Alpha - as being vulnerable to a "improving but still very weak" economy.
Euro zone-wide fears
The agitation in Greek markets come at a time of growing concern about the strength of the euro zone's recovery. Last week, fears of a slowdown in Europe hit investor confidence across the world, resulting in equity market volatility in the U.S., Europe and Asia.
Gross domestic product (GDP) stalled in the second quarter of the year—coming in flat, below expectations—and inflation continued to fall in September, hitting a worryingly low 0.3 percent. Investors are anxious that this slide will continue, with the latest inflation figures due on Thursday.
There are also worries about the outlook for Germany, once dubbed the "strongman" of Europe, as economic data from the country has surprised on the downside over the month. Meanwhile on Friday, ratings agency Standard & Poor's (S&P) cut its outlook for France to negative from stable.
"Rising bond yields are merely a reflection of the troubles that have resurfaced in the currency bloc. We tend to think that the economy is the biggest threat, and if growth does not pick up sometime soon then we could see rising bond yields spread beyond Greece into Portugal, Spain and even Italy," Brook added.
Earlier on Wednesday, German 10-year bunds hit a record low of 0.824 percent, highlighting the divergence between Europe's largest economy and peripheral neighbors.