There is only a small risk that the U.K. will leave the European Union and the country remains committed to reforming and improving its relationship within the continent, the U.K. Business Secretary Vince Cable said on Friday.
"If I were back in business and trying to assign risk, I would probably say...we're talking about five percent risk," Cable told CNBC. He said that there were much bigger risks out there for the EU.
"I would hope the rest of Europe doesn't plan on the assumption Britain will leave," he continued. "There is a high probability we won't. And indeed, we want to be actively engaged in reforming the single currency...We want to do something to improve the regulation in the European Union which is too heavy. We want to work within the European Union to get stronger trade agreements, not just through the WTO, but with the United States, India and China and other countries."
(Read more: Britain's PM Cameron wins symbolic vote on EU referendum)
The U.K.'s Conservative Prime Minister David Cameron, announced earlier this year that the country will hold a referendum in 2017 on whether to stay as member of the EU if his party wins the next general election in 2015. Cable's party, the Liberal Democrats, fought against their Conservative coalition partners over the issue of the vote.
Cable's remarks come after Goldman Sachs warned that it would move most of its European business out of London if the U.K. were to leave the EU. Michael Sherwood, co-chief executive of Goldman Sachs International, said: "In all likelihood we would transfer a substantial part of our European business from London to a euro zone location – the most obvious contenders being Paris and Frankfurt."
In a recent survey by the British Chambers of Commerce (BCC), 61.4 percent believe remaining within the EU was the best outcome.
(Read more: UK economy goes from 'zero to hero' but don't expect give away)
Cable told CNBC that the euro zone's stability was important for the U.K. given its trade links, and that he hoped Britain would push through more liberal and business-friendly regulation in the EU.
He also said he did not expect the euro zone situation to worsen. "The position is certainly a lot better than it was a year ago thanks I think primarily to the European Central Bank, which has made the right interventions," he said.
He added: "The moves have been the right ones, but I think everyone is worried about continued stagnation in some of the southern European countries; the effect that has on their banking system. There is going to have to be a stimulus of demand to get those countries out of the current stagnation they are in."
(Read more: UK's economic growth picks up in third quarter)
The Business Secretary also discussed the U.K. housing market, the source of much debate amongst economists. Some have worried in recent months that a government program designed to kick start the housing market is fuelling dangerous price rises. Help to Buy assists buyers with a 5 percent deposit on properties worth up to £600,000. The government provides the buyer with a 20 percent loan.
With housing indicators turning increasingly positive, concerns have been raised that the U.K. housing market could be forming a bubble. High-profile critics of the program have included both The International Monetary Fund and former Bank of England Governor Mervyn King.
Cable told CNBC, "At the moment, there is not an enormous problem but we need to be very, very careful, because we're already at very, very high levels of price in relation to earnings: there is a problem of affordability for middle income families."
(Read more: Goldman: These are UK's two long-term weaknesses)