Ulyukayev's comments come amid a raft of economic problems sparking concern both inside and outside Russia.
Before the financial crisis hit in 2008, Russia was clocking up economic growth of around 8 percent, but its economy has slowed dramatically, growing by 4.3 percent in 2011 and 3.4 percent in 2012. The International Monetary Fund (IMF) now expects Russia's gross domestic product (GDP) to have expanded by just 1.5 percent in 2013, following a contraction in investment as a series of large energy and public sector projects were completed.
(Read more: VTB Bank CEO warns on Russia GDP growth)
The Russian economy is currently heavily reliant on the oil and gas industries, and Ulyukayev acknowledged the need to diversify away from this.
"We will try to diversify first of all in terms of export subsidies... We are thinking about supporting the exports of mid-sized companies," he said. "Plus, we have to improve the business climate to make it more friendly for different types of businesses."
Given its economic struggles, Russia's hosting of the Winter Olympics in Sochi has proved controversial.
But the head of the country's biggest bank told CNBC that it was the most important thing on the government's agenda - even ahead of the pressing economic concerns.
(Read more: Khodorkovsky isreleased into Russia dominated by Putin)
"The priority for our country is the Olympic Games," German Gref, chief executive of Sberbank, told CNBC at the World Economic Forum in Davos.
The Olympics, due to start in February in Sochi, are proving to be one of the most controversial ever before they even start. There has been renewed focus both on Russia's attitude to homosexuality - current President Vladimir Putin has drawn comparisons between gays and paedophiles - and its problems with terrorism.
The Russian presence in Davos is relatively low-profile compared to previous years, with neither Putin or Prime Minister Dmitry Medvedev attending. Gref lay the blame for this at the feet of the Olympics.
"We have a lot of concerns about the slowing down of economic growth," Gref said. He also called for further structural reforms to move Russia away from its reliance on oil and gas revenues and towards more "innovation" and a stronger private sector.