Fed needs to take away the punch bowl: Lindsey

The Federal Reserve is increasing risk to the U.S. economy by putting off an interest rate increase, and it's time to take away the punch bowl, former Fed Board of Governors member Lawrence Lindsey said Monday.

"Risks are necessarily two-sided, and by delaying action, what that'll mean is when they have to move, they're going to have to move much more quickly. And to me, that's a much more destabilizing type of risk than a gradual adjustment upward in rates," the Lindsey Group CEO said on CNBC's "Squawk Box."

Lindsey said current rates are not appropriate in the grand scheme, noting that unemployment is at 5.5 percent, the economy is growing at about 2 percent and inflation is roughly 1.5 percent.

The central bank has held its benchmark interest rate near zero since December 2008. The Fed is widely expected to hike rates later this year or early next year.

"The question is how do we get to a higher Fed funds rate? The question is do we do it slowly and start sooner, or do we wait until we're forced to do it by the bond market or by events or statistics?" he asked.

Read MoreNo need for near-zero Fed rates anymore: Experts

U.S. markets have now entered a third bubble, Lindsey said, referring to the dotcom and housing bubbles of the late '90s and mid-2000s.

"It's a lot of fun while the bubble is going up, and no one wants be accused of ending the party," he said. "But as William McChesney Martin said, it was the job of the Fed—it is the job of the Fed—to take away the punch bowl before the party really gets going."

Some market watchers have said the Fed cannot raise rates now because the dollar has appreciated considerably, making American goods less competitive on the international market. As such, U.S. firms need the continued boost from low interest rates.

However, Lindsey said there is no empirical evidence that the dollar has any short-term effect on U.S. GDP.

"Remember, we're a nation that exports 15 percent GDP and imports 20 percent, so the gains to the consumer from a stronger dollar are generally larger than any loss in competitiveness," he said.

Further, the run-up in the dollar is largely due to monetary policy in Europe, where the European Central Bank is buying billions of dollars in order to spur stimulus. That bond-buying has also weakened the euro.

"If you're a risk minimizer, you've got two choices," Lindsey said. "You can try to get a little bit of extra competitiveness by weakening the dollar, or you can worry about Europe imploding and stopping them from their only exit strategy, which is to weaken the euro."

Read MoreWilbur Ross: Greece will make a deal, but ...

The lowest-risk approach is to allow Europe to continue weakening the euro, he said.

The Federal Reserve has also historically looked through volatile currency and commodity moves when determining long-term monetary policy, he added.