Jim Grant's warning about the Fed

The market reacted positively to Federal Reserve Chair Janet Yellen's congressional testimony Tuesday, but Fed critic Jim Grant warned that, in the end, the consequences of the central bank's "distortion" of interest rates "will not be wholesome."

"This virus of radical monetary policy is now coursing through the political bloodstream. There's no going back, at least not for the medium term," the founder and editor of Grant's Interest Rate Observer said in an interview with "Closing Bell."

"The world has come to expect that central banks will be there in force in unprecedented ways with the greatest improvisations come times of trouble."

At some point there will be trouble in the form of a bear market or a recession, he added.

"So what then, what do they do then?"

During her testimony Tuesday, Yellen said no rate hike was expected for the next few Federal Open Market Committee meetings. During the later question-and-answer session, she said the central bank would not raise rates before it found confidence in the economic recovery.

Read MoreYellen takes it slow, and that pleases markets

Grant said Yellen didn't actually say very much.

"It was kind of rhetorical yoga, stretching here and stretching there, seeking to offend no one and seeking to disappoint no one," he said.

Stocks climbed to new highs on Yellen's remarks, with both the Dow Jones industrial average and S&P 500 setting a new intraday record.

Grant noted that the majority of the world stock market cap is under the shadow of zero percent interest rates.

"We can't know exactly what valuations would be in the absence of these most extraordinary interventions but I think we would have a very different financial landscape," he said.

Read MoreFed rate hike expectations slip to October

As for where he'd put his money, Grant is still bullish on Russian securities and also likes gold.

"Gold stocks are there for the taking. They just get cheaper and cheaper all the time," he said. "Gold is the great contra investment in a world of central bank dominance. It is the anti-central bank investment."

However, he's having a hard time finding long plays in the U.S. stock market, instead finding much more interesting ideas in the short side of the market, he said.