The reference to "irrational exuberance" — the two words Greenspan is most famous for — hearkens back to remarks he delivered at a 1996 American Enterprise Institute dinner.
"What I was trying to say in the AEI speech … is you never can be quite sure when 'irrational exuberance' arises. I was doing it as part of a much broader speech, and talking about the analysis of markets," Greenspan said on "Squawk Box," reflecting on the investment environment of more than two decades ago.
"I wasn't trying to focus short term. But the press loved that term," he added. "I am sort of now questioning whether it was wise to put it in the speech."
Greenspan said he's trying to ask similar questions in the current environment about when the three-decade bull market in bonds might end, considering "there's only one way" historically low interest rates can go and that's higher.
The Fed has signaled a willingness to increase rates further if the economy continues to cooperate. Central bankers have hiked the cost of borrowing money four times since December 2015, including twice this year alone.
In September 2007, as the financial crisis was heating up, the Fed began incrementally cutting rates and continued until December 2008, when the central bank's benchmark rate bottomed out at a range of zero to 0.25 percent where it stayed for seven years.
In December 2008, hoping to boost the economy, Fed officials also embarked on a series of bond-buying programs, which eventually swelled the central bank's balance sheet to the current $4.5 trillion. Only now is the Fed talking about how to wind down its portfolio of assets.
In making his forecast on bonds, Greenspan said Friday he's learned from the past. "It's a disturbing process because you have to be terribly careful with your words," he added, stressing he's not putting any time frame on it.
The 91-year-old Greenspan's tenure as Fed chairman spanned four presidents and nearly 19 years — starting with his appointment in 1987 by Ronald Reagan and ending with his retirement in early 2006, after President George W. Bush tapped Ben Bernanke to lead the central bank. Bernanke was succeeded by current Fed Chair Janet Yellen in 2014.