The Federal Reserve should taper its massive bond-buying program at this point, but that shouldn't be as big as deal a investors are making it out to be, Goldman Sachs chief Lloyd Blankfein told CNBC on Wednesday.
If the Fed scales back its $85 billion monthly asset purchases by $10 billion, the market will be "very, very happy," but anything more it will be "very sad," Blankfein said on "Squawk Box" from Chicago, where the investment bank just held a CFO conference.
The Fed taper would not be the first factor to start removing "abnormal" conditions on the market, he argued—considering the recent increase in bond yields. "The fact that interest rates were 1.5 percent was an artificial support for the market. And the fact that it's moved 150 or 140 basis points higher represents some withdrawal already."
The decrease in the federal deficit because of the sequester spending cuts and increased taxes has been another significant development in the "normalization" of the financial markets, he added.
(Read more: Here's what the market expects from the Fed)
When Blankfein appeared on CNBC in late June, he said the pullback in stocks at that time on the Fed taper talk was an overreaction. Since the June 24 bottom, the has risen about 7 percent and the Dow Jones Industrial Average has gained nearly 5 percent.
In recent days, stocks have also been largely positive leading up to the September Fed meeting, with the S&P higher in 10 of the past 11 sessions.
When the Fed concludes its two-day meeting Wednesday, the policy statement at 2 p.m. ET is widely expected to contain the declaration the central bank has begun to taper its bond purchases. Policymakers will also issue their latest economic projections, and Fed Chairman Ben Bernanke will hold a 2:30 p.m. ET news conference.
Ahead of all that, Blankfein said the Fed's forward guidance on the economy will be an important measure to watch. "Forward guidance coming from an institution with a current vice chairperson [Janet Yellen] who's going to be the next chairperson has more credibility."
Now that former Treasury Secretary Larry Summers has withdrawn his name from consideration, Yellen is seen as the frontrunner to succeed Bernanke, who is expected to step down at the end of this second term in January.
As for his take on the economy, Blankfein said the recovery trajectory "is not as steep as it should be," but he's still generally bullish.
Last week, Peter Oppenheimer, chief global equity strategist at Goldman, said equities have now entered a "growth" phase—the longer but more moderate stage of a bull market, when equities are driven more by earnings rather than valuations.
(Read more: Goldman Sachs: Reasons to stay bullish on stocks)
Back on March 21, 2012, Goldman said the prospects for making money in equities relative to bonds were as good as they had been for a generation. Since then, the Dow has gained about 17 percent and the S&P increased more than 20 percent.
On the recent addition of Goldman Sachs to the Dow industrials, Blankfein said he did not know in advance of the announcement, and learned about it when it "came over the tape." He added that inclusion in the blue-chip index was a nice validator for the company.
(Read more: Here's what you need to know about the Dow changes)
In the biggest shake-up of the Dow in nearly a decade, Goldman, Visa, and Nike will join the 30-stock index as of the start of trading Monday.
—By CNBC's Matthew J. Belvedere. Follow him on Twitter @Matt_SquawkCNBC.