While there is plenty of speculation about the balance sheet based on various comments, there has been no definitive plan announced by the Fed, and markets expect more clarity once the central bank begins to raise short-term interest rates.
Fed officials, going back to Chairman Ben Bernanke, have said it was not the purchases in QE programs that held down rates, but the fact that the Fed kept the securities on its balance sheet.
Read MoreWall Street predicting Greek tragedy: CNBC Fed survey
"I think the focus is on the first rate hike. They've put out guidance in terms of continuing reinvestments until probably a few quarters after the first rate hike," said John Bellows, Western Asset Management portfolio manager. "I think they've tabled the balance sheet for the moment and we're going to hear more about the balance sheet sometime next year, after the first rate hike."
Bellows said it's not clear whether the Fed will continue purchasing Treasurys, mortgages or a combination, but it may ultimately use the balance sheet to tighten.
"When they tighten, they have two tools to tighten with. One tool is rate hikes, and the other tool is contracting the balance sheet. To some extent, they can trade off those tools. While they might use the rate hike at first, at some point in the next year they could switch and use the balance sheet," he said.
Goncalves said the market is eager for clarity sooner.
"The problem with this is such a large component to the supply and demand of Treasurys. It could create unnecessary and undue volatility if they don't clarify their intentions and wait to the last minute," said Goncavles. The "last minute" would be when $200 billion begins to mature next year.
"$200 billion that's large enough to impact the market. They're going to have say whether they are reinvensting. Are they going cold turkey or are they going to taper?" said Goncalves.
He said if the Fed did nothing, the longer-term duration Treasurys that are set to mature could affect lending rates for corporations and consumers. "Higher yields on the long end, we've already seen play a bigger role in investment and housing activity than the move from zero to slightly higher in short rates."
Goncalves said some Fed officials favor slowly moving to shrink the balance sheet, while others would be more aggressive.
"The balance sheet comes as a natural result of QE, and since markets have been obsessed with QE, why would you want to see QE unwind?. If QE is what got markets and the economy strong enough then can we afford a shrinking of the monetary base? Can we allow the balance sheet to shrink. You're basically unwinding QE," he said.
Goncalves said Dudley's speech led some investors to believe the Fed was discussing the balance sheet and the reinvestment program.
"I think the way the Fed has been discussing the adjustment in financial conditions—which is code for long-term rates, stock market and credit spreads—if one of the three is going the wrong direction, the Fed gets nervous. The fact we had a tightening in long-term rates for the first time in a long time means they've been discussing it," he said.