Read More Evans favors later Fed hike, sees 2.5% 2016 growth
"We're going to pay a lot of attention because she's going to have more insight than the rest of us. Let's see how confident she is," said David Ader, head of Treasury strategy at CRT Capital. "Maybe there's concern that the Fed is doing this (rate hike) because they're committed to it, and the economy is just not that strong. And I think that may not get people to be aggressive buyers, but it certainly is inhibiting people being aggressive sellers."
Yellen is scheduled to speak at the Economics Club of Washington on Wednesday at 12:25 p.m. ET and then testifies the next morning before the Congressional Joint Economic Committee. On the data front, there is the ADP data at 8:15 a.m. ET, and 192,000 private sector payrolls are expected. That compares with expectations of 200,000 for Friday's November government jobs report.
There are also productivity and costs at 8:30 a.m. and the Fed's beige book on the economy at 2 p.m.
Read More Cyber Monday sales top $3 billion, beat forecast
Treasury yields fell Tuesday, as prices rose, and the 10-year was yielding 2.14 percent late in the day, a level it last hit on Nov. 2. The Fed sensitive two-year also saw its yield fall but it recovered some ground by the end of the day, trading above 0.9 percent.
"I think it's getting a little nervous," said Arthur Bass, managing director with COEX Partners. "We're in uncharted water. I'm looking at this as if we're at the tail end of a grand experiment, where central banks have been flooding the market with money at a multiple of more than they ever did in history for the last seven years."
The weak ISM data follows Monday's surprisingly weak Chicago PMI, which also showed contraction. "They've got themselves in a tight corner, where there's all this commitment to tightening and the numbers aren't coming through," Bass said. "If we get another outlier low number Friday, who knows what will happen."
Read More The junk bond market 'keeps me up at night': Trader
Ader said the 10-year yield made an important technical move, breaking out of a recent range. He said a next level could be 2.04 percent. "It's going to take a weak nonfarm payroll number to get us there," he said. He said a very weak jobs number would be a problem for Fed. "That would challenge whether the Fed is going to hike."
As bond yields came in Tuesday, stocks went flying in what some called an early Santa rally. All 10 S&P sectors were higher, led by health care. The S&P 500 gained 1 percent to 2102.
"It's typical divergence we see every so often. The equity market can rally at the drop of a hat. That's the world we've been in. It's the beginning of the month. There's seasonality, and there are several factors that give you a positive bias for the day, whether it's sustainable is the bigger question," said Michael O'Rourke, chief market strategist at JonesTrading.
Read More This is the key markets play in 2016: Goldman's Kostin
Art Hogan, chief market strategist at Wunderlich Securities, said stocks were helped by some fund inflows which occur at the start of the month. "We have plowed through as much negative news as we've seen all year for stocks. Today was no different. We got a clunker of an ISM number and the (stock) market barely blipped," he said. "Today was inflow day, but the other piece of the puzzle is there's some buzz around this ISM manufacturing in contraction."
He said investors are speculating the ISM contractionis a potential recession harbinger though some economists say the number could move higher again next month. At any rate, softer data could slow the pace of Fed rate hikes, and the markets are focused on the possible speed of Fed rate hikes, now that there is a consensus that the first hike will be in December.
What the markets are looking for from Yellen is some sense of the Fed's pace of tightening, which it has emphasized would be slow.
Read More How S&P 500 sectors perform in December
"We've had a parade of Fed speakers, and it's been very well choreographed and orchestrated around December. Does that narrative remain intact, and do we get any hints around monetary policy in 2016. The initial liftoff doesn't matter to us. It's going to be more about what happens in 2016. Are they going to be data dependent? And how much do they hike?" he said.
J.P. Morgan chief U.S. economist Michael Feroli, said in a note that he expects Yellen to defend the reasoning for a possible rate hike. He said Yellen may refer to the rate hike, as the Fed did in recent minutes, by saying "it may well become appropriate to initiate the normalization process at the next meeting."
"Discussion of the pace should feature prominently in both speeches, and we expect that she will note that the timing of the first rate hike matters much less than the pace of subsequent increases, and the committee expects the pace will be very gradual," he wrote.
Now that November is in the rear view mirror, a look at some of the market performance is constructive for the end of the year. High-yield debt performance has gotten a lot of buzz, and that market had an ugly November performance.
There is a divergence between Citigroup's high-yield index, which saw spreads widening 75 basis points in November to 6.94 percent, while its investment grade corporate bond spread tightened by 4 basis points to 1.54 percent.
"In my view, it has very little to do with rates. It has more to do with risk on/risk off mentality. The performance in October was phenomenal. The market rallied, and everyone was on the same side of the trade," said Stephen Antczak, Citigroup fixed-income strategist. But the tone was the complete opposite in November and now.
Read More How US drillers weathered OPEC's new oil order
"It's energy and basic materials certainly under the most pressure," said Antczak, head of U.S. credit strategy team. Oil prices were also 10 percent lower in November.
The lowest-rated bonds got hit the hardest. "They just got destroyed. They're the most vulnerable. Everybody knows that, but I think that really highlights the 'my gosh, we have to be cautious mentality,' " he said. "Avoid anything with any hint of fundamental challenges. I think it's overdone, but I don't see a catalyst to reverse it in the near term."
He also said that to some extent, the junk market reflects the sentiment among high-yield portfolio managers right now. "It's a fear of risk. If you want to get involved, stay in BB. If you want to get involved, stay in health care. Avoid the speed bumps," he said.
"I think it reflects some of the concerns in the broader market. There's been some choppiness in stocks as we know. There's certainly been some choppiness in EM. So to some extent it reflects the broader market," said Antczak.
Standard and Poor's, meanwhile, reported that its distress ratio on junk debt rose to 20.1 percent, up from 19.1 percent in October. It was last at that level in 2009, a year in which the ratio also rose as high as 70 percent. The oil and gas sector accounts for 113 of 361 issues in the distress ratio currently. The oil and gas sector accounted for 37 percent of total distressed debt and had the second highest distress ratio, at 50.4 percent.
"We use it as a gauge when we're looking at where there is stress in the market, specifically the high-yield market, and to look at where we could see a rise in default rates down the road. Typically a rise in the distress typically indicates an increase in default rates," said Diane Vazza, global head of fixed income research at S&P.
"We already started to see that." She noted the default rate was 2.5 in September and rose to 2.8 percent in November, the highest level since 2012.
S&P expects the default rate to rise to 3.3 percent by Sept. 30, 2016.