Despite a small advance on Friday, stocks ended up having their worst week since June as investors moved on from Tuesday's election and worried about the impending US "Fiscal Cliff."
For the week, the Dow fell 2.1 percent, while the S&P 500 lost 2.4 percent and the Nasdaq dropped 2.6 percent.
Since President Barack Obama won re-election on Tuesday, the market is down well over three percent.
On Friday, stocks lost most of their earlier gains after Obama called for higher taxes on the rich.
In a speech Friday, Obama said he was "open to new ideas" on resolving the fiscal cliff but repeated his instance that the rich pay more to cut the federal deficit.
"We can't just cut our way to prosperity," he said. "If we're serious about reducing the deficit we have to combine spending cuts with revenue, and that means asking the wealthiest Americans to pay a little more in taxes."
(Read More: Obama 'Open to New Ideas' But Rich Must Pay More)
The Dow quickly shaved 30 points off earlier gains, while the Nasdaq and Standard & Poor's 500 held modest gains.
"For now, (traders are) concerned with how much is just posturing before the negotiations and how mush is real muscle, with 'I'm not crossing this line,'" Art Cashin, director of floor operations at UBS, said on CNBC. "Luckily nobody set a line in the sand. Otherwise, we'd be down 200 points." (Read More: Markets Still Waiting for Breakthrough on Fiscal Cliff)
Earlier, bank shares helped turn the mood more positive, rising after regulators issued a joint statement saying the Basel III accord, which required banks to raise more capital, would be delayed.
U.S. financial institutions had complained that the Jan. 1 deadline for higher capital and liquidity requirements was too soon. The delay, announced jointly by three regulatory panels, did not specify a future implementation date.
Financials on the S&P 500 rallied on the news rallied and the KBW Bank Index gained 0.8 percent. Bank of America shares jumped 2 percent.
Traders saw the move as at least a modest olive branch from the administration to banks.
"The timing was pretty fascinating. A couple days after he was elected they decide to push out Basel, which a lot of Wall Street as well as community bankers were balking at," Lutz said. "That looks like a concilitaroy move in the right direction."
Internet, semiconductor and biotech shares also helped move the market higher.
Most Dow 30 components were positive, with Cisco and Microsoft helping lead an effort to push technology shares broadly positive.
Disney shares plunged on a disappointing after-hours earnings report Thursday, while McDonald's also continued to suffer from its own weak sales.
Across other markets, oil and lumber futures both rose sharply. Shorter-duration bond prices also gained despite the stock market rally, with the 10-year Treasury note yielding 1.64 percent, up marginally from Thursday's close. Equities also rose despite a stronger dollar, which rallied to push the euro down to 1.27.
Obama will have plenty to tackle besides dealing with the tax increases and spending cuts that going over the cliff will trigger, as the market's focus now broadens beyond the political uncertainty that the bruising presidential campaign brought.
"The oxygen that this race gobbled up is now back to feeding a host of other concerns, from the U.S. fiscal cliff to Eurozone worries to seemingly countless other issues," Nicholas Colas, chief market strategist at ConvergEx, said in his morning note. " Whether this round of grumbling is just a passing fancy or something more lasting is the only question that matters to most traders just now." (Read More: Why US Economy May Be Headed for Another Recession)
Traders shrugged off data showing that import prices rose 0.5 percent, significantly higher than the expectations for no increase, and were up primarily due to a 1.3 percent jump in fuel prices.
In an otherwise quiet day for earnings season, J.C. Penney shares plunged after the mid-market retailer reported earnings and sales that fell well short of Wall Street expectations.
The company said it lose 93 cents a share for the third quarter, against expectations of a 7-cent loss, while sales generated $2.93 billion, against estimates of $3.27 billion. Shares slumped 6 percent in premarket trading after dropping more than 8 percent for the week.
JCPenney is the "only retailer I know where: (1) online sales are not growing; and (2) online sales are far removed from brick and mortar comps," said Brian Sozzi, chief equities analyst at NBG Productions in New York. "Either way, the data says: JC Penney is not a top destination and is nowhere near becoming a top destination in peak seasonal shopping periods."
Technology shares were poised for the best performance, with Nasdaq futures indicating a flat open for the tech-heavy index.
After falling into bear market territory this week Apple shares rose nearly 2 percent.
Also in tech, Kayak Software surged on news it would be acquired by Priceline.com in a deal valued at $1.8 billion.
Groupon shares continued their downward spiral, the victim of poor earnings and a weak model that has driven shares 84 percent lower over the past year.
The Dow briefly added to a weak opening after the University of Michigan reported that consumer confidence had increased to its highest level in more than five years. (Read More: Consumer Sentiment Hits Five-Year High) The average then turned higher, however.
European shares closed narrowly higher as doubts persisted over Greece's new bailout deal. Worries resurfaced on Thursday after German Finance Minister Wolfgang Schaeuble said next week may still be too early to make a decision on granting further aid to Athens.
Asian shares losed ower despite positive data out of China. China's annual consumer inflation eased to 1.7 percent in October from September's 1.9 percent, leaving policymakers with scope to ease monetary policy further to shore up growth.