"It's a huge company and everyone needs to be on the same page. Since (CEO) Kazuo Hirai has come in, he's tried to get the individual units to work together. But Sony is moving way too slow to respond to the competitive challenges," said Tim Bajarin, president, Creative Strategies.
"Ultimately for Sony to emerge they are going to have to be a company that's more aggressive with innovation not only in hardware, but also software and services," he said.
(Read more: Sony rejects Dan Loeb's spin off proposal)
However, despite the company's struggles Bajarin said he's not ready to give up on Sony. "Hirai is very forward thinking - he knows what needs to be done. He has the skill set but he has to work within a bureaucracy," he said.
When responding to a question about Sony's current business structure in the company's post-earnings conference call, Masaru Kato, chief financial officer, noted that the CEO was on top of the situation, according to a transcript posted on Seeking Alpha.
"If it is necessary, Kazou Hirai, our CEO, has always been saying that if he sees signs that the current business model or the action plans that we are taking to reduce losses will not result in achieving our objectives, he is willing to take further actions. Now are we at this point? Maybe we have to see how we do in the third quarter. But whatever we do, I mean, next year, we should be in a position to improve on what we are doing this year," Kato said.
Breaking up the business
Industry watchers say the breakup of Sony's business by splitting its movies and music division off from the rest of the company - as proposed by activist hedge fund Third Point earlier this year - remains a possibility.
In August, Sony rejected the hedge fund's call to spin off its entertainment business, saying it strongly believes that continuing to own 100 percent of the company's entertainment business is fundamental to the company's success.
(Read more: Will the PlayStation 4 Give Sony the Boost It Needs?)
"I think they have to consider what Third Point is saying as a large shareholder. One of their faults is they could adopt some of the ideas they have been given," said Yuuki Sakurai, president of Fukoku Capital Management.
—By CNBC's Ansuya Harjani; Follow her on Twitter: @Ansuya_H