There's no question that we have reaped many positive benefits from Google's technology, Ford said.
The company has reinvented the way people use the Internet and it is involved in numerous goodwill efforts to help bring technology to the masses. For example its Loon project is focused on bringing the Internet to unconnected parts of the world via a balloon that allows people to connect to Wi-Fi.
Google has also led the tech industry in some ways when its comes to offering tools and features that give more transparency and privacy.
The company was one of the first tech companies that began encrypting all email communication with a HTTPS connection. It was also the first search giant to introduce ad preference settings, which enable users to edit what kinds of ads Google serves up.
Google, which declined to comment for this story, was also among the first to implement an incognito window feature, which allows users to search the Web without Google saving a user's data.
But for the company's good deeds to outshine its missteps, it needs to be better about communicating its business model to consumers and making them aware that they are being tracked, experts said.
"What Google and any other company can do is try to be as transparent as possible and communicate over and over again what they want to do," Kracher said. "It's a delicate balance between having access to delicate information and people not understanding that."
So far, Google is not doing enough to educate its users, Ford said. But Google is not the only company guilty of this, he added. Many tech companies fear an open dialog about privacy concerns because they are concerned such a stance could open them to legal liability.
"The issue of legality is often the problem. They don't speak as frankly as they could because they are afraid," Ford said. "But the fundamental question that might be brought to Google's feet is that to what degree do they have a responsibility, not a legal one, but a moral responsibility to educate their users about the business model that fuels their service."
—By CNBC's Cadie Thompson. Follow her on Twitter