The so-called "Heartbleed bug" was discovered in OpenSSL software—an encryption service used by around two-thirds of websites to protect information sent to and from Web pages.
Cybercriminals could use the security hole to steal sensitive personal information. Even more worrying is the fact that the code behind the encryption means that even if the bug is fixed, hackers could regain access to the information.
"It's very widespread. It will affect everyone in one way or another," Simon Eappariello, a senior vice president at iboss Network Security, told CNBC in a phone interview.
"It can expose the crown jewels of security on the Internet: encryption keys. Once those keys are compromised, once that data has been stolen, it's still vulnerable."
Security firm Codenomicon, which identified the bug and published its details online, said Heartbleed allowed attackers to "eavesdrop on communications." The company discovered the threat by simulating the attack on their own systems.
Codenomicon's researchers published the findings on heartbleed.com and urged websites to set up "honeypots that entrap attackers."
Major technology firms told CNBC they were dealing with the problem. A Facebook representative said it had "added protections" against the flaw, while Yahoo said it was "working to implement" a fix. Microsoft said services including Windows were "not impacted" by the security flaw, but "a few services continue to be reviewed and updated with further protections." Google did not immediately reply to a request for comment.
Costs 'real money'
The Heartbleed bug has experts especially worried because an attack can happen without leaving a trace.
"It is always concerning when you can't do traditional forensics and find out what's been going on," Tim Watson, professor and director of Warwick University's Cyber Security Centre, told CNBC in a phone interview.
He added that to mitigate the risks, companies should have people monitoring their networks—which would be expensive.
"We are talking about an issue in this software which costs the world real money," Watson said.
OpenSSL has released an update to fix the problem, but the cleanup operation could see companies requesting that users change their passwords to a range of services.
However, some experts told The Associated Press that changing the passwords won't work until affected websites install the software released Monday to fix the problem.
"This is going to be difficult for the average guy in the streets to understand, because it's hard to know who has done what and what is safe," Codenomicon CEO David Chartier told the news agency.
—By CNBC's Arun Kharpal.