The so-called "Heartbleed bug" which was revealed by security firm Codenomicon, was discovered in OpenSSL software—an encryption service used by around two-thirds of websites to protect information sent to and from web pages.
Cybercriminals could use the security hole to steal sensitive personal information. Even more worrying is the fact that the hackers could regain access to the information if they have stolen a "master key" code, potentially making a password change ineffective.
OpenSSL has released an update to fix the problem, and companies including Google, Yahoo and Facebook have upgraded the software. However, it is not clear whether other companies have done the same, making the universal protection for users difficult.
Read MoreCan your fridge be hacked in the 'Internet of things'?
If a firm hasn't updated the system for its website with the new fix and a user changes their password, this new password will be as vulnerable as if the update hadn't been carried out.
Security experts say that timing is crucial. Users should only change their passwords when a site has fixed the security flaw.
"Passwords are stored in an encrypted format. The latest bug could give hackers access to the skeleton key to open the central file that has all the passwords in it. So you changing the password doesn't matter because this guy with the key can come in and look at your password anyway," Ernest Hilbert, former FBI agent and head of cyber investigations for Europe, Middle East and Asia at risk consultancy Kroll, told CNBC in a phone interview.