How business can shed light on the 'dark net'
Law enforcement agencies around the world have been calling on the tech industry to help eradicate the so-called "dark net" – the hidden, unregulated online marketplace for counterfeit goods, drugs, hitmen and child abuse.
Here is a look at why this vast hidden network is of such concern and how the fight to shut it down is going.
Sinister Marketplace
The "dark net" – which is also called the "dark web" -- is found within the "deep web", a vast online space of databases and websites that are not indexed by traditional search engines like Google or Yahoo whose content makes up what the tech industry calls the "surface web."
Experts often use an iceberg analogy: While the "surface web" that most of us use contains around a billion individual documents is the tip of the iceberg, the "deep web" is the immense submerged part that we cannot see – approximately 550 billion documents and web pages.
Much of this un-indexed content in the "deep web" is off-limits and can only be accessed via specific networks that claim to provide untraceable access to certain web content -- and anonymity for the websites within those networks.
The most famous of these networks is Tor, which stands for "The Onion Router." Tor claims to obfuscate the devices and locations of users by routing their requests through different servers across the world multiple times (hence the onion layer analogy). Other networks include Freenet and I2P ("Invisible Internet Project").
Anyone entering the "dark net" of anonymous websites through an anonymization network such as Tor could encounter openly a sinister marketplace of illegal activity. Those accessing the dark net can find Illegal drugs, counterfeit goods, websites purportedly offering hitmen for hire, the open trade of hacking tools and what one cybercrime expert described as "horrific child abuse and live streaming of rape."
While Tor's ability to offer anonymously hosted websites is not illegal, such veiled networks have allowed criminal organisations to flourish.
"Unfortunately, the anonymity these services [like Tor] create has been exploited by criminals of all kinds," Troels Oerting, head of Europol's (the European-wide law enforcement agency) Cybercrime Centre, told CNBC. "Organized criminals dealing in stolen goods, weapons, drugs and intellectual property rights have (mis)used the dark net to provide 'bullet proof' delivery of all kind of illicit commodity."
Apart from terrorists "enjoying the possibility of anonymity to communicate with cells around the world" and "contract killers" offer their services on the dark net, Oerting added that illegal activities also included the "crime as a service" sector – such as the open selling of hacking tools such as malware.
Cyber fraudsters have expanded their efforts from individuals to the wider corporate world and the mass "harvesting" of private data and it has become common for them to deploy malicious software –"malware" specifically designed to penetrate computers networks en masse.
All this poses an immense challenge for law enforcement agencies in their fight against cybercrime and they have approached cyber technology experts, software developers and internet security firms for advice and security software.
'A Game of Cat and Mouse'
Malware has given cyber-thieves an opportunity to infect and access the data on millions of computers, but it has also given rise to information security companies that provide "threat intelligence solutions" to companies, scanning the dark web's content and alerting companies to their online vulnerabilities or if their data is found circulating where it shouldn't be.
One such U.K.-based firm, Trustwave, hit the headlines in December when it's crack team of "ethical hackers" discovered two million stolen Facebook, Google, LinkedIn and Yahoo passwords that were most likely collected by a cybercriminal gang using malware.
(Read more: Car hacking: The next global cybercrime?)
Managing to highlight the threat of cybercrime to the public -- while advertising Trustwave's services at the same time -- the company's experts believe the data was harvested by a "botnet" – a tool used by criminals to distribute malware to infect a large network of computers that they can then access and control. Such malware is openly traded in the "dark net."
Leading cyber security expert Raj Samani, chief technology officer at computer security software firm McAfee, said that a "public-private partnership" had developed between software developers providing solutions against cybercrime and law enforcement agencies trying to tackle the "dark web."
For his part, Samani is an advisor to Europol's Cybercrime Centre within its "internet security" advisory group alongside global experts from Cisco, Verizon, Microsoft and Infineon. The experts alert agencies of technological and software advances – both by developers and by cybercriminals -- that could help or hinder the fight against cybercrime.
Samani added that with the sale and accessibility of child abuse material online, "there has been a rise in the number of individuals storing such content on corporate resources, resulting in the adoption of technologies from companies like Netclean."
Netclean says it integrates "social responsibility and advanced technology" to stop the spread of child sexual abuse material and its technology is used by multinational companies, government agencies, Internet service providers and law enforcement professionals.
(Read more: Copycat Silk Road drug site reopens after FBI raid)
Despite cybercrime-fighting software becoming a booming business in its own right which helps the private and public sector to tackle illicit networks and criminality, a large part of why law enforcement agencies around the world are perturbed by the "dark net" is its constantly innovating character.
McAfee's Samani likened attempts to catch criminals on the "dark web" to a game of cat and mouse. "Some dark web sites can be infiltrated and people do make mistakes but it's a cat-and-mouse game of criminals evolving their techniques to evade detection and society constantly having to adapt. There is this cycle of constant innovation," he said. "Now you have traditional criminals using technology to make crime more efficient."
Abuse of Privacy
For their part, governments are certainly aware of the problem but have been criticized for doing too little too late to tackle online illegality.
The U.K.'s Prime Minister David Cameron heralded a deal with the likes of Google and Yahoo in which the search engines agreed to outlaw thousands of search terms relating to child abuse. While a step in the right direction, analysts told CNBC that it was unlikely to tackle the illicit content within the "dark net", telling CNBC that most internet users searching such content would not use overt search engines anyway - they would go onto the "dark net."
(Read more: The threat from cybercrime? 'You ain't seen nothing yet')
While "doing nothing" is not an option for governments or law enforcement agencies either, there is also widespread reluctance to allow agencies a "catch-all" approach to internet surveillance. "All of us want privacy on the net but there is also the need of law enforcement agencies to know what's going on," Samani noted. "It's a balance between giving agencies access to data and privacy."
In fact, some cyber crime experts like Professor Tim Watson, Director of the Cyber Security Centre at the De Montfort university in the U.K, told CNBC that "not everything on the 'dark web' is bad and law enforcement agencies should concentrate on certain websites more than others…these agencies can pick up traffic patterns and can then target their efforts."
Watson said the public should be as concerned about the "dark web's" illicit content as much as they are about potential abuses of privacy by governments, a view echoed by the scientific director of the Brussels-based European Privacy Association (EPA).
"Governments can go in two directions now. One direction is to use the budget for security on increased surveillance. That would be the easiest way [for them to go] but it would be the wrong way because it would be based on a presumption of unlawful conduct," Paolo Balboni told CNBC.
Cybercrime experts were also keen to stress that networks like Tor – which was itself created by the U.S. Navy to protect government communications – have many positive benefits such as protecting internet users from identity theft or the communication of political dissidents or journalists in repressive regimes.
For its part, Tor directed CNBC to its online statement on illegal use and "abuse" of its technology. "We take abuse seriously. Activists and law enforcement use Tor to investigate abuse and help support survivors. We work with them to help them understand how Tor can help their work."
It also insisted: "Criminals would still have access to botnets, stolen phones, hacked hosting accounts, the postal system, couriers, corrupt officials, and whatever technology emerges to trade content" whether Tor existed or not.
- By CNBC's Holly Ellyatt, follow her on Twitter @HollyEllyatt. Follow us on Twitter: @CNBCWorld