The so-called "surface web," which is what the vast majority of people access and browse on a daily basis, accounts for just 4 percent of the total content stored online. The other 96 percent makes up the Deep Web, with a small percentage of that content making up what some might call the "Dark Web.
The Deep Web, essentially, refers to anything on the internet that is not indexed by a search engine. Websites that cannot be found through traditional searching methods are said to live in the Deep Web. These include anything protected behind a password or paywall, such as:
- Medical and financial records
- Scientific and academic databases
- Legal documents and government reports
- Subscription-only content
The contents of your email account exist in the Deep Web. Banking information, doctor communications, secure image hosting and anything else that is protected from the public eye is considered to be Deep Web content.
However, there are darker things lurking in the Deep Web as well. Websites designed to be accessed only by special anonymous browsers exist, and these hidden sites often become hotbeds of illegal activity. The online black marketplace, Silk Road, is perhaps the best-known example. These sites, occupying what's known as the Dark Web, may include forums or chat rooms as well as information for illegal activities including drug use, child pornography, human trafficking, political protest and more.
We Provide Vetted, Cost-Effective Private Investigators Nationwide
We've uncovered corruption, vetted partners, tracked down witnesses, and searched the dark web for our clients. Find out what we can do for you:
Accessing the Deep Web
There are many reasons why a private investigator may need to complete a deep web search, from uncovering criminal activity to finding information about missing persons and stolen property. When searching for criminals, it is often necessary to go where the criminals will be. Online, this means venturing into the Dark Web. Online research into private databases and other walled-off documents may also be a necessary part of the job. Additionally, researching a person or event may require a broader and deeper search than what a standard search engine can provide;
Because so much of the Deep Web is hidden and inaccessible from a standard web browser, having the right tools to access it is an essential part of any cyber-investigation.
Top Browsers for Accessing the Deep Web
Many users of the Deep Web require their online activity to be anonymous. This is true both for people using the internet for illegal activity and those whose identities could be compromised by their searches, including whistle blowers and private investigators. For this reason, anonymous browsers have been created that, when coupled with a VPN service, allow for nearly untraceable web activity.
The most common and well-known of these browsers is "The Onion Router," better known as Tor. However, there are others, including:
- Subgraph OS
Some Dark Web sites are designed specifically to work only when accessed by these browsers.
How to Search the Deep Web for Information
Downloading an anonymous browser will help you to access the Deep Web, but it won't allow you to find anything. For that, you will need a search engine that is designed specifically to probe into areas of the internet that traditional search engines do not.
Depending on what an investigator is searching for, a variety of Deep Web search engines may be useful:
- Pipl: An invaluable research tool for finding people, this search engine browses through databases, member directories, court records and other listings. Using a name, phone number or email address, you can locate a person and find all of their online profiles and public records.
- The Wayback Machine: An extension of the Internet Archive, this search engine allows users to view snapshots of websites from previous dates, which can be helpful in identifying information that has changed or that a webmaster is attempting to cover up or deny.
- Core.onion: Onion sites are those hosted on the Tor network and can only be accessed by the Tor browser. Core.onion hosts the Tor Directory, which offers a fairly thorough directory of .onion sites. While not a search engine per se, the directory is invaluable for finding hidden sites and getting started on the Tor network.
- Not Evil: Replacing the now-defunct Torch, Not Evil is a user-supported search engine for .onion sites, accessible through the Tor network. From the Tor browser, it can be accessed at http://xycpusearchon2mc.onion/
Although the Deep Web can offer a host of valuable information that may be helpful in research and investigation, it also poses some dangers. Aside from the illegal activities conducted on some of these sites, many sites are fraudulent, fraught with viruses or otherwise risky to visit without the proper knowledge. This is one reason why it is often best to leave this type of research up to professional private investigators who have the tools and experience necessary to research safely and return with accurate information. A good investigator will also have access to and knowledge of databases and other resources a layperson will not be familiar with, making Deep Web searches faster and more effective.
About Danny Boice:
Danny Boice is the co-founder and CEO of Trustify, the first technology platform linking people to experienced investigators on-demand. Proud father of 5 (plus a dog!) he started Trustify with his wife Jen in 2015 after personally dealing with the difficulties of finding a good investigator in a hurry. Danny’s journey as an entrepreneur and journalist includes founding Speek, a conference call service, as well as serving as an adjunct professor at Georgetown University and being featured in the Washington Post, Fast Company, and The Wall Street Journal (to name a few). Danny and Jen believe in the positive power of private investigators and are actively engaged in partnerships with a variety of nonprofits including Becky’s Fund, an organization devoted to helping the victims of domestic violence.
Catch up with Danny Boice's latest writings on Medium.