How to Search the Deep Web (the Invisible Stuff That Google Doesn't See)

Member

By David Sarokin
User-Submitted Article

Article Rating:

(10 Ratings)

Find It On the Invisible Web

When you need to know the capital of Albania, or find the latest winner on American Idol, a good old Google search is going to get you that information in a jiffy. But it may surprise you to know that the vast majority of information on the internet isn't indexed by Google... or any other search engine for that matter. Information in dynamic databases, or behind most firewalls and subscription log-ins, or simply turned-off, as far as search spiders go, is all invisible to the Googlebot as it roams the web, looking for new content to index and make available. This is the so-called Deep Web, aka the Invisible Web (or the sinister-sounding Darknet, as it's sometimes referred to). How to search the invisible web? Here's what you need to know.

Difficulty: Moderately Challenging

Instructions

Things You'll Need:

A need to know more than Google can tell you

Step 1

**Use Google Deep Web Tools**

Google itself has created some specialty search tools that peer and poke into some of the hard-to-find recesses of the internet.

Google Scholar at scholar.google.com will unearth tons of information that won't show up in a regular Google search. This is largely academic stuff, but if that's what you're looking for, then Google Scholar is the place to turn.

Google Books (books.google.com) is another terrific way to get at information that would be hard to find in a normal search. As the name suggests, the search peers through the content of millions of published books.

There are limits to Google Scholar and Google Books. While they may find relevant content, you often can only see small snippets of the actual results. Getting full access means buying the book or article in full, or starting a subscription with a commercial service for journal articles.
Step 2

**Search Centuries of News**

Google also allows specialty searching on news stories going back several centuries (news.google.com). Be sure to click on the "All Dates" link to pull up older newspaper archives.

The Resources section includes links to other great newspaper archives sites, many of them free, and all of them part of the Deep Web that doesn't readily show up in ordinary searches.
Step 3

**Deep Search Business**

One of the best Invisible Web resources available for business and financial information is the full-text search at EDGAR (just Google the word Edgar to get right to it). The full-text search function will zip through millions of company filings at the Securities and Exchange Commission (SEC), taking you right to the particular annual report or other document with the information you're looking for.
Step 4

**Know Your Specialty Sources**

There are some new search functions coming into being that claim to make inroads into finding materials that Google might overlook. DeepDyve and Wolfram Alpha are two search tools to keep an eye on, though neither of them have a great deal of content to offer thus far.

For other specialty searches, such as finding people, researching family history, or due diligence research on businesses, try the links in the Resources section.