Writers - Can't Find What You Want With Google? Try The Deep Web


You've got a brilliant idea for an article, but your research is pulling up only scant information. What do you do? Dive into the Deep Web!


The Deep Web, or Silent Web as it's sometimes called, is a hidden resource that every writer should know about. Whether you want information for an article, for an historical novel or even for a play or film script, the Deep Web may find it for you when Google can't.

There is considered to be between 350 and 500 million web pages that can be indexed by the likes of Google, MSN, Yahoo and others. The Deep Web is actually not included in these and is reckoned to be many times greater in size!

The visible web is indexed by 'spiders', small robot programs that wander about from site to site gathering details on links and pages to send back to base from which the search engine's index is compiled.

But some web sites can't be accessed by spiders. The spider can call at the door, index the address, but can't get in to record the contents. The reasons for this are varied, and include the fact that website owners can deliberately ban spiders from entering their sites if they wish. Another is that if the site comprises of databases, these are anathema to spiders which can't do anything with them. And it is sites like these, made up of databases full of fascinating information on an bewildering variety of subjects, that mainly comprise the Deep Web.

There are thousands, maybe even millions, of such databases out there on the web, but you need to know where they are and what their address is in order to get into them.

The quality of information on the Deep Web is reckoned to be very high compared to the rest of the web, partly because so many universities and other technical and scientific facilities contribute.

Some databases, such as SJLibrary.org (a collaboration between San Jose State University and San Jose Public Libraries) restrict access to part or all of their content to their students and faculties, but there are plenty of others that do not.

So, it's time to dip your toes into this vast ocean and see what you can find. To start you off here are some sites to try:

Direct Search is a good one (http://www.freepint.com/gary/direct.htm)

Intute - I found this an excellent site http://www.intute.ac.uk/

Complete Planet http://aip.completeplanet.com/

InfoMine http://infomine.ucr.edu/

FindArticles is a database of - you guessed it - articles http://findarticles.com/

For more information on the subject of Deep Web go to deepweb.com http://www.deepweb.com/




Mervyn Love writes on several topics including creative writing. His website http://www.WritersReign.co.uk has a mind-boggling array of resources, articles and links to keep any writer happy for hours. Subscribe to the WritersReign Article Writing course here: http://www.writersreign.co.uk/WRac.html