Week 04: Pushers and Pullers on the Web

Tutorial Exercises

• Pick a search engine and find out how it indexes, how it ranks, what searching options it provides and what search results. Discuss its pros and cons. What type of searches would you use it for? What type of searches would you not use it for?
  
• Pick a subject gateway and find out all you can about how it selects, documents and organises access to resources. Discuss its pros and cons. What type of search would you use it for? What type of searches would you not use it for?
  
• Try and use your search engine to locate your subject gateway, without using its name or other easy identifier - i.e. assume you don't know it exists and are just looking generally in that subject area. Did you find it? Where was it ranked?
  
• Here is an analysis of the state of play with search engines in July 2002 taken from data in Search Engine Watch, http://webraft.its.unimelb.edu.au/136045/prot/gifs/searchengines2002.gif. How has it changed? Are there any new players?
  

Lab Exercise

• In this lab exercise we will look at the resource Technology in Australia 1788-1988 and compare methods for accessing information. The exercises are also designed to highlight issues associated with the conversion of a book to the web. You should try each exercise with each of the different access tools, i.e.
  
  
  
  • Table of contents
    
  • Index
    
  • ht://Dig Search - An open source site search engine first developed at San Diego State University in 1995.
  
  
• The 'best' technique for each exercise should be noted along with any comments and problems, e.g. usability of the interfaces, ease of understanding the results, navigation, coverage etc.
  
  
  
  1. In what year was the 'ute' first produced?
     
  2. Who was the chemist responsible for the development of Vegemite? What role did he play in the Royal Australian Chemical Institute?
     
  3. I'm interested in cotton technology - What's the best access point?
     
  4. I'm interested in the development of dairy technology in Australia - What's the best starting point?
     
  5. Who is G.W. Hills?
     
  6. What roles have women played in the technological development of Australia?
     
  7. Find an image of CSIRAC.
     
  8. What role did Traeger play in the development of the Royal Flying Doctor Service?
     
  9. What New Zealander played what role in the Snowy Mountains Hydro Scheme?
     
  
  
• What conclusions can you draw about the value of traditional methods versus search engines? Comment on the usability of the resource? What does a user need to know to use it effectively? Is that information available? How would you make it available and guide the user to it?

Review Questions

1. Define each of the following:-Search engine, Crawler based search engine, Web directory, Web portal, Meta search engine, Subject gateway, Invisible/Hidden/Deep web, Static pages, Dynamic pages, Ranking or Relevance algorithm, Dublin Core Metadata, Subject Gateway
   
2. Name and describe the basic components of a crawler-based search engine.
   
3. According to the author of www.lookoff.com, what are the possible downsides of using meta-search engines? Does that match with your experience?
   
4. Describe three methods of ranking search results.
   
5. How should search engines be evaluated?
   
6. What are the advantages/disadvantages of automated indexing techniques?
   
7. What are the advantages/disadvantages of human powered indexing techniques?
   
8. What are the key differences between a crawler based search engine and a Web directory? What impact does it have on searching strategies?
   
9. Name four steps you could take to improve your web searching skills?
   Move out of your comfort zone and try a new search engine. How does it compare with your preferred search engine? Will you convert?
   
10. What are the difficulties in accessing and indexing the invisible, deep or hidden Web? What does this mean for searching and discovery of information on the Web?
    
11. Compare this page with the version in Google's cache. What can you deduce about when the site was last visited by the Google web crawler?
    
12. Last year, Helen loaded two resources onto our web site before heading off on four weeks holiday. When she returned one had been indexed by Google and the other had not. Can you think of reasons why this may have occured?

Articles

• Bradley, Phil, 'The relevance of underpants to searching the Web', Ariadne, no. 24, UKOLN, 21 June 2000, http://www.ariadne.ac.uk/issue24/search-engines/.
• Grossan, Bruce, 'Search Engines What they Are, How They Work, and Practical Suggestions for Getting the Most Out of Them', Web reference.com, internet.com Corp., 28 August 1996, http://www.webreference.com/content/search/index.html.
• Medeiros, Norm, 'XML and the Resource Description Framework: The Great Web Hope', Online, September 2000, Online Inc, 2000, http://www.onlinemag.net/OL2000/medeiros9.html.
• Notess, Greg R., 'On the Net: Dead Search Engines', Online, vol. 26, no. 3, Information Today, Inc., May/June 2002, http://www.onlinemag.net/may02/OnTheNet.htm.
• Smith, Alastair G., 'Testing the Surf: Criteria for Evaluating Internet Information Resources', The Public-Access Cumputer Systems Review, vol. 8, no. 3, 1997, http://info.lib.uh.edu/pr/v8/n3/smit8n3.html.
• Sullivan, Danny, 'The Major Search Engines and Directories', searchenginewatch.com, 29 April 2003, http://searchenginewatch.com/links/article.php/2156221.
• Sullivan, Danny, 'Where Are They Now? Search Engines We've Known & Loved', searchenginewatch.com, 4 March 2003, http://www.searchenginewatch.com/sereport/article.php/2175241.
• The Clever Project, 'Hypersearching the Web', Scientific American, June 1999, Scientific American Inc, 1999, http://www.sciam.com/article.cfm?articleID=000BC474-9440-1CD6-B4A8809EC588EEDF.
• Zick, Laura., 'The Work of Information Mediators: A Comparison of Librarians and Intelligent Software Agents', First Monday, vol. 5, no. 5, May 2000, http://www.firstmonday.org/issues/issue5_5/zick/.

Reports

• Hubbard, John, Indexing the Internet, 16 April 2000, http://www.tk421.net/essays/babel.shtml.

Resources

• Australian Subject Gateways, National Library of Australia, http://www.nla.gov.au/initiatives/sg/index.html.
• Internet Search Engine Help & Tutorials, lookoff.com, 2000, http://www.lookoff.com/.
• Manning, Gerard, The WWW Virtual Library, WWW Virtual Library, 1994, http://vlib.org/Home.html.
• Notess, Greg R., Search Engine Showdown The User's Guide to Web Searching, Notess.com, 2000, http://searchengineshowdown.com/.
• Sullivan, Danny, Search Engine Watch, internet.com Corp., 1996, http://searchenginewatch.com/.

• 'Subject Gateways to the Disciplines', in Meta Matters, National Library of Australia, 1999, http://www.nla.gov.au/meta/sg/gateways.html.
• Search Tools for Web Sites, Intranets and Portals, Search Tools Consulting & Avi Rappoport, 1998, http://www.searchtools.com/.
• Barlow, Linda, The Spider's Apprentice: A Helpful Guide to Search Engines, Monash Information Services, 1997, http://www.monash.com/spidap.html.
• Coffman, Steve, 'Building Earth's Largest Library: Driving into the Future', Searcher, vol. 7, no. 3, Information Today Inc, March 1999, http://www.infotoday.com/searcher/mar99/coffman.htm.
• Dodge, Martin, An Atlas of Cyberspaces, Cyber-Geography Research, Centre for Advanced Spatial Analysis, University College London., 1997, http://www.cybergeography.org/atlas/atlas.html.