Does Not Find All the Pages

There are a few possible reasons why SiteSurfer Builder's robot would would not find all the pages on a site:

1. Pages are Excluded by Robots Directives. If robots support is enabled in SiteSurfer Builder, and either the web site has a robots.txt file or pages with ROBOTS meta tags, then some pages of the site will either (1) not be indexed or (2) not crawled to find links to other pages or both (1 and 2).

2. Pages are Excluded by Name Patterns. If a page's address matches a pattern in the Patterns options page, and that pattern is enabled and set to Exclude, then that page will be neither indexed nor crawled for links to other pages.

3. Invalid or Missing Links. If the starting page from which you indexed the site did not have links that extend to all parts of the site--either directly or through child pages--or if some of the links are malformed, SiteSurfer may not be able to find every page. If you use the local mirror method of indexing a web site off of your hard drive, then this problem will not affect finding pages, since every file under the directory containing the root page--and every subdirectory--will be located.