Link Checking Software
Link checking software such as Xenu’s Link Sleuth enables a webmaster to locate broken links, as well as count the number of URLs on a site.
Link Sleuth is a free to download piece of software, but there are many others available. The software works by spidering links in your site the same way a search engine would, thus enabling you to see if there are any pages that are not spidered but you feel are important. Link checking will also help in spotting if the same content is available on 2 or more URLs.
It is therefore very important to spider a new site with link checking software before launching it, to prevent any indexing errors. It is also wise to periodically run the link checking software as a site grows.
