In order to avoid unwanted written content in the lookup indexes, site owners can instruct spiders never to crawl certain data files or directories from the conventional robots.txt file in the root directory in the area. Additionally, a web page could be explicitly excluded from a internet search engine's databases https://www.facebook.com/acourseinmiraclesdavid/