Managing robots.txt file effectively

Managing robots.txt file effectively

  Sometimes web owners feel the need of excluding some of their web pages from being indexed, they can tell the search engine spiders ( robots ) to ignore parts of their site in a few different ways.  One way to tell search engines which files and folders on your Website to avoid is with the use of the Robots meta tags. But since not all search engines read meta tags, the Robots meta tags can simply go unnoticed. A…

Read More Read More