# Robots.txt file from http://www.nofleasusa.com # All robots will spider the domain except directory /cgi-bin/ /images/ /lp/ User-agent: * Disallow:/cgi-bin/ Disallow:/images/ Disallow:/admin/ Disallow:/cache/ Disallow:/docs/ Disallow:/download/ Disallow:/email/ Disallow:/extras/ Disallow:/pub/ Disallow:/media/ Disallow:/product_pics/ Disallow:/README_FILES/ Disallow:/site_pics/ Disallow:/test/ Disallow:/thumbs/ Disallow:/lp/