# Basic robots.txt file User-agent: * # Crawl-delay: 1 # Directories