User-agent: *
Disallow:
Crawl-delay: 10         # non-standard
Request-rate: 1/10      # extended standard
Visit-time: 0000-0700  # extended standard