## robots.txt

User-agent: *
Disallow: /cgi-bin/        # various scripts that should not be indexed

User-agent: Mediapartners-Google* 
Disallow: