User-agent: * Disallow: / User-agent: Slurp Disallow: / # Allow State net's robot in User-agent: get-nh.track Allow: / User-agent: get-nh.text Allow: /