引言：
七牛云存储robots.txt搜索引擎规则 七牛云存储设置了自定义域名后,可能会造成重复收录的问题,所以需要设置robots.txt 来禁止搜索引擎对自定义域名的收录。
七牛云默认robots.txt的是禁止所有的引擎收录的：
# robots.txt generated at http://portal.qiniu.com User-agent: Baiduspider Disallow: / User-agent: Sosospider Disallow: / User-agent: sogou spider Disallow: / User-agent: YodaoBot Disallow: / User-agent: Googlebot Disallow: / User-agent: Bingbot Disallow: / User-agent: Slurp Disallow: / User-agent: MSNBot Disallow: / User-agent: googlebot-image Disallow: / User-agent: googlebot-mobile Disallow: / User-agent: yahoo-blogs/v3.9 Disallow: / User-agent: psbot Disallow: / User-agent: * Disallow: /
robots.txt 规则同样也阻止了对图片等必要资源的收录，可以稍作修改后再上传到七牛云存储即可。
# robots.txt generated at http://portal.qiniu.com # edit for statice domain User-agent: Baiduspider Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: Baiduspider-image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: 360Spider Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: 360Spider-Image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: Sosospider Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: sogou spider-Image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: YodaoBot Disallow: / User-agent: Googlebot Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: Googlebot-image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: Bingbot Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: Bingbot-image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: Slurp Disallow: / User-agent: MSNBot Disallow: / User-agent: googlebot-image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: googlebot-mobile Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.webp$ Disallow: / User-agent: yahoo-blogs/v3.9 Disallow: / User-agent: psbot Disallow: / User-agent: * Disallow: /
