Файл robots.txt

Источник Robots.txt: на сайте http://bestmebel.com/robots.txt на момент обработки проекта.

User-agent: *
Disallow: /mailer/unsubscribe/
Disallow: /blog/
Disallow: /forgotpassword/
Disallow: /login/
Disallow: /my*
Disallow: /*/reviews/
Disallow: /captcha.php?rid=*
Disallow: /signup/
Disallow: /webasyst/
Disallow: /checkout/
Disallow: /cart/
Disallow: /cart2/
Disallow: /search/
Disallow: /compare/
Disallow: /*?*
Allow: /*?utm*
Sitemap: https://bestmebel.com/sitemap.xml

User-agent: Yandex
Disallow: /mailer/unsubscribe/
Disallow: /blog/
Disallow: /forgotpassword/
Disallow: /login/
Disallow: /my*
Disallow: /*/reviews/
Disallow: /captcha.php?rid=*
Disallow: /signup/
Disallow: /webasyst/
Disallow: /checkout/
Disallow: /cart/
Disallow: /cart2/
Disallow: /search/
Disallow: /compare/
Disallow: /*?*
Allow: /*?utm*
Clean-param: utm_source&utm_medium&utm_campaign&utm_term&utm_content&utm_nooverride&utm_expid&src&gclid&itaword
Host: https://bestmebel.com
Sitemap: https://bestmebel.com/sitemap.xml