Файл robots.txt

Источник Robots.txt: на сайте http://ryazan.hh.ru/robots.txt на момент обработки проекта.

User-agent: Yandex
Disallow: /auth/*
Disallow: /account/*
Disallow: /area_switcher*
Disallow: /vacancy/*
Disallow: /search/vacancy/advanced
Disallow: /search/resume/advanced
Disallow: /rss/*
Disallow: /admin/*
Disallow: *?*

User-agent: YandexDirect
Allow: *?*

User-agent: Googlebot
Disallow: /auth/*
Disallow: /account/*
Disallow: /area_switcher*
Disallow: /vacancy/*
Disallow: /search/vacancy/advanced
Disallow: /search/resume/advanced
Disallow: /rss/*
Disallow: /admin/*
Disallow: *?*

User-agent: LCC
Disallow: /

User-agent: ltx71 - (http://ltx71.com/)
Disallow: /

User-agent: *
Disallow: /auth/*
Disallow: /account/*
Disallow: /area_switcher*
Disallow: /vacancy/*
Disallow: /search/vacancy/advanced
Disallow: /search/resume/advanced
Disallow: /rss/*
Disallow: /admin/*
Disallow: *?*

Host: https://ryazan.hh.ru
Sitemap: https://ryazan.hh.ru/sitemap/main.xml