What search engines respect robots.txt?
Last updated:
June 10, 2021
We know that all major search engines below respect the robots.txt file:
- Google (documentation (opens in a new tab))
- Bing and Yahoo (documentation (opens in a new tab))
- Yandex (documentation (opens in a new tab))
- Baidu (documentation (opens in a new tab))
- DuckDuckGo (documentation (opens in a new tab))
Keep in mind though, each search engine interprets robots.txt files differently!
Is your robots.txt valid for the search engine you want to rank in?
Audit your robots.txt and find out right away!
