What searche engines respect robots.txt?
Last updated:
September 11, 2020
We know that all major search engines below respect the robots.txt file:
- Google (documentation (opens in a new tab))
- Bing and Yahoo (documentation (opens in a new tab))
- Yandex (documentation (opens in a new tab))
- Baidu (documentation (opens in a new tab))
- DuckDuckGo (documentation (opens in a new tab))
Keep in mind though, each search engine interprets robots.txt files slightly different!
