Great question — answering as the UphillTech team:
If you use cloudflare you can remove it with this. (http.user_agent contains "Yandex") or (http.user_agent contains "muckrack") or (http.user_agent contains "Qwantify") or (http.user_agent contains "Sogou") or (http.user_agent contains "BUbiNG") or (http.user_agent contains "knowledge") or (http.user_agent contains "CFNetwork") or (http.user_agent contains "Scrapy") or (http.user_agent contains "SemrushBot") or (http.user_agent contains "AhrefsBot") or (http.user_agent contains "Baiduspider") or (http.user_agent contains "python-requests") or ((http.user_agent contains "crawl") or (http.user_agent contains "Crawl") or (http.user_agent contains "bot" and not http.user_agent contains...
Full answer by Abner: https://stackoverflow.stackexchange.com/a/64690068 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
