We’re seeing errors flagged in Search Console because pages are indexed even though they’re blocked in our robots.txt file. A better approach to block them would be to add the meta noindex tag (follow and nofollow should both be options for us to select from) to individual pages and/or directories.
Support for this would be great. Robots.txt is not a perfect solution for indexation management as Google occasionally ignores it - especially if the URLs have been indexed in the past.
We've resolved this issue through a script that checks the URL against a series of rules and injects / updates the robots element for the page - but native solutions are of course better as it doesn't require JS execution.
Hi
What other pages do you want to have that flexibility? We also noticed that there are some conflicts between Robots.txt and canonicals inside some pages and actively working on them to improve. You can also find this idea interesting:
I''d put this idea to ''open'' to collect more votes and feedback.
'If you ever had a profile with us, there's no need to create another one.
Don't worry if your email address has since changed, or you can't remember your login, just let us know at community@gainsight.com and we'll help you get started from where you left.
Else, please continue with the registration below.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
