Skip to main content

Hi Gainsight team,

I’m referring to this article here:

https://www.cloudflare.com/press-releases/2025/cloudflare-just-changed-how-ai-crawlers-scrape-the-internet-at-large/

Cloudflare, Inc. [...] today announced it is now the first Internet infrastructure provider to block AI crawlers accessing content without permission or compensation, by default. Starting today, website owners can choose if they want AI crawlers to access their content, and decide how AI companies can use it. AI companies can also now clearly state their purpose – if their crawlers are used for training, inference, or search – to help website owners decide which crawlers to allow.

As Gainsight is using Cloudflare, this is highly relevant to all Gainsight platforms. 

Can you give us some information about how Gainsight has set up this new configuration? In addition to that, can you give us some insights about how Gainsight will ensure its platforms to support (or even boost) AI crawlers to process the content?

See also:

Thanks! :)

Hey ​@bjoern_schulze 

I’ve raised this with the team internally and will loop back soon.

 

Interested in this too! 

Hi ​@bjoern_schulze - We use Cloudfront.  I don’t believe we use Cloudflare.  

For the broader question of handling AI crawlers, in our platform it’s currently possible to block the main AI crawlers via the robots.txt file if desired.  Of course, as is the case with other web crawlers, any crawler designed specifically to ignore those instructions will do so, but robots.txt should cover most.  

And while don’t have any news right now around handling llms.txt files (we’re looking into it), the way we structure content for SEO (added rich schema markup) already helps AI models to understand our content.

Hi ​@bjoern_schulze, I’m interested in this topic too, thanks for adding your question here!

​@Kenneth R would you mind elaborating on your last point about schema markup helping AI models to understand our content? 

​@Ambar D - I don’t know all the technical detail but what we basically mean is that the community pages are structured in a way that helps both search engine crawlers and AI models understand and make sense of the contents of the page.

Terms & ConditionsCookie settingsAccessibility statement