The AI Content Control feature in the SpeedBoostr app allows you to control how AI platforms access your website's public content.
What is its purpose?
The purpose of this feature is to manage the LLMs.txt (Large Language Models Text File). It helps AI platforms such as ChatGPT, Claude, Gemini, Google AI, Meta AI, Perplexity, and others understand which content on your website should be available for AI to access and use.
Benefits
- AI platforms can easily discover your products and website pages.
- Protect your content from being used for AI training without your preference.
- Increases the chances of your products appearing in AI-generated recommendations.
- Maintain customer privacy, as customer and checkout data remain protected.
- Choose which AI crawlers are allowed or blocked based on your business needs.
How does it work?
The feature updates your website's robots.txt file (or uses AI-specific crawling rules) to manage access for supported AI crawlers.
Typically, you can:
- AllowAI bots to access your public content.
- BlockAI bots from accessing your public content.
How to Use the LLMs.txt Feature
First, enable the "llms.txt is published" toggle.
- ON → An llms.txt file will be generated, allowing AI platforms to discover the content you have selected.
- OFF → > The llms.txt file will not be published.
Included Content
This section allows you to choose which parts of your website should be available to AI models.
1. Products
ON → AI AI platforms can discover your product pages.
Example:
- Product name
- Product description
- Product details
2. Collections
ON → AI AI platforms can access your collection/category pages.
Examples:
- Shorts
- Summer Collection
3. Blog Articles
ON → AI AI platforms can discover your blog posts.
Examples:
- Buying Guides
- Product Tutorials
4. Pages
ON → AI platforms can discover your important website pages.
Examples:
- Wishlist
- Contact Us
- Order History
- Profile
- Rewards
Preview Button
Click the Preview button to see how your llms.txt file will be generated and what information will be available to AI platforms.
Manage URLs
This option allows you to manually manage URLs.
You can:
- Include specific URLs.
- Exclude specific URLs.
This gives you greater control over which pages AI platforms can access.
In short, AI Content Control gives you control over whether AI platforms can use your website's public content, while keeping customer and checkout data private