# Applebot

`Applebot` is [Apple](https://apple.com/)'s web crawler that discovers and indexes publicly available web content to power search and suggestion features across [Apple](https://apple.com/) products, including Spotlight, Siri, and Safari Suggestions. Crawled content also feeds into [Apple](https://apple.com/) Intelligence and generative AI features. `Applebot` can drive referral traffic to your site through clickable search results, suggestions, titles, snippets, and links presented to users across [Apple](https://apple.com/) platforms.

## Should I allow and optimize for Applebot to drive organic growth?

`Applebot` directly powers search results and suggestions across Spotlight, Siri, and Safari on hundreds of millions of [Apple](https://apple.com/) devices. Allowing it means your content can appear as clickable results with titles, snippets, and links that drive users back to your site. Blocking `Applebot` removes your content from these surfaces entirely. Even the training component has indirect value: your content helps shape the AI models behind [Apple](https://apple.com/) Intelligence features, which may surface your brand or information in AI-powered responses. For most sites, allowing `Applebot` (and optionally blocking only `Applebot`-Extended if you want to opt out of training) is the right call.

Here's how to optimize for Applebot:

- Allow Applebot in your robots.txt to appear in Spotlight, Siri, and Safari Suggestions
- Ensure JavaScript, CSS, and XHR resources are not blocked so Applebot can fully render your pages
- Add a sitemap reference in robots.txt to help Applebot discover all your pages
- Use descriptive title tags and meta descriptions, since these appear as snippets in Apple search results
- Implement structured data (JSON-LD) to help Applebot understand your content
- Support universal links and deep links to enable direct app launches from Apple search results
- Block Applebot-Extended (not Applebot) if you want to opt out of AI training while keeping search visibility

## Data Usage & Training

Content crawled by `Applebot` may be used to help train [Apple](https://apple.com/)'s foundation models. [Apple](https://apple.com/) applies filters to remove profane or low-quality content and reduce inclusion of personally identifiable information. You can opt out of training use specifically by blocking `Applebot`-Extended in your robots.txt, which stops training crawls while still allowing `Applebot` to index your content for search and suggestions.

## How Applebot Accesses Content

Here's how Applebot accesses your site and understands your content:

- Fetches HTML and fully renders JavaScript (including CSS and XHR requests)
- Uses a browser-like user-agent string containing '`Applebot`/0.1' and a link to apple.com/go/`applebot`
- Respects robots.txt Disallow and Allow directives, plus Sitemap references
- Honors meta robots tags including noindex, nosnippet, and nofollow
- IP addresses can be verified via reverse DNS (PTR records under `applebot`.apple.com) and the published CIDR list

`Applebot` performs periodic, ongoing crawls to maintain [Apple](https://apple.com/)'s server-side search index. This is continuous scheduled crawling rather than on-demand fetching.

## How to Block or Control Applebot

To block all `Applebot` crawling:

```
User-agent: `Applebot`
Disallow: /
```

To block only AI training use while allowing search indexing:

```
User-agent: `Applebot`-Extended
Disallow: /
```

For page-level control, use meta robots tags (noindex, nosnippet, nofollow). You can verify and block `Applebot` by IP using the published CIDR ranges at [http://search.developer.apple.com/applebot.json.](http://search.developer.apple.com/applebot.json.) For removal requests, contact `applebot`@apple.com or use [Apple](https://apple.com/)'s Intelligence Privacy Inquiries form.

## Common Issues & Troubleshooting

Watch out for these common problems when working with Applebot:

- Blocking JavaScript, CSS, or XHR resources prevents `Applebot` from rendering pages fully, which degrades index quality
- Blocking only `Applebot`-Extended does not stop search crawling; `Applebot` will still index your pages for Spotlight and Siri
- Robots.txt rules targeting other crawlers (like `Googlebot`) can unintentionally affect `Applebot` if no `Applebot`-specific rules are present
- Crawl-delay is not supported, so you cannot throttle `Applebot` through robots.txt
- Sites relying solely on IP blocking may miss new CIDR ranges if they don't periodically refresh from `applebot`.json

## Quick Reference

**Platform**: [Apple](/content/ai-agent-directory?platforms=Apple/index.html)  
**Agent Category**: [Search Engine Crawler](/content/ai-agent-directory?tags=Search%20Engine%20Crawler/index.html) [AI Training](/content/ai-agent-directory?tags=AI%20Training/index.html)  
**Growth Value**: High  
**Official Documentation**: [support.apple.com/en-us/119829](https://support.apple.com/en-us/119829)  
**User Agent String**: `applebot`  
**robots.txt Entry**: 
```
User-agent: applebot
Disallow: /
```

## Frequently Asked Questions

### Does Applebot respect robots.txt?

### What is the difference between Applebot and Applebot-Extended?

### Does Applebot use my content to train AI models?

### How do I verify that a request is really from Applebot?

### Can Applebot render JavaScript?

### Does allowing Applebot drive traffic to my site?

### Does Applebot support Crawl-delay?