## 📖 Overview
AnyCrawl is a high‑performance crawling and scraping toolkit:
- **SERP crawling**: multiple search engines, batch‑friendly
- **Web scraping**: single‑page content extraction
- **Site crawling**: full‑site traversal and collection
- **High performance**: multi‑threading / multi‑process
- **Batch tasks**: reliable and efficient
- **AI extraction**: LLM‑powered structured data (JSON) extraction from pages
LLM‑friendly. Easy to integrate and use.
## 🚀 Quick Start
📖 See full docs: [Docs](https://docs.anycrawl.dev)
## 📚 Usage Examples
💡 Use the [Playground](https://anycrawl.dev/playground) to test APIs and generate code in your preferred language.
> If self‑hosting, replace `https://api.anycrawl.dev` with your own server URL.
### Web Scraping (Scrape)
#### Example
```typescript
curl -X POST https://api.anycrawl.dev/v1/scrape \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer YOUR_ANYCRAWL_API_KEY' \
-d '{
"url": "https://example.com",
"engine": "cheerio"
}'
```
#### Parameters
| Parameter | Type | Description | Default |
| --------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| url | string (required) | The URL to be scraped. Must be a valid URL starting with http:// or https:// | - |
| engine | string | Scraping engine to use. Options: `cheerio` (static HTML parsing, fastest), `playwright` (JavaScript rendering with modern engine), `puppeteer` (JavaScript rendering with Chrome) | cheerio |
| proxy | string | Proxy URL for the request. Supports HTTP and SOCKS proxies. Format: `http://[username]:[password]@proxy:port` | _(none)_ |
More parameters: see [Request Parameters](https://docs.anycrawl.dev/en/general/scrape#request-parameters).
#### LLM Extraction
```bash
curl -X POST "https://api.anycrawl.dev/v1/scrape" \
-H "Authorization: Bearer YOUR_ANYCRAWL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"json_options": {
"schema": {
"type": "object",
"properties": {
"company_mission": { "type": "string" },
"is_open_source": { "type": "boolean" },
"employee_count": { "type": "number" }
},
"required": ["company_mission"]
}
}
}'
```
### Site Crawling (Crawl)
#### Example
```typescript
curl -X POST https://api.anycrawl.dev/v1/crawl \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer YOUR_ANYCRAWL_API_KEY' \
-d '{
"url": "https://example.com",
"engine": "playwright",
"max_depth": 2,
"limit": 10,
"strategy": "same-domain"
}'
```
#### Parameters
| Parameter | Type | Description | Default |
| -------------- | ----------------- | ----------------------------------------------------------------------------------------- | ----------- |
| url | string (required) | Starting URL to crawl | - |
| engine | string | Crawling engine. Options: `cheerio`, `playwright`, `puppeteer` | cheerio |
| max_depth | number | Max depth from the start URL | 10 |
| limit | number | Max number of pages to crawl | 100 |
| strategy | enum | Scope: `all`, `same-domain`, `same-hostname`, `same-origin` | same-domain |
| include_paths | array
-green){kind=icon}
[](https://opensource.org/licenses/MIT)
[](http://makeapullrequest.com)
[](https://github.com/any4ai/anycrawl)
[](https://docs.anycrawl.dev)
