Add it to your agent
One command in Claude Code, a minute anywhere else. It signs you in on first use and adds 5,000 free credits.
DoubleCrawl gets your AI agent the pages its built-in fetch can't open. Add one tool, keep working, and pay only for pages that arrive.
Works with
Everything in between is our job: pick the level, retry when a page fails, and check that you got the real page.
One command in Claude Code, a minute anywhere else. It signs you in on first use and adds 5,000 free credits.
When a page won't open, your agent reaches for DoubleCrawl. Or call POST /v1/request from your code.
Right level, best route, automatic retries. Nothing to set up.
An error page isn't a page, and you don't pay for it. If it isn't the real page, we try again, on us.
The site's own HTML, ready for your agent or parser. If it doesn't arrive, you don't pay.
It opens in your browser. It fails in your agent. So you paste the page into the chat by hand. DoubleCrawl is built to be found and set up from inside the chat, so the next fetch just works.
fetch_page tool.Connect DoubleCrawl to Claude, ChatGPT, Cursor or your own agent over MCP. Prefer code? It's a single endpoint, no SDK required. You get the site's own page, and three headers tell you how it went.
curl https://api.doublecrawl.com/v1/request \
-H "Authorization: Bearer $DOUBLECRAWL_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://shop.example/running-shoes", "country": "de"}'import os, requests
r = requests.post(
"https://api.doublecrawl.com/v1/request",
headers={"Authorization": f"Bearer {os.environ['DOUBLECRAWL_API_KEY']}"},
json={"url": "https://listings.example/austin", "country": "us"},
timeout=40, # we retry for you for up to 30 s
)
if r.headers["X-DD-Target-Outcome"] == "delivered":
html = r.textconst res = await fetch("https://api.doublecrawl.com/v1/request", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.DOUBLECRAWL_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ url: "https://shop.example/running-shoes", country: "us" }),
signal: AbortSignal.timeout(40_000),
});
if (res.headers.get("x-dd-target-outcome") === "delivered") {
const html = await res.text();
}body := strings.NewReader(`{"url": "https://listings.example/paris", "country": "fr"}`)
req, _ := http.NewRequest("POST", "https://api.doublecrawl.com/v1/request", body)
req.Header.Set("Authorization", "Bearer "+os.Getenv("DOUBLECRAWL_API_KEY"))
req.Header.Set("Content-Type", "application/json")
client := &http.Client{Timeout: 40 * time.Second}
res, err := client.Do(req)
if err != nil { log.Fatal(err) }
defer res.Body.Close()
fmt.Println(res.Header.Get("X-DD-Target-Outcome")) // delivered$ch = curl_init('https://api.doublecrawl.com/v1/request');
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_HTTPHEADER => ['Authorization: Bearer ' . getenv('DOUBLECRAWL_API_KEY'), 'Content-Type: application/json'],
CURLOPT_POSTFIELDS => json_encode(['url' => 'https://listings.example/warsaw', 'country' => 'pl']),
CURLOPT_RETURNTRANSFER => true,
CURLOPT_TIMEOUT => 40,
]);
$html = curl_exec($ch);No match. Any MCP client with Streamable HTTP works with https://mcp.doublecrawl.com/mcp
https://mcp.doublecrawl.com/mcpclaude mcp add --transport http doublecrawl https://mcp.doublecrawl.com/mcphttps://mcp.doublecrawl.com/mcp{
"mcpServers": {
"doublecrawl": {
"url": "https://mcp.doublecrawl.com/mcp"
}
}
}{
"mcpServers": {
"doublecrawl": {
"serverUrl": "https://mcp.doublecrawl.com/mcp"
}
}
}code --add-mcp '{"name": "doublecrawl", "type": "http", "url": "https://mcp.doublecrawl.com/mcp"}'{
"mcpServers": {
"doublecrawl": {
"httpUrl": "https://mcp.doublecrawl.com/mcp"
}
}
}[mcp_servers.doublecrawl]
url = "https://mcp.doublecrawl.com/mcp"{
"context_servers": {
"doublecrawl": {
"url": "https://mcp.doublecrawl.com/mcp"
}
}
}import os
from langchain_mcp_adapters.client import MultiServerMCPClient
client = MultiServerMCPClient({
"doublecrawl": {
"transport": "streamable_http",
"url": "https://mcp.doublecrawl.com/mcp",
"headers": {"Authorization": f"Bearer {os.environ['DOUBLECRAWL_API_KEY']}"},
}
})
tools = await client.get_tools()import os
from agents import Agent, Runner
from agents.mcp import MCPServerStreamableHttp
async with MCPServerStreamableHttp(params={
"url": "https://mcp.doublecrawl.com/mcp",
"headers": {"Authorization": f"Bearer {os.environ['DOUBLECRAWL_API_KEY']}"},
}) as doublecrawl:
agent = Agent(name="Researcher", mcp_servers=[doublecrawl])
result = await Runner.run(agent, "How many flats are for sale in Warsaw?")import { createMCPClient } from '@ai-sdk/mcp';
const doublecrawl = await createMCPClient({
transport: {
type: 'http',
url: 'https://mcp.doublecrawl.com/mcp',
headers: { Authorization: `Bearer ${process.env.DOUBLECRAWL_API_KEY}` },
},
});
const tools = await doublecrawl.tools();import os
from crewai import Agent
from crewai_tools import MCPServerAdapter
params = {
"url": "https://mcp.doublecrawl.com/mcp",
"transport": "streamable-http",
"headers": {"Authorization": f"Bearer {os.environ['DOUBLECRAWL_API_KEY']}"},
}
with MCPServerAdapter(params) as tools:
researcher = Agent(role="Market Researcher", goal="Collect prices", tools=tools)https://mcp.doublecrawl.com/mcpNo key yet? The server signs you in on first use and adds 5,000 free credits. Step-by-step guides →
Pick yours for a one-minute setup.
Whatever you or your agent need from the web, DoubleCrawl brings back the real page.
Give your agent live web access that doesn't stop at the first error page. Research and comparisons that actually finish.
Track prices and availability across shops and marketplaces, in the countries you sell in.
Read thousands of company websites and score them against your ideal customer.
Live pages for your knowledge base or search index.
New listings from property portals and classifieds, as often as you check.
Follow reviews, ratings and mentions of your brand and your competitors.
Fares, rates and availability from booking sites, airlines and ticketing platforms.
If your agent can't open a page, try it through DoubleCrawl.
Connect your agent →Browser tools are great for the apps you're logged into. Your agent doesn't need your account to read the rest of the web, and neither does your laptop.
Keep your browser for the tools you're logged into. Send the rest of the web through DoubleCrawl.
Fetch is ready today. Search, map, extract and interact are coming to the same account.
Your agent asks for a URL and gets the page. Some pages still won't open. When that happens, you'll know, and you won't pay.
Pass a country code to see prices, results and stock as locals do.
Ask a question, get live results your agent can open.
Join the waitlist →List every page on a site before your agent reads a single one.
Join the waitlist →Click, type and scroll on a page, without your own browser.
Join the waitlist →Describe the fields in plain words and get JSON back.
Join the waitlist →More ways to read the web, on the same account.
Join the waitlist →Start with 5,000 free credits. That's 5,000 pages at Level 1, 1,250 at Level 2 or 200 at Level 3. Every plan includes every level, country targeting and the MCP server. Plans differ mainly in credits and how many requests you can run at once, plus the extras listed on each plan.
From $0.045 per 1,000 credits above 12M. 1,000+ concurrent requests, SLA, dedicated routes, annual contract.
Move the slider to your monthly volume.
We detect the level automatically and bill the one it actually took. You never pick a level up front, and every response tells you what it cost.
| Level | Credits | Typical targets |
|---|---|---|
| Level 1 | 1 | APIs, news, blogs, most sitesUnlimited transfer |
| Level 2 | 4 | Marketplaces, classifieds, busy sites500 KB per page included |
| Level 3 | 25 | The hardest pages500 KB per page included |
No. We charge credits only for delivered pages. Failed tries, timeouts and our own retries are on us.
1 credit at Level 1, 4 at Level 2 and 25 at Level 3. We pick the level automatically and return the cost with every response in X-DD-Credits.
Yes. DoubleCrawl has an MCP server, an llms.txt file and errors written for agents, so Claude, ChatGPT, Cursor and custom agents can set it up and fetch pages for you.
The page, or an honest error. Some pages still won't open. When that happens you get a clear refused result, never an empty page, and it costs 0 credits.
No. DoubleCrawl runs on our servers, so it needs neither your browser nor your account.
The number of requests you can have in flight at the same time. Free has 5, Start 20, Growth 50, Business 150, Scale 500 and Enterprise 1,000 or more. On pay as you go, ask us. Requests over the limit get a clear error.
Yes. Pass a two-letter country code. Without one, we pick the location with the best success rate for the site.
Today you get the page's HTML as the site serves it. Markdown, JSON extraction and screenshots are coming.
They are next on the list. Fetch is the only product live today, and we won't put dates on the rest until they are ready.
Pages behind a login you don't own, and anything that breaks the law or our acceptable use policy.
Your first 5,000 credits are on us. Add DoubleCrawl to your agent and ask it for the page it couldn't get.