Deep Search

SERP and page bodies in one round-trip. Built for RAG.

POST /v1/search/deep 5 + 5 / page Search & Discovery

Same multi-engine SERP as /v1/search, but with the top-N pages fetched, anti-bot-cleared, and parsed into readable text in the same response. Stop bolting a fetcher onto every search call.

The hard part

Search-then-fetch is a two-step pattern that gets brittle in production: one slow site holds up the whole batch, retries pile up, and you end up paying for failed fetches that came back blocked. One round-trip with a per-page deadline, and failed pages don't bill.

What it does

Single round-trip: SERP plus body text in one JSON.
Per-page fetch budget scales with `depth` (~2.5s fast → ~12s stealth; default ~3.5s) so slow news sites don't block the response.
Returns extracted text, final URL, status, and elapsed time per page.
Pages that fail fetching don't bill. You pay only for what you got.
`depth=stealth` allows full anti-bot escalation on page fetches for hard sites (slower, maximum coverage).

Send a call

curl -X POST https://scrapenest.dev/v1/search/deep \
  -H "Authorization: Bearer sn_••••" \
  -H "Content-Type: application/json" \
  -d '{
        "query": "fastapi vs litestar 2026",
        "num_results": 8,
        "fetch_top": 5
      }'

import httpx

r = httpx.post(
    "https://scrapenest.dev/v1/search/deep",
    headers={"Authorization": "Bearer sn_••••"},
    json={
        "query": "fastapi vs litestar 2026",
        "num_results": 8,
        "fetch_top": 5,
    },
    timeout=60,
)
data = r.json()
for page in data["pages"]:
    print(page["url"], len(page.get("text") or ""))

const r = await fetch("https://scrapenest.dev/v1/search/deep", {
  method: "POST",
  headers: {
    "Authorization": "Bearer sn_••••",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    query: "fastapi vs litestar 2026",
    num_results: 8,
    fetch_top: 5,
  }),
});
const { pages } = await r.json();

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
)

func main() {
    body, _ := json.Marshal(map[string]any{
        "query":       "fastapi vs litestar 2026",
        "num_results": 8,
        "fetch_top":   5,
    })
    req, _ := http.NewRequest("POST",
        "https://scrapenest.dev/v1/search/deep", bytes.NewReader(body))
    req.Header.Set("Authorization", "Bearer sn_••••")
    req.Header.Set("Content-Type", "application/json")
    resp, _ := http.DefaultClient.Do(req)
    defer resp.Body.Close()
    var out struct{ Pages []struct{ URL, Text string } }
    json.NewDecoder(resp.Body).Decode(&out)
    for _, p := range out.Pages { fmt.Println(p.URL, len(p.Text)) }
}

Request body

{
  "query": "fastapi vs litestar 2026",
  "num_results": 8,
  "fetch_top": 5,
  "render": "auto"
}

Response shape

{
  "query": "fastapi vs litestar 2026",
  "results": [
    {
      "position": 1,
      "url": "https://blog.litestar.dev/comparison",
      "title": "FastAPI vs Litestar in 2026",
      "snippet": "A side-by-side look at routing, DI, and performance …",
      "engine": "bing",
      "engines": ["bing", "google"],
      "score": 0.92,
      "published_at": null,
      "thumbnail": null
    }
  ],
  "pages": [
    {
      "url": "https://blog.litestar.dev/comparison",
      "final_url": "https://blog.litestar.dev/comparison",
      "status": 200,
      "method": "fetch",
      "title": "FastAPI vs Litestar in 2026",
      "text": "Litestar 3.0 ships … (cleaned readable text)",
      "structured": null,
      "error": null,
      "elapsed_ms": 812
    }
  ],
  "took_ms": 2110,
  "fetched_count": 5,
  "cache_hit": false,
  "credits_charged": 30
}

What it costs

5 credits base for the SERP, plus 5 credits per page actually fetched. Pages that fail (timeout, block, 4xx) do not bill.