Batch URLs

One call. Up to 10 URLs. Billed only for the ones that land.

POST /v1/scrape/batch Tiered · per URL Utilities

Hand us a list of up to 10 URLs and get back one array of results, fetched concurrently. Each URL carries its own status, cleaned text, proxy tier, and credit line, so a slow or blocked page never holds up — or fails — the rest. The render options you set apply to every URL in the batch.

The hard part

Fanning out a list of URLs yourself means juggling a concurrency pool, per-URL retries, partial failures, and a rate limiter so you don't trip the target. Then you still have to stitch the results back together and reason about which ones to bill. We fan out concurrently, isolate each failure, and hand back one array with a per-URL bill.

What it does

Up to 10 URLs per call, fetched concurrently.
Each URL gets its own row: `status_code`, `method`, `proxy_used`, `text`, and `credits_charged`.
One URL failing never fails the batch — undeliverable URLs return an `error` and bill 0; a delivered 404 bills 1 credit.
Shared options apply to every URL: `render`, `force_proxy`, `country`, `return_markdown`, `include_html`, `block_ads`.
`succeeded` plus a per-URL `credits_charged` make the bill obvious.
Same anti-bot escalation and tier-aware pricing as Any URL.

Send a call

curl -X POST https://scrapenest.dev/v1/scrape/batch \
  -H "Authorization: Bearer sn_••••" \
  -H "Content-Type: application/json" \
  -d '{ "urls": ["https://example.com", "https://example.org"], "render": "auto" }'

import httpx

r = httpx.post(
    "https://scrapenest.dev/v1/scrape/batch",
    headers={"Authorization": "Bearer sn_••••"},
    json={
        "urls": [
            "https://example.com",
            "https://example.org",
        ],
        "render": "auto",
    },
    timeout=120,
)
data = r.json()
print(data["succeeded"], "/", data["requested"], "ok")
for row in data["results"]:
    print(row["status_code"], row["url"])

const r = await fetch("https://scrapenest.dev/v1/scrape/batch", {
  method: "POST",
  headers: {
    "Authorization": "Bearer sn_••••",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    urls: ["https://example.com", "https://example.org"],
    render: "auto",
  }),
});
const data = await r.json();
console.log(`${data.succeeded}/${data.requested} ok`);

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
)

func main() {
    body, _ := json.Marshal(map[string]any{
        "urls":   []string{"https://example.com", "https://example.org"},
        "render": "auto",
    })
    req, _ := http.NewRequest("POST",
        "https://scrapenest.dev/v1/scrape/batch", bytes.NewReader(body))
    req.Header.Set("Authorization", "Bearer sn_••••")
    req.Header.Set("Content-Type", "application/json")
    resp, _ := http.DefaultClient.Do(req)
    defer resp.Body.Close()
    var out struct {
        Requested int `json:"requested"`
        Succeeded int `json:"succeeded"`
    }
    json.NewDecoder(resp.Body).Decode(&out)
    fmt.Printf("%d/%d ok\n", out.Succeeded, out.Requested)
}

Request body

{
  "urls": [
    "https://example.com",
    "https://example.org"
  ],
  "render": "auto"
}

Response shape

{
  "results": [
    {
      "url": "https://example.com",
      "final_url": "https://example.com/",
      "status_code": 200,
      "method": "direct",
      "proxy_used": null,
      "elapsed_ms": 412,
      "title": "Example Domain",
      "text": "Example Domain. This domain is for use in examples …",
      "ok": true,
      "credits_charged": 1
    },
    {
      "url": "https://example.org",
      "status_code": null,
      "elapsed_ms": 88,
      "ok": false,
      "credits_charged": 0,
      "error": "upstream_failed"
    }
  ],
  "requested": 2,
  "succeeded": 1,
  "credits_charged": 1
}

What it costs

Each URL is billed exactly like Any URL — at the tier that actually rendered it (1 credit for a plain fetch, up to 40 for a full stealth render). Undeliverable URLs don't bill; a delivered 404 'not found' bills 1 credit. Your total is the sum of the per-URL charges.

For a single URL, use Any URL (/v1/scrape/url) — same engine, simpler response.
Need AI extraction or extract_rules per URL? Call Any URL individually; batch is built for bulk fetch.