Live web access for AI agents

Give an LLM agent the live web over a REST API. Each scrape reports how complete its structured data is and how the page was fetched, so the agent can decide what reaches the model.

Get an API key Try it in the playground

What agents need from web access

An agent that reads raw HTML or bare markdown gets no signal about quality. It cannot tell a fact from a navigation link, or a full extraction from a thin one, so it passes everything downstream as equally reliable.

Onboarding is the second gap. If each agent deployment needs a person to sign up for an API key, the agent cannot start on its own. It needs a key it can mint itself, within a rate limit.

How SuperScraper fits

Agents call the REST API with a Bearer key. Hosted MCP is coming soon; until then, REST is the path.

Plain HTTP: Call scrape, extract, map, search, enrich and crawl with a Bearer key. No SDK required.
A score an agent can branch on: A /v1/scrape response carries metadata.completeness, a 0 to 1 score for how complete the page’s structured data is, plus metadata.fetchMethod. The agent can use a complete result as is, flag a thin one for review, or retry with forcePlaywright.
A source URL to cite: Every response includes the url it was read from, and /v1/scrape adds the fetch steps that ran. An agent writing a report can cite the url without another lookup.
Search when there is no URL: POST a query to /v1/search for ranked results. Add scrapeResults: true to get each page’s content in the same call.

Endpoints

POST/v1/scrapeA URL as markdown or structured data, with a completeness score.
POST/v1/extractA URL plus your schema, returned as typed fields in data.
POST/v1/searchA query to ranked results. scrapeResults: true adds page content.
POST/v1/mapEvery URL on a domain, for site navigation.
POST/v1/keys/provisionA free-tier key the agent mints itself, with no signup.

Check the result before the agent acts

Call /v1/extract with a schema and check that the fields you need came back. A schema request returns data and model metadata without a completeness score, so check the fields themselves.

const res = await fetch('https://api.superscraper.dev/v1/extract', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.SS_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    url: 'https://example.com/product',
    schema: {
      productName: 'string',
      price: 'number',
      availability: 'string',
    },
  }),
});

const { data } = await res.json();

const required = ['productName', 'price'];
if (data && required.every((k) => data[k] != null)) {
  // Every required field came back: use it
  return data;
}
// Missing fields: flag for review
return { ...data, _needsReview: true };

Questions

How does an agent call SuperScraper?

Over the REST API with a Bearer key. POST /v1/scrape, /v1/extract, /v1/search and /v1/map are the main calls. Hosted MCP is coming soon.

Can an AI agent provision its own API key without a human signing up?

Yes. POST /v1/keys/provision returns a free-tier key with no signup. The agent calls it once at onboarding and stores the key for later calls. The free tier includes 1,000 credits a month.

What is the difference between /v1/scrape and /v1/search for agents?

/v1/scrape fetches a URL the agent already has and returns markdown or structured data. /v1/search takes a query and returns matching pages, plus their content when you set scrapeResults: true. Use search when the agent starts from an instruction rather than a URL.

How should an agent use the completeness score?

On /v1/scrape, metadata.completeness is a 0 to 1 score for how complete the page’s structured data is. One way to use it: above 0.85, use the result as is; from 0.5 to 0.85, include it with a caveat; below 0.5, drop it or flag it for review. Tune the cutoffs to your task. metadata.fetchMethod says separately how the page was fetched.

Other use cases

Give your agent the live web

Get a free key and call the REST API. 1,000 credits a month, no card.

Get an API key Try it in the playground