Competitive intelligence

Track competitor pricing, features and messaging. Extract the same fields on your own schedule and compare runs, so a price change shows up as a changed field.

Get an API key Try it in the playground

Why competitor monitoring is noisy

A page-change alert fires on any HTML difference: a moved nav link, a new script tag, a cookie banner. After a few weeks of false alarms, people stop reading the alerts, and the real price change goes unnoticed.

Coverage is the other gap. A pricing change can touch the pricing page, the comparison table, a few blog posts and the case-study calls to action. A monitor that watches one page misses the rest.

How SuperScraper fits

Extract structured fields, run the calls on your schedule and compare the JSON across runs. You alert on fields that changed.

Compare fields across runs: Extract against a JSON schema on every run and compare the result to the previous one. A price change shows up as a new value in one field.
You own the schedule: Call /v1/extract or /v1/batch from a cron job, a queue or a workflow tool you already run. Store each result and compare it to the last. Hosted scheduled monitoring is not available.
Cover the whole site: /v1/map lists the URLs on a competitor’s domain from its sitemap. /v1/crawl fetches a site as an async job and returns each page as markdown.
Skip thin runs: On /v1/scrape, metadata.completeness scores how complete the page’s structured data is. With a schema, check the fields you need for null and skip runs where they came back empty.

Endpoints

POST/v1/mapEvery URL on a domain, from its sitemap or homepage links.
POST/v1/crawlA whole site as an async job, returned as markdown.
POST/v1/extractSpecific fields from any page, against your schema.
POST/v1/batchMany URLs in one request, up to your plan’s cap.

Set up a pricing monitor

Extract the pricing structure against a schema. Run the same call from your scheduler and compare each result to the last one.

Example responses use fictional data.

# One extraction. Run it from your own scheduler.
curl -X POST https://api.superscraper.dev/v1/extract \
  -H "Authorization: Bearer $SS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://competitor.example/pricing",
    "schema": {
      "plans": {
        "type": "array",
        "items": {
          "name": "string",
          "priceMonthly": "number",
          "priceAnnual": "number",
          "features": "string[]"
        }
      }
    }
  }'

Questions

How do I monitor a competitor pricing page for changes?

POST to /v1/extract with the URL and a JSON schema for the fields you want, such as plan name, price and features. Run that call on your own schedule from a cron job or workflow tool, store each result and compare it to the previous run. Hosted scheduled monitoring with change alerts is not available.

Can I monitor a whole competitor site?

Yes. Use /v1/map to list the URLs on the domain, then send them to /v1/extract as a urls array with one schema, or fetch them with /v1/batch up to your plan’s cap. Re-run on your schedule and compare results.

How does the API handle JavaScript-heavy pricing pages?

The fetch cascade tries a plain HTTP fetch first and moves to a real browser when the page needs JavaScript. On a /v1/scrape response, metadata.fetchMethod says which method returned the page. Send "forcePlaywright": true to start with the browser.

Is this legal for competitive research?

Reading public web pages for competitive research is common practice; search engines and price comparison sites do the same. You are responsible for each site’s terms of service and the law where you operate. Scrape and crawl respect robots.txt by default.

Other use cases

Start watching your competitors

Run any endpoint in the playground, or get a free key. 1,000 credits a month, no card.

Get an API key Try it in the playground