Competitive intelligence
Monitor competitors' pricing, features, and messaging. Scheduled feeds extract structured data on every run and surface only what changed, so you get a field delta, not a wall of noise.
Why competitive monitoring is hard
Most web change alerts trigger on any HTML difference, a nav link moves, a script tag changes, a cookie banner appears. That creates more noise than signal. By the time something real changes, you've already tuned out the alerts.
The second problem is depth. A competitor's pricing restructure might live on twenty different pages, the main pricing page, feature comparison tables, blog posts, case study CTAs. A single-page monitor misses most of it.
How SuperScraper fits
Extract structured fields on a schedule. Compare structured JSON across runs. Alert on real field changes, not HTML noise.
Structured diffs, not raw HTML
Extract fields against a JSON schema on every run. Compare the current result to the previous one in structured form, a pricing change shows up as a field delta, not a wall of HTML diff.
Scheduled feeds out of the box
POST /v1/feeds creates a recurring scrape on any URL with a cron schedule. Each run stores the result and surfaces what changed, so you can alert on actual delta rather than polling manually.
Map + crawl for full site coverage
/v1/map discovers every URL on a competitor's domain in one call. /v1/crawl processes them all and returns structured markdown. Use both to audit a competitor's entire feature and pricing surface.
Confidence scores filter noise
Rendered JavaScript pricing tables score differently from JSON-LD structured data. The _confidence field lets you weight signals appropriately and skip low-confidence extractions from noisy pages.
/v1/feeds
Recurring scrape on a cron schedule, alert on delta
/v1/map
Discover every URL on a competitor's domain
/v1/crawl
Process an entire site and return structured markdown
/v1/extract
LLM schema extraction, pull specific fields from any page
/v1/batch
Process up to 100 URLs in one request
Set up a pricing monitor in two calls
First, extract the competitor's pricing structure against a schema to see what the response looks like. Then create a feed that runs on a schedule.
# Step 1, one-off extraction to validate your schema
curl -X POST https://superscraper-production-1381.up.railway.app/v1/extract \
-H "Authorization: Bearer $SS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://competitor.com/pricing",
"schema": {
"plans": {
"type": "array",
"items": {
"name": "string",
"priceMonthly": "number",
"priceAnnual": "number",
"features": "string[]"
}
}
}
}'
# Response
{
"plans": [
{ "name": "Starter", "priceMonthly": 49, "priceAnnual": 39, "features": ["..."] },
{ "name": "Pro", "priceMonthly": 149, "priceAnnual": 119, "features": ["..."] }
],
"_confidence": 0.91,
"_extraction_method": "llm-schema"
}
# Step 2, schedule it as a recurring feed
curl -X POST https://superscraper-production-1381.up.railway.app/v1/feeds \
-H "Authorization: Bearer $SS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://competitor.com/pricing",
"schedule": "0 8 * * 1",
"schema": { "plans": { "type": "array", "items": { ... } } },
"webhook": "https://yourapp.com/webhooks/ci-alert"
}'Frequently asked questions
How do I monitor a competitor pricing page for changes?
POST to /v1/feeds with the URL, a cron schedule, and a JSON schema describing the fields you want (plan name, price, features). Each run extracts the fields and stores the result. You get a webhook or email when a field changes from the previous run.
Can I monitor a whole competitor site, not just one page?
Yes. Use /v1/map to discover all URLs on the domain, then /v1/batch to process up to 100 at a time with a schema. For ongoing monitoring, POST each discovered URL as a separate feed.
What extraction methods does the API use for JavaScript-heavy pricing pages?
The fetch cascade tries plain fetch first, then escalates to Playwright stealth automatically if the page requires JavaScript. The _extraction_method field in the response tells you which method succeeded. You can also force Playwright with "forcePlaywright": true in the request body.
Is this legal for competitive research?
Scraping publicly available web pages for competitive research is broadly accepted practice and is the basis of search engines, price comparison sites, and competitive intelligence tools. You are responsible for complying with each site's terms of service and applicable law. SuperScraper does not bypass authentication and does not access private data.
Start monitoring your competitive landscape
Run any endpoint live in the playground, or grab a free key. 1,000 credits a month, no card.