ScrapingMap
Map Endpoint
Map (URL Discovery)
Discover all URLs on a website via sitemaps, robots.txt, and HTML link extraction. Returns a deduplicated, filtered list optionally ranked by relevance.
POST
/v1/mapDiscover URLs from a website using multiple discovery strategies.
Request Body
| Parameter | Type | Description |
|---|---|---|
urlrequired | string | Base URL to discover links from |
limit | number | Max number of URLs to return (1-5000). Defaults to 100 |
search | string | Keyword to filter and rank discovered URLs by relevance. |
includeSubdomains | boolean | Include URLs from subdomains (e.g., blog.example.com when base is example.com). Defaults to false |
ignoreQueryParameters | boolean | Strip query parameters from URLs for deduplication. Defaults to true |
Example Request
Example Response
Response Structure
| Field | Type | Description |
|---|---|---|
success | boolean | Whether URL discovery succeeded |
links | string[] | Discovered URLs |
total | number | Total number of URLs returned (equal to links array length) |
cost | number | Cost in USD ($0.002 per request) |
error | string | Error message if discovery failed |
timestamp | string | ISO timestamp of the response |
Map + Fetch Pattern — SDK Only
A common pattern is to discover URLs with map, then fetch their content. The Node.js SDK makes this easy:
Pricing
Flat rate of
$$0.002 per request, regardless of how many URLs are discovered.