Workspaces / REST API
Scrape website
Pipeline recommended website scraping via workspace Spider.cloud BYOK. Collect markdown using scrape or bounded crawl. No AI extraction, platform key, or automatic paid analysis. Use public URLs. Returns source pages for table research.
POST /api/v1/research/scrape-websiteDeterministic operationParameters
workspaceIdRequired · string- See the input schema for this field.
credentialIdOptional · string- See the input schema for this field.
urlRequired · string- See the input schema for this field.
modeOptional · string- See the input schema for this field.scrape · crawl
limitOptional · integer- See the input schema for this field.
depthOptional · integer- See the input schema for this field.
curl -X POST "https://app.pipeline.help/api/v1/research/scrape-website" \
-H "Authorization: Bearer $PIPELINE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"workspaceId":"00000000-0000-4000-8000-000000000000","url":"https://example.com"}'Full input schema
{
"type": "object",
"properties": {
"workspaceId": {
"type": "string",
"format": "uuid"
},
"credentialId": {
"type": "string",
"format": "uuid"
},
"url": {
"type": "string",
"format": "uri",
"maxLength": 2048
},
"mode": {
"type": "string",
"enum": [
"scrape",
"crawl"
],
"default": "scrape"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 20,
"default": 5
},
"depth": {
"type": "integer",
"minimum": 0,
"maximum": 2,
"default": 1
}
},
"required": [
"workspaceId",
"url"
]
}