Workspaces / CLI
Scrape website
Pipeline recommended website scraping via workspace Spider.cloud BYOK. Collect markdown using scrape or bounded crawl. No AI extraction, platform key, or automatic paid analysis. Use public URLs. Returns source pages for table research.
pipeline scrape-websiteDeterministic operationParameters
workspaceIdRequired · string- See the input schema for this field.
credentialIdOptional · string- See the input schema for this field.
urlRequired · string- See the input schema for this field.
modeOptional · string- See the input schema for this field.scrape · crawl
limitOptional · integer- See the input schema for this field.
depthOptional · integer- See the input schema for this field.
pipeline scrape-website \
--workspace-id "00000000-0000-4000-8000-000000000000" \
--url "https://example.com"Full input schema
{
"type": "object",
"properties": {
"workspaceId": {
"type": "string",
"format": "uuid"
},
"credentialId": {
"type": "string",
"format": "uuid"
},
"url": {
"type": "string",
"format": "uri",
"maxLength": 2048
},
"mode": {
"type": "string",
"enum": [
"scrape",
"crawl"
],
"default": "scrape"
},
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 20,
"default": 5
},
"depth": {
"type": "integer",
"minimum": 0,
"maximum": 2,
"default": 1
}
},
"required": [
"workspaceId",
"url"
]
}