cliProvider-backed

Scrape website

pipeline scrape-website

Pipeline recommended website scraping via workspace Spider.cloud BYOK. Collect markdown using scrape or bounded crawl. No AI extraction, platform key, or automatic paid analysis. Use public URLs. Returns source pages for table research.

Parameters

workspaceIdrequired

string

No description.

credentialId

string

No description.

urlrequired

string

No description.

mode

string

No description.

scrape | crawl

limit

integer

No description.

depth

integer

No description.

Example

CLI commandbash

pipeline scrape-website \
  --workspace-id "00000000-0000-4000-8000-000000000000" \
  --url "https://example.com"

Response

Success response schemajson schema

{
  "type": "object",
  "properties": {
    "success": {
      "type": "boolean"
    },
    "provider": {
      "type": "string",
      "enum": [
        "spider"
      ]
    },
    "pages": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "url": {
            "type": "string"
          },
          "content": {
            "type": "string"
          }
        }
      }
    }
  },
  "required": [
    "success",
    "provider",
    "pages"
  ]
}

Exit codes

1: Invalid input or local configuration.
2: The API or tool returned an error.
3: Authentication failed.
4: A network request failed.