web-scraping

Name: web-scraping
Author: tatat

Extract structured data from web pages using CSS selectors and XPath

by tatat(0) 0 installs

Free

Free account. Installing gives you the manifest plus copy-paste snippets.

See reviews

About

Imported from tatat/agents-playground (skills/web-scraping/SKILL.md). Install upstream with npx skills add tatat/agents-playground --skill web-scraping. Copyright stays with the author.

Web Scraping

Extract structured data from web pages.

Capabilities

Fetch HTML content from URLs
Parse and extract specific elements (tables, lists, text)
Handle pagination
Output in JSON or CSV format

Supported Selectors

CSS selectors: .class, #id, tag
XPath expressions
Text patterns (regex)

Rate Limiting

Always respect robots.txt and implement delays between requests. Default delay: 1 second between requests.

Example

Scrape product names and prices from example.com/products
Output as JSON with fields: name, price, url

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

yaml

targets:
  - https://api.opensmartroute.ai/api/v1/registry/tatat-agents-playground-web-scraping/manifest   # or paste the manifest below

Manifest

An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.

tatat-agents-playground-web-scraping.ocm.jsonjson

{
  "ocm": "1",
  "id": "tatat-agents-playground-web-scraping",
  "kind": "skill",
  "name": "web-scraping",
  "description": "Extract structured data from web pages using CSS selectors and XPath",
  "publisher": "tatat",
  "version": "1.0.0",
  "capabilities": {
    "domains": [
      "general"
    ],
    "tags": [
      "skill-md",
      "github"
    ],
    "languages": [
      "en"
    ]
  },
  "quality_prior": 0.6,
  "examples": [
    "Extract structured data from web pages using CSS selectors and XPath"
  ],
  "primary": false,
  "metadata": {
    "source": {
      "provider": "github",
      "repository": "https://github.com/tatat/agents-playground",
      "path": "skills/web-scraping/SKILL.md",
      "ref": "0c2aeef5e8e3718edc833a281f8cb5c52ea9cd63",
      "url": "https://github.com/tatat/agents-playground/blob/0c2aeef5e8e3718edc833a281f8cb5c52ea9cd63/skills/web-scraping/SKILL.md",
      "key": "tatat/agents-playground/skills/web-scraping/SKILL.md"
    }
  },
  "instructions": "# Web Scraping\n\nExtract structured data from web pages.\n\n## Capabilities\n\n- Fetch HTML content from URLs\n- Parse and extract specific elements (tables, lists, text)\n- Handle pagination\n- Output in JSON or CSV format\n\n## Supported Selectors\n\n- CSS selectors: `.class`, `#id`, `tag`\n- XPath expressions\n- Text patterns (regex)\n\n## Rate Limiting\n\nAlways respect robots.txt and implement delays between requests.\nDefault delay: 1 second between requests.\n\n## Example\n\n```\nScrape product names and prices from example.com/products\nOutput as JSON with fields: name, price, url\n```",
  "cost": {
    "context_tokens": 143
  }
}

Fetch it by URL: GET /api/v1/registry/tatat-agents-playground-web-scraping/manifest?version=1.0.0