crawler
Public
Forked from bakit/crawler
Forked from bakit/crawler
Forked from bakit/crawler
1 Download
1 Download
Last updated
Updated 7 days agoby1 Download
1 Download
Last updated
Updated 7 days agobyProject Files
README
Web search and page crawling tools for LM Studio.
searchWebSearch the web and return the full content of the top results.
Parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
query | string | (required) | Specific search query |
maxResults | number | 20 | Pages to fetch (1–50) |
siteFilter | string | none | Restrict to a domain |
crawlSpecificPageProject Files
README
Web search and page crawling tools for LM Studio.
searchWebSearch the web and return the full content of the top results.
Parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
query | string | (required) | Specific search query |
maxResults | number | 20 | Pages to fetch (1–50) |
siteFilter | string | none | Restrict to a domain |
crawlSpecificPageCrawl a specific URL with advanced extraction options.
Parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
url | string | (required) | Absolute URL to crawl |
include | string[] | ["content"] | Data types: content, rawHtml, links, infoboxes, recentUpdates |
{
"query": "GTA 6 release date 2025",
"maxResults": 10,
"siteFilter": "wikipedia.org"
}
Crawl a specific URL with advanced extraction options.
Parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
url | string | (required) | Absolute URL to crawl |
include | string[] | ["content"] | Data types: content, rawHtml, links, infoboxes, recentUpdates |
{
"query": "GTA 6 release date 2025",
"maxResults": 10,
"siteFilter": "wikipedia.org"
}
{
"url": "https://example.com",
"include": ["content", "links", "infoboxes"]
}
{
"url": "https://example.com",
"include": ["content", "links", "infoboxes"]
}