Command · Web & Custom Endpoints
Basic Website Scrape
Crawl a public website without any third-party service. Combines pages into a single Markdown file in a project folder.
ImportAuth: None
How it works
Basic Website Scrape fetches public pages with no credentials at all and stores the text as Markdown. Use it for small, static sites.
01
Before you start
- A publicly reachable URL that does not require login or JavaScript rendering
02
What you need to make the connection
| Field | Where to get it |
|---|---|
| None | No API key or account is required. |
Credentials are stored encrypted in raia Command and are never shown again after saving. Rotate them at the provider if they are exposed.
03
Setup steps
- 01
In raia Command, open Settings → Connectors, find the connector, and click Connect.
- 02
Enter the URL and, optionally, a crawl depth.
- 03
Pick the destination folder and run the import.
- 04
Choose the destination project folder, run the first import, then set a sync schedule. Every run is recorded in Sync History.
04
Good to know
- JavaScript-rendered or login-gated pages will come back empty — use Firecrawl instead.
05