Documentation Index
Fetch the complete documentation index at: https://docs.skop.dev/llms.txt
Use this file to discover all available pages before exploring further.
Get your API key and make your first request
Follow these simple steps to start extracting documents from websites using natural language prompts.
Step 1: Get your API key
Step 2: Make your first request
Create a scraping job by sending a POST request to the /scrape/ endpoint:
const response = await fetch('https://api.skop.dev/scrape/', {
method: 'POST',
headers: {
'Authorization': 'Bearer sk-your-api-key-here',
'Content-Type': 'application/json'
},
body: JSON.stringify({
website: 'https://example.com',
prompt: 'Find meeting minutes from 2025',
parameters: {
single_page: false,
confidence_threshold: 0.7,
file_type: 'document'
}
})
})
const job = await response.json()
console.log('Job created:', job.job_id)
{
"job_id": "job_4fc79a89797e",
"status": "pending",
"message": "Job created successfully and queued for processing",
"estimated_completion": "2025-07-24T21:00:00Z",
"created_at": "2025-07-24T20:50:00Z"
}
Step 3: Check job status
Poll the job status until completion:
const checkStatus = async (jobId) => {
const response = await fetch(`https://api.skop.dev/scrape/status/${jobId}`, {
headers: {
'Authorization': 'Bearer sk-your-api-key-here'
}
})
const status = await response.json()
console.log('Status:', status.status)
if (status.status === 'completed') {
// Job is done, get results
return getResults(jobId)
} else if (status.is_active) {
// Still running, check again in 5 seconds
setTimeout(() => checkStatus(jobId), 5000)
}
}
Step 4: Get your results
Once the job is completed, retrieve the extracted documents:
const getResults = async (jobId) => {
const response = await fetch(`https://api.skop.dev/scrape/results/${jobId}`, {
headers: {
'Authorization': 'Bearer sk-your-api-key-here'
}
})
const results = await response.json()
console.log(`Found ${results.documents.length} documents`)
return results
}
Complete working example
Here’s a complete function that handles the entire workflow:
const scrapeDocuments = async (website, prompt) => {
const apiKey = 'sk-your-api-key-here'
const baseURL = 'https://api.skop.dev'
try {
// 1. Create job
const jobResponse = await fetch(`${baseURL}/scrape/`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
website,
prompt,
parameters: {
confidence_threshold: 0.7,
file_type: 'document'
}
})
})
const job = await jobResponse.json()
console.log('Job created:', job.job_id)
// 2. Poll for completion
const pollStatus = async () => {
const statusResponse = await fetch(`${baseURL}/scrape/status/${job.job_id}`, {
headers: { 'Authorization': `Bearer ${apiKey}` }
})
const status = await statusResponse.json()
console.log(`Status: ${status.status} (${status.total_documents_found} docs found)`)
if (status.is_active) {
setTimeout(pollStatus, 5000)
} else {
// Get results
const resultsResponse = await fetch(`${baseURL}/scrape/results/${job.job_id}`, {
headers: { 'Authorization': `Bearer ${apiKey}` }
})
const results = await resultsResponse.json()
console.log('Results:', results)
return results
}
}
return pollStatus()
} catch (error) {
console.error('Error:', error)
}
}
// Usage
scrapeDocuments('https://example.com', 'find annual reports from 2024')
.then(results => {
console.log(`Successfully extracted ${results.documents.length} documents`)
})
Next steps
API Reference
Explore all endpoints and parameters
Authentication
Learn about API key formats and security
Error Handling
Handle errors and rate limits properly
Rate Limits
Understand request limits and best practices
