FoxScrape API

v1 Documentation

FoxScrape API Documentation

A powerful web scraping API that allows you to extract data from any website with JavaScript rendering support, screenshots, and advanced data extraction capabilities.

Base URL

https://www.foxscrape.com/api/v1

Key Features

What makes FoxScrape powerful

JavaScript Rendering

Execute JavaScript and wait for dynamic content to load

Screenshots

Capture full page or element screenshots

Data Extraction

Extract structured data using CSS selectors

Header Forwarding

Forward custom headers and cookies

Authentication

FoxScrape API uses API key authentication. Include your API key in the request headers.

API Key

Include your API key in the Authorization header

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://www.foxscrape.com/api/v1?url=https://example.com"

Rate Limits

Current rate limiting information

Free Tier

1,000 requests per month

Free

Pro Tier

100,000 requests per month

$29/month

Basic Usage

Get started with simple web scraping requests.

Simple GET Request

Basic HTML scraping without JavaScript

cURL Example

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://www.foxscrape.com/api/v1?url=https://example.com"

JavaScript Example

const response = await fetch('https://www.foxscrape.com/api/v1?url=https://example.com', {
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY'
  }
});
const html = await response.text();

With JavaScript Rendering

Enable JavaScript execution for dynamic content

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://www.foxscrape.com/api/v1?url=https://example.com&render_js=true"

JSON Response

Get structured response with metadata

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://www.foxscrape.com/api/v1?url=https://example.com&json_response=true"

Parameters

Complete reference of all available parameters.

Core Parameters

Essential parameters for every request

Parameter	Type	Required	Description
`url`	string	Required	The URL to scrape
`render_js`	boolean	Optional	Enable JavaScript rendering
`json_response`	boolean	Optional	Return JSON response with metadata

Browser Options

Control browser behavior and rendering

Parameter	Type	Required	Description
`device`	string	Optional	Device type: desktop or mobile
`window_width`	number	Optional	Browser window width in pixels
`window_height`	number	Optional	Browser window height in pixels
`wait_browser`	string	Optional	Wait condition: load, domcontentloaded, networkidle0, networkidle2
`wait_for`	string	Optional	CSS selector or XPath to wait for
`wait`	number	Optional	Additional wait time in milliseconds
`block_resources`	boolean	Optional	Block images, CSS, and fonts for faster loading
`consent_action`	string	Optional	Handle cookie consent: accept, reject, or off

Content Options

Control what content is returned

Parameter	Type	Required	Description
`return_page_source`	boolean	Optional	Include HTML source in response
`return_page_text`	boolean	Optional	Include plain text in response
`return_page_markdown`	boolean	Optional	Include markdown in response

JavaScript Rendering

Execute JavaScript scenarios to interact with dynamic web pages.

JavaScript Scenarios

Define custom JavaScript interactions

Basic Scenario

{
  "instructions": [
    {"wait": 2000},
    {"click": "#load-more"},
    {"wait_for": ".new-content"}
  ]
}

Advanced Scenario

{
  "instructions": [
    {"click": "#login-button"},
    {"wait_for": "#username"},
    {"fill": ["#username", "myuser@example.com"]},
    {"fill": ["#password", "mypassword"]},
    {"click": "#submit"},
    {"wait_for": ".dashboard"}
  ]
}

Available Actions

All supported JavaScript actions

Parameter	Type	Required	Description
`click`	string	Optional	Click on an element by CSS selector
`wait`	number	Optional	Wait for specified milliseconds
`wait_for`	string	Optional	Wait for element to appear
`wait_for_and_click`	string	Optional	Wait for element and click it
`fill`	array	Optional	Fill input field: [selector, value]
`scroll_x`	number	Optional	Scroll horizontally by pixels
`scroll_y`	number	Optional	Scroll vertically by pixels
`infinite_scroll`	object	Optional	Scroll until no new content loads
`evaluate`	string	Optional	Execute custom JavaScript code

Screenshots

Capture screenshots of web pages or specific elements.

Basic Screenshot

Capture a screenshot of the entire page

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://www.foxscrape.com/api/v1?url=https://example.com&screenshot=true" \
  --output screenshot.png

Screenshot Parameters

Control screenshot behavior

Parameter	Type	Required	Description
`screenshot`	boolean	Optional	Enable screenshot capture
`screenshot_full_page`	boolean	Optional	Capture full page (not just viewport)
`screenshot_selector`	string	Optional	CSS selector for element screenshot
`screenshot_type`	string	Optional	Image format: png, jpeg, or webp

Element Screenshot

Capture screenshot of specific element

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://www.foxscrape.com/api/v1?url=https://example.com&screenshot=true&screenshot_selector=.hero-section" \
  --output element.png

JSON Response with Screenshot

Get screenshot as base64 in JSON response

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://www.foxscrape.com/api/v1?url=https://example.com&screenshot=true&json_response=true"

Data Extraction

Extract structured data from web pages using CSS selectors and XPath.

Basic Extraction

Extract simple data using CSS selectors

Extract Title and Price

{
  "title": "h1",
  "price": ".price"
}

cURL Request

curl -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","extract_rules":{"title":"h1","price":".price"}}' \
  "https://www.foxscrape.com/api/v1"

Advanced Extraction

Extract complex data with attributes and transformations

Extract with Attributes

{
  "title": {
    "selector": "h1",
    "attr": "textContent"
  },
  "image": {
    "selector": "img",
    "attr": "src"
  },
  "price": {
    "selector": ".price",
    "coerce": "number"
  }
}

List Extraction

Extract lists of items

Extract Product List

{
  "products": {
    "type": "list",
    "selector": ".product",
    "item": {
      "title": ".title",
      "price": ".price",
      "image": {
        "selector": "img",
        "attr": "src"
      }
    }
  }
}

Extraction Rules Reference

All available extraction options

Parameter	Type	Required	Description
`selector`	string	Optional	CSS selector or XPath (prefix with 'xpath://')
`attr`	string	Optional	Attribute to extract (default: textContent)
`kind`	string	Optional	Output type: text, html, or attr
`required`	boolean	Optional	Whether field is required
`default`	any	Optional	Default value if extraction fails
`coerce`	string	Optional	Convert to: string, number, or boolean
`normalize`	boolean	Optional	Normalize whitespace

Examples

Real-world examples of common scraping tasks.

E-commerce Product Scraping

Extract product information from an e-commerce site

Extract Rules

{
  "title": "h1.product-title",
  "price": {
    "selector": ".price",
    "coerce": "number"
  },
  "description": ".product-description",
  "images": {
    "type": "list",
    "selector": ".product-images img",
    "item": {
      "selector": "",
      "attr": "src"
    }
  },
  "availability": ".stock-status"
}

Complete Request

curl -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://shop.example.com/product/123",
    "render_js": true,
    "extract_rules": {
      "title": "h1.product-title",
      "price": {"selector": ".price", "coerce": "number"},
      "description": ".product-description",
      "images": {
        "type": "list",
        "selector": ".product-images img",
        "item": {"selector": "", "attr": "src"}
      },
      "availability": ".stock-status"
    }
  }' \
  "https://www.foxscrape.com/api/v1"

Social Media Scraping

Scrape posts from a social media platform

JavaScript Scenario

{
  "instructions": [
    {"wait": 3000},
    {"scroll_y": 1000},
    {"wait": 2000},
    {"scroll_y": 1000},
    {"wait": 2000}
  ]
}

Extract Rules

{
  "posts": {
    "type": "list",
    "selector": ".post",
    "item": {
      "author": ".author-name",
      "content": ".post-content",
      "timestamp": ".timestamp",
      "likes": {
        "selector": ".like-count",
        "coerce": "number"
      }
    }
  }
}

News Article Scraping

Extract article content and metadata

Extract Rules

{
  "headline": "h1.article-title",
  "author": ".author-name",
  "publish_date": ".publish-date",
  "content": ".article-content",
  "tags": {
    "type": "list",
    "selector": ".tags a",
    "item": {"selector": "", "attr": "textContent"}
  },
  "featured_image": {
    "selector": ".featured-image img",
    "attr": "src"
  }
}

Error Handling

Understanding and handling API errors.

HTTP Status Codes

Common response codes and their meanings

Parameter	Type	Required	Description
`200`	number	Optional	Success - Request completed successfully
`400`	number	Optional	Bad Request - Missing or invalid parameters
`401`	number	Optional	Unauthorized - Invalid or missing API key
`402`	number	Optional	Payment Required - Quota exceeded
`429`	number	Optional	Too Many Requests - Rate limit exceeded
`500`	number	Optional	Internal Server Error - Server-side error
`502`	number	Optional	Bad Gateway - Target website unreachable
`504`	number	Optional	Gateway Timeout - Request timeout

Error Response Format

Structure of error responses

{
  "message": "Error description",
  "code": "ERROR_CODE",
  "details": {
    "parameter": "url",
    "reason": "Invalid URL format"
  }
}

Common Error Scenarios

How to handle typical errors

Missing URL

{
  "message": "Missing url",
  "code": "MISSING_PARAMETER"
}

Invalid URL

{
  "message": "Invalid URL format",
  "code": "INVALID_URL"
}

Website Unreachable

{
  "message": "ENOTFOUND example.com",
  "code": "NETWORK_ERROR"
}

Retry Logic

Implementing retry logic for robust scraping

async function scrapeWithRetry(url, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      const response = await fetch(`https://www.foxscrape.com/api/v1?url=${url}`, {
        headers: {
          'Authorization': 'Bearer YOUR_API_KEY'
        }
      });
      
      if (response.ok) {
        return await response.text();
      }
      
      if (response.status === 429) {
        // Rate limited - wait before retry
        await new Promise(resolve => setTimeout(resolve, 1000 * (i + 1)));
        continue;
      }
      
      if (response.status >= 500) {
        // Server error - retry
        continue;
      }
      
      // Client error - don't retry
      throw new Error(`HTTP ${response.status}`);
      
    } catch (error) {
      if (i === maxRetries - 1) throw error;
      await new Promise(resolve => setTimeout(resolve, 1000 * (i + 1)));
    }
  }
}