AI Parser (Closed Beta)

Overview

The AI Parser helps you to turn any website into clean, structured data automatically.

By providing example URLs, the service creates intelligent parsers that can extract structured data from similar pages. This documentation describes all available endpoints and their usage.

Base URL: https://aiparser.scraperapi.com

Authentication

All endpoints require an API key to be provided either in the request body or as a query parameter.

Endpoints

Create a Parser

Creates a new parser based on example URLs.

POST /parsers

Example Request Body:

{
  "name": "Example_Name",
  "api_key": "your_scraperapi_key",
  "urls": [
    "https://example.com/product-01"
  ]
}

Response:

{
    "id": "parser_id",
    "version": 0
}

Optional Fields:

  "scraper_params": {
    "render": true,
    "country_code": "US",
    "premium": true,
    "session_number": 1,
    "keep_headers": true,
    "device_type": "desktop",
    "ultra_premium": false,
    "follow_redirect": true,
    "retry_404": false
  },
  "fields": [
    {
      "name": "products.name",
      "description": "Product name",
      "type": "string",
      "selector": "CSS or XPath (optional)"
    },
    {
      "name": "price",
      "description": "Product price",
      "type": "number"
    }
  ]

• Maximum of 3 URLs allowed.

• More URLs of the same type improve parser quality.

• All URLs must be valid and accessible.

• All scraper_params fields are optional.

• Optional: fields lets you predefine desired fields ( name , description , optional type and selector) to guide generation.

• You can use dot notation for nested fields, e.g., "products.name".

The version field in the response plays a role to identify different versions of a parser. To create further versions you can use the Update Parser Fields endpoint.

Parse a URL

Uses an existing parser to extract data from a URL.

GET /parse/{parser_id}/{version}?
api_key=your_scraperapi_key&url=https://example.com/product-02

Path Parameters:

• version (optional): Parser version to use. Defaults to latest if omitted.

Query Parameters:

• api_key - Your ScraperAPI Key.

• url - The URL to parse.

• Additional ScraperAPI parameters can be added as query parameters. List of supported params can be found here.

Response:

{
    "parser": "<parser_id>",
    "version": 0,
    "result": {
        "products": [
	    {
                "name": "Product 02",
		"part_number": "ABCDEF-0002",
		"brand": "Example Product 02",
	        "price": 1212.02,
		"images": [
		    {
		        "url": "https://example.com/product02/img01"
		    },
		    {
		        "url": "https://example.com/product02/img02"
		    }
	        ],
	        "stock": 81,
	        "rating": "4 Stars"
	    },
	    {
                "name": "Product 03",
		"part_number": "ABCDEF-0002",
		"brand": "Example Product 03",
	        "price": 3331.3,
		"images": [
		    {
		        "url": "https://example.com/product03/img01"
		    },
		    {
		        "url": "https://example.com/product03/img02"
		    }
	        ],
	        "stock": 12,
	        "rating": "5 Stars"
	    }
	]
    }
}

• The parser must exist and be valid.

List Parsers

Retrieves a list of all parsers for your account.

GET /parsers?api_key=your_scraperapi_key

Response:

[
    {
        "id": "parser_id",
        "name": "Product Parser",
        "status": "FINISHED",
        "generated_at": "2025-09-20T10:00:00Z",
        "version": 5
    }
]

Get Parser Details

Retrieves detailed information about a specific parser.

GET /parsers/{parser_id}/{version}?api_key=your_scraperapi_key

Path Parameters:

• version (optional): Parser version to retrieve. Defaults to latest if omitted

Response:

{
  "id": "parser_id",
  "name": "Product Parser",
  "created_at": "2025-03-20T10:00:00Z",
  "generated_at": "2025-03-20T10:10:00Z",
  "error": null,
  "example_results": [
    {
      "url": "https://example.com/product01",
      "result": {
      "products": [
        {
          "name": "Product 01",
          "part_number": "ABCDEF-0001",
	  "brand": "Example Product 01",
	  "price": 111.11,
	  "images": [
	    {
	      "url": "https://example.com/product02/img01"
	    },
            {
              "url": "https://example.com/product02/img02"
            }
	  ],
	  "stock": 34,
	  "rating": "4 Stars"
	}
      ]
    }
  ],
  "fields": [
    {
      "name": "products",
      "type": "array"
    },    
    {
      "name": "products.name",
      "type": "stirng"
    },    
    {
      "name": "products.part_number",
      "type": "string"
    },
    {
      "name": "products.brand",
      "type": "string"
    },
    {
      "name": "products.price",
      "type": "number"
    },
    {
      "name": "products.images",
      "type": "array"
    },
    {
      "name": "products.images.url",
      "type": "string"
    },
    {
      "name": "products.stock",
      "type": "number"
    },
    {
      "name": "products.reating",
      "type": "string"
    },
    
  ],
  "version": 2
}

Delete Parser

Deletes a specific parser.

DELETE /parsers/{parser_id}?api_key=your_scraperapi_key

Response:

• 204 No Content on success - validates that the parser has been deleted.

Update Parser Fields

Update, rename, or remove fields from a parser. All modifications create a new parser version. Some changes require processing time, while others apply immediately.

PATCH /parsers/{parser_id}/{version}?

Path Parameters:

• version (optional): Base version to update. Defaults to latest if omitted

Request Body:

{
  "api_key": "your_scraperapi_key",
  "add_fields": [
    {
      "name": "products.name",
      "description": "Name of the product",
      "type": "string"
    }
  ],
  "modify_fields": [
    {
      "name": "price",
      "description": "Price in USD",
      "type": "number"
    }
  ],
  "rename_fields": [
    {
      "name": "products.name",
      "new_name": "products.title"
    }
  ],
  "remove_fields": [
    "obsolete_field"
  ]
}

Response:

• If processing time is required (when add_fields or modify_fields are present) you will get a 202 Accepted response with the newly created version and schedules generation.

• If only synchronous changes are requested (rename_fields and/or remove_fields) you will get 200 OK response with the newly created version. Changes apply immediately

• Only api_key is required; all other properties are optional.

• Field objects support name , description , optional type ( string , number , array ) and optional selector.

• Use dot notation for nested fields (e.g., "products.name" ).

ScraperAPI Parameters

The following parameters can be used to customize the scraping behavior

Parameter	Type	Description
render	Boolean	Enable JavaScript rendering
country_code	String	Two-letter country code (e.g., “US”)
premium	Boolean	Use residential proxies
session_number	Integer	Session number for maintaining IPs
keep_headers	Boolean	Preserve custom headers
device_type	String	“desktop” or “mobile”
ultra_premium	Boolean	Use advanced unblocking mechanisms.
follow_redirect	Boolean	Follow HTTP redirects. True by default.
retry_404	Boolean	Retry on 404 errors

Best Practices

1. URL Selection

• Provide multiple URLs (up to 3) of the same type.

• Ensure URLs are accessible.

• Include URLs with different variations of the same content.

1. Parser Usage

• If results are not satisfactory, retry generating either with the same parameters or try modifying the parameters or use the newly introduced PATCH /parsers endpoint.

• Different scrapes may yield different results due to website variations.

• Use appropriate ScraperAPI parameters (where necessary) based on the target website.

1. Error Handling

• Always check for error responses.

• Handle rate limits and timeouts appropriately.

• Monitor parser performance and adjust parameters as needed.

Rate Limits

Please refer to your ScraperAPI plan for specific rate limits. The AI Parser service inherits these limits from your ScraperAPI account.

Support

For support or questions about the AI Parser service, please contact [e-mail] or [e-mail].

Credits & Usage Limits

Each tester account is allocated a certain number of parser creation and usage credits. If your credits are exhausted, requests will return an HTTP 402 Payment Required status.

To continue using the parser, you can request additional credits by reaching out to us at [e-mail].

Updates

1. 2025-09-01

• Added optional version path parameter to GET /parse/{parser_id}/{version}? and GET /parsers/{parser_id}/{version}? . If omitted, latest version is used.

• Introduced PATCH /parsers/{parser_id}/{version}? to add/modify/rename/remove fields. Adding/modifying fields creates a new version; renaming/removing applies immediately.

• POST /parsers now accepts optional fields to guide parser generation. Supports dot notation for nested fields like "products.name" .

• Optional parameters are explicitly marked in the docs.

Tried AI Parser?

Take this 2 mins survey to help us improve our upcoming releases.