API Documentation

Welcome to urlDNA API documentation.

Before you start using the urlDNA APIs, create a FREE account to obtain your API key. If you already have an account, you can find your API key on your profile page.

Python package

Install

                
            

Basic usage

                
            

Quota Limits

The free plan allows you to make up to 100 API requests daily for search and show scan functionality. You can create unlimited new scans!

To increase your daily API quota limit, please contact us.

User agents

This section explains how to retrieve a list of available user agents that you can specify when submitting a new scan. User agents identify the software (browser and operating system) used to access the webpage during the scan.

Request

                
            

Response attributes

The following table details the attributes returned by the API when you request a list of user agents:

AttributeDescription
browserThe primary browser or operating system identified by the user agent string.
deviceDevice type (DESKTOP or MOBILE) associated with the user agent.
shareScore indicating the relative usage share of this user agent among all available options.
user_agentThe complete HTTP user agent string.

Response

                
            

Viewports

This section explains how to retrieve a list of available viewports that you can specify when submitting a new scan. Viewports define the screen size and resolution used for scraping the webpage.

Request

                
            

Response attributes

The following table details the attributes returned by the API when you request a list of viewports:

AttributeDescription
deviceDevice type (DESKTOP or MOBILE) for the viewport.
widthScreen width of the viewport in pixels.
heightScreen height of the viewport in pixels.

Response

                
            

Scan

This section explains how to retrieve information about a specific scan using the urlDNA GET scan API endpoint.

Request

                
            

Response attributes

This section details the attributes returned by the GET scan API endpoint when you retrieve information about a specific scan.

AttributeDescription
scanContains attributes submitted during the scan (e.g., URL, device) and extracted URL attributes (domain, protocol).
certificateInformation about the website's SSL/TLS certificate (if applicable).
console_messagesList of all console messages encountered during the scan.
cookiesList of all cookies found on the scanned webpage, including name, value, and security information.
domThe complete HTML source code retrieved from the scanned webpage.
faviconInformation about the website's favicon, including its hash and a downloadable URL (if available).
http_transactionsList of all HTTP transactions recorded during the scan.
ip_addressDetailed information about the IP address associated with the scanned URL.
maliciousIndicates if the URL may be malicious (use with caution as this is a beta feature).
pageContains meta tags, headers, and plain text content extracted from the scanned webpage.
screenshotURL for downloading a screenshot image of the scanned webpage.
technologiesList of all technologies detected on the scanned webpage.

Response

                
            

New scan

Scan new URL.

  1. Submit a New Scan: Provide the URL you want to scan and any desired options. The URL will be pushed to a processing queue.
  2. Retrieve Scan Status: Once submitted, use the GET scan API endpoint with the scan ID you receive to check the scan status.
  3. View Scan Results: When the status is "DONE" the scan is complete, and you can access all the extracted attributes. This process might take a few minutes depending on system load.

Request attributes

The following table details the attributes you can specify when submitting a new scan:

AttributeDescriptionRequiredDefault
urlThe URL you want to scan (must include HTTP or HTTPS)Yes
deviceDevice type for scraping (DESKTOP or MOBILE).NoDESKTOP
user_agentUser agent string to use for scraping.NoMozilla/5.0 (Windows NT 10.0;Win64;x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36
widthScreen width for scraping (use valid viewports).No1920
heightScreen height for scraping (use valid viewports).No1080
waiting_timeSeconds to wait after the page loads (3-15 seconds).No5
private_scanSet scan visibility to private (you only see results).Nofalse

Request

                
            

Response attributes

The following table details the attributes returned by the GET scan API endpoint when you retrieve scan information:

AttributeDescription
idUnique identifier for the scan
status Current status of the scan:
  • PENDING: Scan is queued for processing.
  • RUNNING: Scan is currently being processed.
  • DONE: Scan is complete and results are ready.
  • ERROR: An error occurred during processing.
submitted_dateDate and time the scan was submitted (UTC format).

Response

                
            

Search

This section explains how to search for existing scans using the urlDNA API.

Search types

  • Domain: You can search for scans by domains.
  • Custom Query Language (CQL): For more advanced searches, you can leverage the CQL syntax (details available in the search section within the urlDNA interface).

Request attributes

The following table details the required attribute for search requests:

AttributeDescriptionRequiredDefault
querySearch text or CQL expression.Yes

Request

                
            

Response attributes

The following table details the attributes returned by the SEARCH endpoint:

AttributeDescription
deviceDevice used for scanning (DESKTOP or MOBILE).
domainDomain name of the scanned URL.
heightScreen height used during scanning.
idUnique scan ID.
originWhether the scan originated from the API or the urlDNA interface.
private_scanWhether the scan was private.
protocolScanned URL protocol (HTTP or HTTPS).
statusScan status (DONE).
submitted_dateDate and time the scan was submitted.
submitted_urlThe original URL submitted for scanning.
target_urlThe final URL scanned (after redirects, if any).
user_agentUser agent string used during scraping.
widthScreen width used during scanning.

Response