API Documentation
Welcome to urlDNA API documentation.
Before you start using the urlDNA APIs, create a FREE account to obtain your API key. If you already have an account, you can find your API key on your profile page.
Python package
Install
Basic usage
- Python Package Documentation: You can find detailed documentation for the Python package on PyPI: https://pypi.org/project/urlDNA/
- GitHub Repository: https://github.com/urldna/urldna
Quota Limits
The free plan allows you to make up to 100 API requests daily for create, search and show scan functionality.
To increase your daily API quota limit, please contact us.
User agents
This section explains how to retrieve a list of available user agents that you can specify when submitting a new scan. User agents identify the software (browser and operating system) used to access the webpage during the scan.
Request
Response attributes
The following table details the attributes returned by the API when you request a list of user agents:
| Attribute | Description |
|---|---|
| browser | The primary browser or operating system identified by the user agent string. |
| device | Device type (DESKTOP or MOBILE) associated with the user agent. |
| share | Score indicating the relative usage share of this user agent among all available options. |
| user_agent | The complete HTTP user agent string. |
Response
Viewports
This section explains how to retrieve a list of available viewports that you can specify when submitting a new scan. Viewports define the screen size and resolution used for scraping the webpage.
Request
Response attributes
The following table details the attributes returned by the API when you request a list of viewports:
| Attribute | Description |
|---|---|
| device | Device type (DESKTOP or MOBILE) for the viewport. |
| width | Screen width of the viewport in pixels. |
| height | Screen height of the viewport in pixels. |
Response
Scan
This section explains how to retrieve information about a specific scan using the urlDNA GET scan API endpoint.
Request
Response attributes
This section details the attributes returned by the GET scan API endpoint when you retrieve information about a specific scan.
| Attribute | Description |
|---|---|
| scan | Contains attributes submitted during the scan (e.g., URL, device) and extracted URL attributes (domain, protocol). |
| certificate | Information about the website's SSL/TLS certificate (if applicable). |
| console_messages | List of all console messages encountered during the scan. |
| cookies | List of all cookies found on the scanned webpage, including name, value, and security information. |
| dom | The complete HTML source code retrieved from the scanned webpage. |
| favicon | Information about the website's favicon, including its hash and a downloadable URL (if available). |
| http_transactions | List of all HTTP transactions recorded during the scan. |
| ip_address | Detailed information about the IP address associated with the scanned URL. |
| malicious | Indicates if the URL may be malicious (use with caution as this is a beta feature). |
| page | Contains meta tags, headers, and plain text content extracted from the scanned webpage. |
| screenshot | URL for downloading a screenshot image of the scanned webpage. |
| technologies | List of all technologies detected on the scanned webpage. |
Response
New scan
Scan new URL.
- Submit a New Scan: Provide the URL you want to scan and any desired options. The URL will be pushed to a processing queue.
- Retrieve Scan Status: Once submitted, use the GET scan API endpoint with the scan ID you receive to check the scan status.
- View Scan Results: When the status is "DONE" the scan is complete, and you can access all the extracted attributes. This process might take a few minutes depending on system load.
Request attributes
The following table details the attributes you can specify when submitting a new scan:
| Attribute | Description | Required | Default |
|---|---|---|---|
| url | The URL you want to scan (must include HTTP or HTTPS) | Yes | |
| device | Device type for scraping (DESKTOP or MOBILE). | No | DESKTOP |
| user_agent | User agent string to use for scraping. | No | Mozilla/5.0 (Windows NT 10.0;Win64;x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36 |
| width | Screen width for scraping (use valid viewports). | No | 1920 |
| height | Screen height for scraping (use valid viewports). | No | 1080 |
| waiting_time | Seconds to wait after the page loads (3-15 seconds). | No | 5 |
| private_scan | Set scan visibility to private (you only see results). | No | false |
Request
Response attributes
The following table details the attributes returned by the GET scan API endpoint when you retrieve scan information:
| Attribute | Description |
|---|---|
| id | Unique identifier for the scan |
| status | Current status of the scan:
|
| submitted_date | Date and time the scan was submitted (UTC format). |
Response
Search
This section explains how to search for existing scans using the urlDNA API.
Search types
- Domain: You can search for scans by domains.
- Custom Query Language (CQL): For more advanced searches, you can leverage the CQL syntax (details available in the search section within the urlDNA interface).
Request attributes
The following table details the required attribute for search requests:
| Attribute | Description | Required | Default |
|---|---|---|---|
| query | Search text or CQL expression. | Yes |
Request
Response attributes
The following table details the attributes returned by the SEARCH endpoint:
| Attribute | Description |
|---|---|
| device | Device used for scanning (DESKTOP or MOBILE). |
| domain | Domain name of the scanned URL. |
| height | Screen height used during scanning. |
| id | Unique scan ID. |
| origin | Whether the scan originated from the API or the urlDNA interface. |
| private_scan | Whether the scan was private. |
| protocol | Scanned URL protocol (HTTP or HTTPS). |
| status | Scan status (DONE). |
| submitted_date | Date and time the scan was submitted. |
| submitted_url | The original URL submitted for scanning. |
| target_url | The final URL scanned (after redirects, if any). |
| user_agent | User agent string used during scraping. |
| width | Screen width used during scanning. |
Response