How to Conduct Bulk Domain Analysis with SE Ranking’s API


Analyzing thousands of domains for SEO insights can be tiresome. It often requires significant time and resources, but you can use API technology to make the process of conducting bulk domain checks more accessible and efficient. This makes it easier to analyze competitive niches and find better guest blogging opportunities. In this article, we will explore how SE Ranking’s API empowers users to perform comprehensive domain analysis at scale. You’ll discover how this translates to valuable insights that can greatly streamline your workflow and drive better results for your SEO projects. We will also look at the different ways you can integrate this API into your SEO routine.

Let’s jump right in!

TL;DR

To get domain data for multiple websites, you can utilize our API, specifically the Competitive Research and Backlink Checker API. These APIs provide comprehensive information, such as organic traffic, keyword count, domain trust score, referring domains, backlinks, and more. What’s even better, you don’t need coding skills to utilize these APIs.

To get domain analysis data with Python, you must run a code in Google Colab. Once executed, the results will be displayed in Google Sheets. To view each metric individually, apply a specific formula in the table. You can select the metrics that you need. 

All of this data will help you conduct quick competitive and niche analysis, assess the backlink profile of every domain, or perform other types of SEO analysis.

An overview of APIs: what are they and how do they work?

The SE Ranking team has created several APIs. They were designed to simplify your work, both with large-scale domain data analytics and other SEO tasks. Now you can retrieve raw data without having to manually log into the SE Ranking platform. This is ideal for companies that manage tons of data and accounts (i.e., SEO agencies).

This API is available with a Business subscription only.

To analyze domains in bulk with SE Ranking, use the following two APIs: Competitive Research and Backlink Checker

The Competitor Research API presents a convenient format for accessing domain statistics from both organic and paid search results, including traffic, keywords, and other metrics. Meanwhile, the Backlink Checker API provides data that you can use to conduct backlink profile analyses. Both of these APIs combined offer nearly all the numeric metrics available on the platform versions of these tools.

Here are some of the key metrics available.

Competitive Research API:

  • Total domain traffic within the specified location
  • Number of keywords a domain ranks for in the specified location
  • Number of newly acquired keywords
  • Number of keywords that have dropped from the SERPs
  • Number of keywords with consistent ranking positions
  • Number of keywords with improved ranking positions
  • Number of keywords with decreased ranking positions
  • Total number of keywords ranked in positions 1-5, 6-10, and 11-20
  • And more

Backlink Checker API:

  • Domain Trust score
  • Total number of backlinks for the URL
  • Total number of referring domains for the URL
  • Total number of dofollow/nofollow backlinks
  • And more

This isn’t the complete list. You can find all parameters in our Backlink Checker API and Competitor Research API documents.

How to optimize your workflow with our API 

You don’t need to know how to code to fully utilize our APIs. Simply run the provided Python code and access the results through Google Sheets. 

Let’s begin by taking a closer look at the codes you will be working with.

Available codes

You can conduct a domain analysis using one of the following two scripts.

1. [Competitive Research + Backlink Checker API] 

Pros: You get more data from two modules at once.

Cons: You still expend Backlink Checker credits. Due to the large amount of data to process, the code operates slower.

You will need to copy and paste the code below into Google Colab. But please note that you must add your API key to the api_key parameter. This can be found within the API section of SE Ranking’s Settings

API key within the API section of SE Ranking’s Settings

Competitive Research + Backlink Checker API code:

import requests
import gspread
from google.colab import auth, drive
from oauth2client.client import GoogleCredentials
import pandas as pd
from gspread_dataframe import set_with_dataframe
from google.auth import default
auth.authenticate_user()
creds, _ = default()


def read_table(table_name:str) -> pd.DataFrame:
gc = gspread.authorize(creds)
worksheet = gc.open(table_name).sheet1
rows = worksheet.get_all_values()
frame = pd.DataFrame.from_records(rows)
frame.columns = frame.iloc[0]
frame = frame.drop(index=[0])
frame = frame.fillna('')
return frame


def save_table(table_name:str, dataframe:pd.DataFrame) -> None:
  gc = gspread.authorize(creds)
  worksheet = gc.open(table_name).sheet1
  set_with_dataframe(worksheet, dataframe)


def request_to_api(research:pd.DataFrame) -> list:
   result = list()
   for index, row in research.iterrows():
       domain = row['domain']
       research_result = get_research_api_result(domain)
       backlinks_result = get_backlinks_api_result(domain) if (row['backlinks'] == '') else row['backlinks']
       result.append([domain, research_result, backlinks_result])
   return result


def get_api_headers() -> str:
   api_key = 'ADD YOUR API KEY'
   return dict(Authorization=api_key)


def get_research_api_result(domain:str) -> str:
   params = dict(domain=domain)
   api_url="https://api4.seranking.com/research/uk/overview/?"
   result = requests.get(api_url, params=params, headers=get_api_headers())
   return result.json()


def get_backlinks_api_result(domain:str) -> str:
   report = create_backlink_checker_report(domain)
   report_id = str(report.get('report_id'))
   api_url="https://api4.seranking.com/backlink-reports/" + report_id + '/overview'
   result = requests.get(api_url, headers=get_api_headers())
   delete_backlink_checker_report(report_id)
   return result.json()


def create_backlink_checker_report(domain:str) -> str:
   params = dict(mode="domain", target=domain)
   api_url="https://api4.seranking.com/backlink-reports"
   result = requests.post(api_url, json=params, headers=get_api_headers())
   return result.json()


def delete_backlink_checker_report(report_id:str) -> None:
   api_url="https://api4.seranking.com/backlink-reports/" + report_id
   requests.delete(api_url, headers=get_api_headers())


def main():
   table_name="competitive-research-API"
   table = read_table(table_name)
   result_table_requests_to_api = request_to_api(table)
   result_table_to_dataframe = pd.DataFrame(result_table_requests_to_api, columns=['domain', 'result', 'backlinks'])
   save_table(table_name, result_table_to_dataframe)
main()

2. [Competitive Research API] 

Pros: The script operates faster, and you don’t use up credits.

Cons: You acquire less data and do not receive information on domain trust (domain authority), number of backlinks, referring domains, etc.

Competitive Research code:

Don’t forget to input your API key into the api_key parameter.

import requests
import gspread
from google.colab import auth, drive
from oauth2client.client import GoogleCredentials
import pandas as pd
from gspread_dataframe import set_with_dataframe
from google.auth import default
auth.authenticate_user()
creds, _ = default()


def read_table(table_name:str) -> pd.DataFrame:
gc = gspread.authorize(creds)
worksheet = gc.open(table_name).sheet1
rows = worksheet.get_all_values()
frame = pd.DataFrame.from_records(rows)
frame.columns = frame.iloc[0]
frame = frame.drop(index=[0])
frame = frame.fillna('')
return frame


def save_table(table_name:str, dataframe:pd.DataFrame) -> None:
  gc = gspread.authorize(creds)
  worksheet = gc.open(table_name).sheet1
  set_with_dataframe(worksheet, dataframe)


def request_to_api(research:pd.DataFrame) -> list:
   result = list()
   for index, row in research.iterrows():
       domain = row['domain']
       research_result = get_research_api_result(domain)
       result.append([domain, research_result])
   return result


def get_api_headers() -> str:
   api_key = 'ADD YOUR API KEY'
   return dict(Authorization=api_key)


def get_research_api_result(domain:str) -> str:
  params = dict(domain=domain)
  api_url="https://api4.seranking.com/research/uk/overview/?"
  result = requests.get(api_url, params=params, headers=get_api_headers())
  return result.json()


def main():
  table_name="competitive-research-API"
  table = read_table(table_name)
  result_table_requests_to_api = request_to_api(table)
  result_table_to_dataframe = pd.DataFrame(result_table_requests_to_api, columns=['domain', 'result'])
  save_table(table_name, result_table_to_dataframe)
main()

Here are a few key parameters to pay attention to:

  • This is how your API key appears in the code:
API key in a code
  • In api_url, you can change the location from which to retrieve traffic/keyword data. 
api_url parameter in a code
  • These are the columns in your Google Sheets file.
Name of columns in Google Sheets file

Now, let’s move on to our step-by-step guide to utilizing these codes.

How to use it

Step 1. Open Google Colab, create a new notebook and paste one of the two codes provided in the previous section.

Google Colaboratory
Competitive Research API script

Step 2. Open the Colab Notebooks folder in your Google Drive and create a Google Sheet file.

Colab Notebooks folder in Google Drive

Note: Ensure that the file name is an exact match with the table_name in the code, as shown in the example below.

Table name in a code
Table name for API

Step 3. Assign names to the columns based on the script being used. If using the Competitive Research API script exclusively, columns A and B should be called domain and result, respectfully. For the Competitive Research and Backlink Checker API scripts, in addition to the domain and result columns, you need to add a third column called backlinks

See the screenshot below showing the second version with three columns. 

If necessary, you can rename the columns and update the code accordingly.

Then, enter your domains in the first column without http:// and www.

List of domains to analyze with API

Step 4: Go to Colab, click the button to run the code, and grant access if required.

Running code in Google Colab

Step 5: After the script finishes running, check back into the Google Sheets file at a later time to get the results. Here’s what the table will look like:

Table with API data

Step 6. To isolate the metric you specified, extract it from the JSON data. Just input the =REGEXEXTRACT function or come up with your own solution. Below is the formula for accessing the traffic_sum parameter, which represents the total organic traffic in a specified location:

=REGEXEXTRACT(B2,”traffic_sum’:\s([0-9]+)”)

  • B2 is the column containing your JSON data (i.e., your result column). If you need data from another column, you can change it in the formula.
  • traffic_sum is the metric of interest. If you want to retrieve other data, you can change this parameter in the formula. 
  • \s([0-9]+) is the regular expression that searches for sequences where the whitespace character is immediately followed by one or more digits. You don’t need to change it. 

Step 7. Once you have entered the formula into the first cell, drag it down to copy the formula to all cells. Be sure to obtain data from each domain on the list.

=REGEXEXTRACT function

Here is a list of key metrics that you can use in formulas to obtain important domain data:

  • keywords_count — Total number of keywords a domain ranks for in the specified location.
  • top1_5 — Total number of keywords that rank in the top 1-5.
  • keywords_new_count — Total number of new keywords.
  • domain_inlink_rank — Domain Trust score.
  • total_domains — Total number of referring domains for a URL.
  • total_backlinks — Total number of backlinks for a URL.

You can find each of these metrics in JSON format in our Backlink Checker API (for the backlinks column) and Competitor Research API (for the result column) documents.

Exploring API use cases

Wondering how can this data benefit you? Let’s explore some areas in SEO where you can benefit from API.

  • Competitive and niche analysis

You can assess the strength of your competitors (or domains in your niche) based on factors such as organic traffic, number of keywords, backlinks, and domain trust. This helps you gauge the overall competitiveness of the niche and where you stand compared to these websites.

To expand your domain list and find more websites from within your niche, you can get SERP results for the keywords in your niche. Just use SE Ranking’s SERP Competitors feature under the My Competitors tab to get this information. 

Here’s how:

  1. Select the desired keyword group from the dropdown menu (ensure that you have previously added all necessary keywords to a single group within the project).
  2. Choose how many results from the SERP you need (100/50/30/20/10).
  3. Select the Domain mode.
SE Ranking's SERP Competitors feature under the My Competitors tab

Then, export data. Select the All keywords from the group option to obtain a list of domains that rank in the top 10-100 for the keywords you want to examine.

Export SERP data from SE Ranking

After you have chosen the websites you want to examine, add them to the Google Sheets file and conduct a domain SEO analysis. It will compare the organic traffic, number of keywords, backlinks, referring domains, and domain trust scores of these sites. This can provide valuable insights into your niche’s competitive landscape.

Domain SEO analysis with SE Ranking's API

To conduct an in-depth competitive analysis, use specialized tools like SE Ranking’s Competitive Research and Backlink Checker.

  • Guest blogging and digital PR

You can also streamline your link building efforts by pinpointing the most relevant domains to contact for potential collaboration opportunities. Conducting a quick initial research study with API helps you evaluate domain quality before committing to investing in further resources. 

For instance, you can assess their organic traffic and Domain Trust scores to pinpoint potential sources for deeper analysis and further collaboration outreach. While this approach may not offer insights into site relevance and other important qualitative metrics, it is a good starting point for your campaign. 

Total traffic and Domain Trust for domains

Wrapping up

SE Ranking’s API functionality is an ideal solution for businesses managing multiple large-scale projects at once. It smoothens the process of analyzing huge datasets and automating SEO processes.

We hope that the API codes and guide we provided help simplify your bulk domain analysis efforts. We designed our APIs for everyone, even those without coding skills.



Source link

Leave a Comment

Your email address will not be published. Required fields are marked *