Interesting Insights from Job Board Data Analyses


By: Preetish Panda February 28, 2018

Revenues for the online recruitment vertical have grown at an average rate of 14.6% over the past five years, according to the research firm IbisWorld. And the momentum isn’t expected to stop: The industry made $4.4 billion in 2017 and is anticipated to rake in $6.4 billion in 2022 according to the same business intelligence company.

Unemployment is near record lows, leaving companies struggling to find the right talent. Wages are also finally increasing which is making it difficult to lure people away from current jobs. Considering this, the job boards are in sweet spot right now; however, the market is getting crowded by new entrants trying to get a piece of pie. This essentially means job boards need to concentrate on getting high quality job listings in order to cater to the rising demand.

In this post we’ll cover some of the compelling insights that we unveiled by analyzing the job data feed delivered by JobsPikr — a platform to access fresh job posts from company sites and job boards.


This dataset is comprised of data extracted from some of the popular US-based job boards such as CareerBuilder, Dice, Monster and Indeed. The total record count was more than 150,000 for the job postings crawled in the last 30 days. Given below are the data fields:

  • Crawl timestamp
  • Job category
  • City
  • Company name
  • Company description
  • Contact person email
  • Contact person phone number
  • Job board name
  • Job description
  • Job requirement
  • Job title
  • Job type
  • Post date
  • Salary
  • State

Note that all the job boards won’t have above-mentioned data fields — it varies depending on the portal.

Exploratory analyses

We’ll perform exploratory analyses on the vital data fields to understand the present state of the job market. Then, we’ll move to text mining of job requirements and descriptions.

Top job categories

Given below is the chart depicting the top job categories — we can see that ‘sales’, ‘engineering’, ‘marketing’ have captured the top three spots.

Job categories

Top locations in terms of job postings

Here we’ll uncover the top 10 locations in the United States with most number of job openings. The chart shows that ‘Atlanta’, ‘San Francisco’ and ‘New York City’ feature as the top three cities in this case.

job locations

Top companies with job listings

Let’s now find out the companies that have listed job openings. We can see that the leading recruitment and employment agencies like ‘CyberCoders’, ‘Robert Half’, ‘REED’ and ‘Hays’ have captured a major share. Deloitte as a service network company has got the third spot.

top companies

Top job titles

Let’s now analyse the job titles to figure the most in-demand positions. We can see can that cumulatively there is substantial demand for managerial positions cutting across all other job roles. Other than that engineering and development related roles have significant openings. Next up are the sales and consulting roles. The chart also shows that ‘Java’ and ‘SAP’ related job titles are in demand as well.

Top terms in job titles

Salary range

The predominant salary range falls in between $80k to $120k. Other salary ranges that are more than 300k and go up to $500k are outliers.

Salary histogram united states

Job type

We can see that majority of the job openings are on ‘full time’ basis and ‘temporary’ jobs are least in number.

job type

Text mining

Now we’ll perform text mining techniques to on the job description and job requirements fields to understand the top terms used to describe the job and the associated skills.

Top terms in job description

Let’s find out the most frequently used words in the job description using the word cloud given below:

wordcloud-job description-us

We can see that ‘Experience’ is the most frequently used word in job description. Other words with higher frequency are ‘work’, ‘years’, ‘team’, ‘skills’.

Top skills

Now let’s uncover the most in-demand skills by going through the job requirements posted in the job listings. Here is the word cloud created using the frequency of occurrence of the term.

wordcloud-job requirement

Clearly degree, diploma and certification are most common and predominant elements in the job requirements in terms of education. Further analyses of the word frequency reveals the following top 10 skills:

  • SQL
  • Java
  • Security
  • Data
  • Excel
  • HTML
  • Systems
  • JavaScript
  • Agile
  • C

It is quite evident that older technologies like Java and C are still in demand. Apart from that, skills around data analysis and security are also sought after.


In this study, we analyzed the clean job feeds delivered by JobsPikr to understand the current US job market. Apart from exploratory analyses, other advanced techniques such as topic modeling can also be applied for better job matching in job boards.

Let us know in the comments section if you would like to uncover any additional insight.


Leave a Reply

Your email address will not be published. Required fields are marked *