Applying Topic Modelling on JobsPikr Data to Create Robust Jobs

Hiring the right candidate and matching the skills required for the job requirement is a major bottleneck for any organisation. The problem is huge and there are continuous efforts to mitigate the same by different companies. According to this TechCrunch post, there has been a proliferation of new startups in the recruitment space with investors doling out large sum of capital. Now, if we look at the ways through which companies get candidate profiles, it’d be quite evident that there are thousands of job boards and recruitment agencies to get in touch with prospects apart from own career section in the official site. While the initial wave of job marketplaces and boards solved the problem of creating talent pool for companies, another problem emerged when the aggregators started to scale both in terms of recruiters and applicants. That problem is closely associated with the ability to match the right candidate with the right employer. Needless to say that a job board’s competitive advantage lies in providing highly accurate matching service. In order to achieve that, we need to consider the following primary factors:

  • Ability to populate fresh job listings
  • Building strong algorithms for matching of the candidates with the jobs

In this post, we will explore how that can be done by using the job data provided by JobsPikr. For those who might not be familiar with JobsPikr, it is a job data delivery platform powered by PromptCloud’s proprietary machine learning technique that performs automated crawls on daily basis to extract job data directly from popular job boards.

As we can see that the job data delivered by JobsPikr is from the job boards, there is more accuracy, freshness, and comprehensiveness. Thus, the data quality is far superior to the data you get by crawling other job boards. This aspect of JobsPikr handles the first factor mentioned above. More on this is topic can be found out in our previous post.

Coming to the crux of this post, i.e., a solid algorithm for job matching we need to first understand the data fields delivered by JobsPikr. Here are the data fields that you would get from a typical job listing:

  • URL
  • Job post date
  • Company name
  • Job title
  • Job text

Matching using ‘job title’ is quite straightforward, but the ‘job text’ is a gold mine for us in this context. This brings us to the concept of Topic Modelling and how it can be applied to the ‘job text’. Note that this post doesn’t cover the complete job matching process, rather it focuses on Topic Modelling, which can be used to strengthen the existing matching process.

What is Topic Modelling?

In the simplest terms, Topic Modelling can be considered as an unsupervised machine learning technique that can be used to analyse, annotate, organize and search large volume of unlabeled text. This technique is particularly useful for finding latent pattern in large collection of text by extracting cluster of words that are closely related and frequently occur together. For example, a good topic model would create the following word cluster for Education related topic – “learn”, “teacher”, “college”, ” career” and the following for Business related topic – “finance”, ” corporate”, “investment”, ” acquisition”. Essentially, the topics are created by cluster of words and the documents are created by different topics. The key factor is that documents won’t belong to a single topic — they would be a mixture of different topics. That means a single document can be formed by combinations of multiple topics. Here is an illustration:

Although you can find a lot of articles on this subject, click here too see a great paper to learn more.

How to Apply Topic Modelling to JobsPikr Data

There are various algorithms (Explicit semantic analysis, Latent semantic analysis, Latent Dirichlet allocation, etc.) and libraries available for topic modelling that can be applied to the text corpus. As discussed earlier, the ‘job text’ delivered by JobsPikr contains the job description along with qualification and other associated details available in a job listing. We can consider the ‘job text’ of each listing as documents and perform topic modelling to find out the underlying topics of job listing from different geographies, industries and companies.

Four different topics have been formed based on the content of ‘job text’. One topic for the subsidiaries like shopbop and East Dane, another one the equal opportunity employment, finally third and fourth topic for engineering and management related job listings. Once the topics are identified, we can find out the probability of different words being generated from the topics and the per-document-per-topic probabilities (estimation of percentage of words in a document being generated from a topic). This means various sections of the candidate profile (e.g. skills, experiences, education) can be analyzed to find out the maximum probability of finding them from the documents. This probability estimation is nothing but the closeness of the candidate profile to the job listing. For example, someone with skills such as scripting, GIT, AWS, Jenkins, etc. would have a high probability for a topic that can be labelled as “DevOps” and this particular topic would have maximum contribution to the document (job text of a listing) with requirement for DevOps engineer. Similarly, various sections of candidate profile can be given weight-age in terms of closeness match to finally arrive at the most suitable opportunity.

Wrapping Up

As personalisation is a key factor in any business domain, the recruitment industry must also adopt the same by deploying cutting edge technologies. One facet of personalisation would be showing candidate the right job opportunity via machine learning techniques. Currently, there are numerous open source ML frameworks released by tech giants that can be used by any company to solve their business problems.

In this post, we covered why job matching is crucial to any job board and how the data delivered by JobsPikr can be instrumental in building a solution that can provide competitive edge to the aggregators. Now it’s time for you to explore options that can help you make your job board highly relevant to both job seekers and employers.

Acquire clean and up-to-date job listings data in a structured format via JobsPikr.

CTA Banner


  1. Ronald

    October 15, 2020 at 8:25 am

    Terrific post however I was wondering if you could write a litte more on this
    topic? I’d be very thankful if you could elaborate a little bit more.

    Thank you!

    • Tarun

      October 15, 2020 at 1:16 pm

      Ho Ronald, We surely will. Subscribe to our social media channels to stay engaged with the new content that we post on a regular basis.

  2. biet thu lau dai tay ho

    October 13, 2020 at 12:42 am

    Нellⲟ Tһere. I foսnd your blog uѕing msn. Thiѕ is an extremely
    ѡell written article. I will be sure to bookmark іt and come baсk to гead more of your usefᥙl іnformation.
    Τhanks fοr the post. I’ll ceгtainly return.

  3. Kathrin

    October 11, 2020 at 6:44 pm

    I think the admin of this site is in fact working hard for his site, as here every
    material is quality based data.

  4. judi kartu

    October 9, 2020 at 1:27 pm

    Greetings Ι am sо haрpy I foᥙnd yοur
    blog, I really foսnd you Ьy error, while I ᴡas browsing ߋn Digg for sοmething
    еlse, Regаrdless Ι ɑm here now and would just lіke tⲟ ѕay cheers for a tremendous post ɑnd а аll roynd exciting blog (Ӏ
    also love the theme/design), Ι don’t һave tіmе to loⲟk oѵer it all att the minute but I hаve book-marked it and ɑlso included your RSS feeds,
    soo, when I havе, timе I will be back to read morе, Pⅼease ⅾⲟ keep uρ tһe
    superb ѡork.

  5. free spins coin master

    October 9, 2020 at 3:25 am

    Thanks in favor of sharing such a pleasant idea,
    article is pleasant, thats why i have read it entirely

Comments are closed.

About JobsPikr

JobsPikr provides fresh job data feed directly from the prominent job boards across geographies. It has been developed by our parent company, PromptCloud – a pioneer in Data-as-a-Service with deep domain expertise.

Get in Touch

Stay Connected
Quick Links
The latest JobsPikr news, articles, and resources, sent straight to your inbox every month.
We’ll never share your details. See our Privacy Policy

Copyright © JobsPikr . All rights reserved.