What is “real-time” labor market information and why is it important?

Real-time labor market information, also called real-time jobs data, is based on analysis of the millions of job listings posted every day by employers. A job description provides important insight into what employers are thinking. By its very nature, a job description means an employer has had to spell out the specific skills and qualifications they need to get a particular job done. Employers also have every incentive to get job descriptions right (bad or delayed hires can cost a lot of time and money). By analyzing job listings, researchers can find out which companies and sectors are hiring, which skills are in demand, how hard they are to find, and which jobs and skill sets are emerging.

Real-time data represent an important complement to traditional survey-based labor market data, such as those produced by government agencies.  First, job postings data provide a timely view on current market conditions.  Some established labor data reports are based on observations that can be up to four years old, while the closely watched federal jobs report is released monthly. Fresh data is especially important in market areas experiencing rapid change.

In addition, real-time data can break down the job market to a precise level of detail.  Traditional labor market data are structured around broad job categories, and all jobs within those categories are presumed to be identical in terms of the skills, experience, and education they require.  By contrast, real-time job market data can be much more specific, reflecting how jobs differ within and across sectors and geographies.

How are real-time labor data collected?

Real-time jobs data are compiled by scanning the internet daily using bots that seek out job postings on job boards, corporate websites, and other places where job ads are posted. For example, Burning Glass technology scans more than 40,000 sources, and at any given time, tracks about 3.4 million unique, currently active openings.

Our software extracts topline information about each job such as title, occupation, employer, and location, and then uses natural-language technology to read each job description to identify specific occupations, skills, and qualifications that employers are seeking. We are able to break down jobs into dozens of  distinct elements and make comparisons among them.

How do real-time data compare to other sources?

Most labor market information, such as the data produced by the Bureau of Labor Statistics or other government statistical agencies, is collected using surveys of employers, job seekers, or the general public. These data are excellent at tracking macroeconomic trends in employment—in other words, the big picture. Because these surveys have been done consistently for decades, they are good for comparing data over time, and for making projections about the job market.

An important drawback is that it takes time to compile traditional data series. Depending on the report, the jobs data can be up to two years behind the market. The official government catalog of occupations, O*NET, is updated annually, but specific occupations may go 10 years or more without an update. The best-known U.S. government labor source, the monthly jobs report (formally known as the Employment Situation Summary), is also the fastest.

By contrast, real-time data can provide a picture of what the job market looked like as recently as yesterday. Like other internet search engines, the Burning Glass spiders can pick up on job postings as soon as they are put up (or taken down).

In addition to being faster, real-time data are also more granular. Put another way, real-time data are microeconomic data, because they are focused on the specific details of job postings and how they affect employers and job seekers. Employers are constantly adjusting job descriptions to match a changing marketplace. Real time data will see those changes more quickly. In addition, the skills required for a job can vary by region or industry, which bigger-picture data may be unable to capture. Burning Glass is able to track a number of elements, such as the specific skills of individual jobs, that aren’t available in other data sets.  For example, a community college trying to construct a welding program needs to know the specific mix of skills local welding jobs require so that they can plan curriculum accordingly.

This isn’t a question of better or worse, it’s a matter of big picture versus small picture. You need both to get a full view of the job market.

Why does the level of detail matter?

Data are only useful if you can act on them. For data to be actionable, they need to be specific. It is one thing to know that there is high demand for “computer programmers.” But it’s much more useful to know how many companies need Ruby expertise vs Java programming, and how that demand differs in Boston vs. Boise. With that level of detail, employers, academic institutions, and workforce agencies can make detailed plans about how to train and hire workers. And workers can make more informed career decisions,

Are there jobs that are missed?

Most jobs are now posted online. A Georgetown University study, using 2013 data, found this figure to be between 60% and 70% of all jobs. Since then, our analysis shows that the share has grown to roughly 85%. The jobs that aren’t online now are usually in small businesses (the classic example being the “help wanted” sign in the restaurant window) and union hiring halls. Lower-income, lower-skill jobs are also less likely to be posted online than higher-skill jobs. Overall, however, research shows the online job market has consistently expanded over the last few years.

What about jobs that are posted in multiple places?

Because so much effort is invested into the comprehensiveness of data capture, including scanning over 40,000 different sources daily, it is quite common to find the same job posted multiple times across sites or even within the same site.  To ensure that real-time figures provide a high fidelity picture of the current job market, removing duplicate postings is an essential part of the collection process. Sophisticated algorithms that take into account a range of factors help identify and root out duplicates.  Overall, close to 80% of all the postings Burning Glass collects are marked as duplicates.

Do employers really say what they mean in job postings?

Employers are surprisingly articulate in describing what they want.  Although there are plenty of poorly written or generic job postings, when aggregated across hundreds of thousands or millions of postings, there is significant consistency among employers about the skill and qualification requirements for a given type of job. Importantly as well, the requirements employers articulate vary considerably and predictably across different types of jobs.  In other words, employers for Compensation and Benefits Analysts tend to ask for similar skills and qualifications, but what they ask of Compensation and Benefits Analysts is different from what employers tend to ask of Human Resources Managers.

In fact, a recent academic study published by the National Bureau of Economic Research concluded that job postings have become more specific over the last few years. The paper found that online job postings were 12% more likely to ask for specific cognitive skills, educational requirements, or experience levels in 2015 than in 2007.

That said, it is important to realize that a job posting is not a comprehensive representation of all of the requirements of a job; employers tend to emphasize what they feel they may not otherwise find. For example, a job posting for a lawyer might not specify a J.D. degree or admission to the bar—an employer would assume a lawyer would have those qualifications. In other cases, a requirement might serve as a proxy for a skill the employer needs. In a job where communication skills are important, an employer might choose to ask for a bachelor’s degree, on the assumption that a college graduate is more likely to have those skills.  Still, even where stated requirements are intended as a proxy, they are still requirements nonetheless – and therefore relevant and accurate in assessing the hiring landscape.