Working Paper Series no. 900: Encouraging and Directing Job Search: Direct and Spillover Effects in a Large Scale Experiment

In this paper, we provide experimental evidence on the potential social value of a large-scale public recommendation algorithm developed by the French public employment service (PES). The platform is called “La Bonne Boîte” (“The adequate firm”, henceforth LBB). The platform started in 2015, and is based on an algorithm predicting hirings at the firm/occupation level. The goal of PES with this service is to provide job seekers with access to the so-called “hidden market” of firms that recruit without necessarily posting job ads. On the business-as-usual mode, the LBB website directs job seekers toward a list of firms most likely to hire them according to the location and occupation criteria they enter. We partner with PES to test the impact of this service using a randomized encouragement design: we send emails to about 800,000 registered job seekers (the treatment group) to encourage them to use LBB, and measure the impact on job finding rates. To analyze mechanisms and potential improvements, we use the encouragement emails sent to treated job seekers one step further, in the form of targeted recommendations toward specific firms within and outside their occupation of reference. While we introduce some random variation when making these recommendations, we also discipline ourselves using a simple, flexible equilibrium model at the commuting zone level. The model takes into account information on local tightness across occupations and makes educated guesses on key parameters (occupational mobility costs, firms' screening technology) to optimize recommendations in order to maximize the expected local aggregate employment. In a second step, we analyze ex post whether these presumed optimal recommendations were indeed effective. Specifically, we ask two questions: (i) Is there a positive private return to the email's recommendations and encouragement to search via LBB? This is directly identified by the reduced form effect comparing treated job seekers (who received the email) to control ones (who received no email). (ii) Do the recommendations generated by the ex-ante model strike the right balance, in terms of the breadth of occupational search, between congestion and mobility costs? Answering this second question is harder, as it involves estimating interference effects: recommendations made to a given job seeker, if they lead to a change in their application behavior, are likely to have external effects by displacing other job seekers. We build upon the recent literature on interference in randomized trials, in particular Hu, Li and Wager (2022), to estimate not only the direct effect but also the indirect effect of recommendations.

We find that the e-mails' recommendations and search encouragement increase by around 1.5% the job finding rate of female job seekers (+0.26pp from a baseline of 17.43%, see graph 1b). This result is driven by an activation effect of our intervention that led to an overall increase in search effort for women. Contrary to women, men appear to concentrate their search effort on the narrow set of LBB and recommended firms (“targeting effect”, see graph 1a). This redirection of men’s job search effort leads to a relative fall in job finding among non LBB firms and rise in job finding in LBB firms, resulting in an overall effect which is not statistically different from zero.

In a next step we document that the “targeting effect” (i.e. an increase in the likelihood that specific matches between pairs of job seekers and firms occur) is especially strong when we recommend firms hiring outside of a jobs seeker’s origin occupation. This underlines the ability of the recommender system to redirect job seekers’ search effort, leaving room for a potentially beneficial reallocation of job seekers’ search effort across labor markets. We find that recommending job seekers to search in nearby occupations significantly reduces congestion frictions in their origin occupation.  This is especially true when origin occupation were initially slack, i.e. when congestion frictions on the job seekers’ side were initially high. Overall, this provides evidence, in a real set-up, that recommender systems can be used to reduce mismatch unemployment due to informational frictions.