Artificial intelligences that are used in the application process do not seem to be as neutral as they say . Even a bookshelf in the background should be enough to influence the judgment of the AI.

The application process should be as fair and objective as possible – last but not least, anti-discrimination laws should ensure that. At the same time, we all carry prejudices with us, whether we like it or not. In this context, the promises made by many companies and startups that outsource the selection of applicants to artificial intelligence sound tempting: AI has no personal taste, is not guided by impulses and is absolutely neutral. But is it really like that?

Algorithms react to accessories such as glasses or headscarves

In a large-scale research, Bayerischer Rundfunk tested the behavior analysis AI of the start-up Retorio from Munich. It turned out that the algorithms come to different results if certain factors such as glasses or a headscarf change in the application video.

In As a first step, an actress recorded a short video application, which was then classified by the AI ​​using the so-called Big Five. This is a model that breaks down a person’s personality into five main dimensions.

Openness, conscientiousness, extraversion, compatibility and Neuroticism are the big five of every person’s personality. (Screenshot: BR / t3n)

In the following the actress spoke the same text with the same facial expressions and tonality as possible again – but this time she wore glasses or a scarf that covered her hair. The Big Five results then differed significantly each time.

Especially in relation to the The actress has to accept a loss of conscientiousness when she wears glasses. (Screenshot: BR / t3n)

The headscarf changes the actress’s Big Five results, sometimes drastically. (Screenshot: BR / t3n)

Also further attempts with wigs, a pony hairstyle and other tops resulted in strongly fluctuating Big Five results. Confronted with these observations, Retorio points out that the AI ​​also assesses the external impact of an applicant: “As in a normal job interview, such factors are also included in the evaluation.” The Retorio algorithm is designed precisely to: also assess the effect and impression a person makes on others. Systematic biases such as age, gender or origin would, however, be factored out by the AI.

The background also plays a role

In a further series of tests, the BR experimented with various backgrounds and technical factors. Video recordings of test candidates were subsequently changed. For example, a framed picture was placed behind a candidate – and the Big Five results were promptly different.

A framed picture in the background makes the candidate appear much less neurotic, says the AI. (Screenshot: BR / t3n)

A wall of books behind the candidate also led to significant deviations in the result.

The wall of books in the background even lowers neuroticism further. (Screenshot: BR / t3n)

Changes to the brightness of a video also led to different AI results.

The worse illuminated Image makes the candidate appear much less conscientious. (Screenshot: BR / t3n)

However, the BR did not succeed in targeting the Big Five results change – conclusions such as “in a bright room you appear more extroverted” cannot be drawn with it. Deviations are not systematic and can vary from person to person. This is the general problem of machine learning, explains the computer science professor Katharina Zweig, interviewed by the BR: “The fundamental problem with face recognition, through machine learning, is that we never know exactly what pattern in an image is these machines react. ”

Retorio points out at this point that the quality of the recording is in the hands of the person making the video . It is up to him or her to try out if necessary until light, sound and all other factors are optimal.

Binding rules are needed

Computer science professor Zweig therefore strongly advocates regulations in the area of ​​machine learning and the AI. But it is not enough just to look at computer technology; The social development processes are also important.

Most read