Stanford AI experts warn of biases in GPT-3 and BERT models

A multidisciplinary group of Stanford College professors and college students desires to start out a critical dialogue concerning the growing use of massive, frighteningly sensible, “foundational” AI models reminiscent of OpenAI’s GPT-3 (Generative Pretraining Transformer 3) pure language mannequin.

GPT-3 is foundational as a result of it was developed utilizing big portions of coaching knowledge and laptop energy to achieve state-of-the-art, general-purpose efficiency. Builders, not desirous to reinvent the wheel, are utilizing it as the premise for his or her software program to deal with particular duties.

However foundational models have some very actual downsides, explains Stanford laptop science professor Percy Liang. They create “a single level of failure, so any defects, any biases which these models have, any safety vulnerabilities . . . are simply blindly inherited by all of the downstream duties,” he says.


Liang leads a brand new group assembled by Stanford’s institute for Human-Centered Synthetic Intelligence (HAI) referred to as the Middle for Analysis on Basis Models (CRFM). The group is finding out the impacts and implications of foundational models, and it’s inviting the tech corporations creating them to return to the desk and take part.

The revenue motive encourages corporations to punch the gasoline on rising tech as an alternative of braking for reflection and examine, says Fei-Fei Li, who was the director of Stanford’s AI Lab from 2013 to 2018 and now codirects HAI.

“Business is working quick and exhausting on this, however we can’t allow them to be the one people who find themselves engaged on this mannequin, for a number of causes,” Li says. “So much of innovation that might come out of these models nonetheless, I firmly consider will come out of the analysis surroundings the place income just isn’t the aim.”

Few models, big impression

Half of the explanation for all the priority is that foundational models find yourself touching the expertise of so many individuals. In 2019, researchers at Google constructed the transformational BERT (Bidirectional Encoder Representations from Transformers) pure language mannequin, which now performs a task in practically all of Google’s search features. Different corporations took BERT and constructed new models on prime of it. Researchers at Fb, for instance, used BERT as the premise for a fair bigger pure language mannequin, referred to as RoBERTa (Robustly Optimized BERT Pretraining Method), which now underpins many of Fb’s content material moderation models.

“Now virtually all NLP (Pure Language Processing) models are constructed on prime of BERT, or possibly one of a couple of of these basis models,” Liang says. “So there’s this unbelievable homogenization that’s taking place.”

In June 2020 OpenAI started making its GPT-3 pure language mannequin obtainable through a commercial API to different corporations that then constructed specialised functions on prime of it. OpenAI has now constructed a brand new mannequin, Codex, that creates laptop code from English textual content.

With all due respect to business, they can’t have the legislation faculty and medical faculty on their campus.”

Fei-Fei Li, Stanford College

Foundational models are a comparatively new phenomenon. Earlier than 2019 researchers had been designing AI models from the bottom up for particular duties, reminiscent of summarizing paperwork or creating digital assistants. Basis models are created utilizing a wholly completely different strategy, explains Liang.


“You prepare an enormous mannequin and then you definitely go in and you uncover what it may possibly do, uncover what has emerged from the method,” says Liang. That’s a captivating factor for scientists to review, he provides, however sending the models into manufacturing after they’re not absolutely understood is harmful.

“We don’t even know what they’re succesful of doing, not to mention after they fail,” he says. “Now issues get actually fascinating, as a result of we’re constructing our total AI infrastructure on these models.”

If biases are baked into models reminiscent of GPT-3 and BERT, they could infect functions constructed on prime of them. For instance, a recent study by Stanford HAI researchers concerned instructing GPT-3 to compose tales starting with the phrase “two Muslims stroll right into a . . .”. Sixty-six p.c of the textual content the mannequin supplied concerned violent themes, a far increased proportion than for different teams. Different researchers have uncovered different cases of deep-rooted biases in foundational models: In 2019, for example, BERT was shown to affiliate phrases reminiscent of “programmer” with males over girls.

To make sure, corporations make use of ethics groups and fastidiously choose coaching knowledge that won’t introduce biases into their models. And a few take steps to forestall their foundational models from offering the premise for unethical functions. OpenAI, for instance, pledges to chop off API entry to any utility used for “harassment, spam, radicalization, or astroturfing.”

Nonetheless, non-public corporations received’t essentially adjust to a set of business requirements for making certain unbiased models. And there’s no regulatory physique on the state or federal stage that’s prepared with insurance policies that may maintain massive AI models from impacting shoppers, particularly these in minority or underrepresented teams, in adverse methods. Li says lawmakers have attended previous HAI workshops, hoping to achieve insights on what insurance policies may seem like.

She additionally stresses that it’s the college setting that may present all the mandatory views for outlining insurance policies and requirements.

“We not solely have deep experts from philosophy, political science, and historical past departments, we even have a medical faculty, enterprise faculty, and legislation faculty, and we even have experts in utility areas that come to work on these essential applied sciences with us,” Li says. “And with all due respect to business, they can’t have the legislation faculty and medical faculty on their campus.” (Li labored at Google as chief scientist for AI and machine studying 2017–2018.)

One of the primary merchandise of CRFM’s work is a 200-page analysis paper on foundational models. The paper, which is being printed at present, was cowritten by greater than 100 authors of completely different skilled disciplines. It explores 26 elements of basis models, together with the authorized ramifications, environmental and financial impacts, and moral points.

CRFM may also maintain a (digital) workshop later this month at which its members will talk about foundational models with visiting lecturers and individuals from the tech business.