New technology, old patterns: Why artificial intelligence struggles to outgrow gender bias
Machine learning and artificial intelligence can feel very modern, but language models are decades behind in one key element, according to UN Women: gender bias.
Ahead of three key UN summits focusing on AI, the organization has warned that these platforms can subtly, and consistently, reinforce old gender stereotypes.
The AI models that we are using today, “answering our questions and drafting our emails, finishing our sentences for us, or scheduling our doctor’s appointments, unfortunately, these are gender biased,” said Jayathma Wickramanayake, UN Women Lead on Digital Technologies.
A 2024 UNESCO study of stories and sentences generated by AI models found that female-coded names were more likely to be associated with concepts like “home,” “family” and “children.” Male-coded names were more likely to be linked to concepts like “business,” “executive,” “salary” and “career,” due to this bias in the content the models had ingested.
This problem is made more obvious by looking at the concept of “historical” models, trained on data only available up to a single point in time. For example, “Talkie”, a model trained by Harvard on pre-1931 public domain content from its libraries, was found to “reproduce racist or discriminatory attitudes that were common at the time.”
The researchers added “moderation layers and warnings for problematic outputs.”
In the run-up to the UN summits, Wickramanayake called for a focus on gender equality in every stage of the technology’s lifecycle.
“We know that the whole of the world’s community, the leaders from member states, from private sector, from civil society, from academia are going to be in Geneva, deciding the future of AI governance,” she told Straight Arrow.
How do AI models become biased?
The bias in AI models comes from a few places, Wickramanayake told Straight Arrow. Firstly, Large Language Models are based on statistical correlations and relationships between words in text.
That text includes decades of historical information, literature, content and user inputs on social media, which inherently contains information linking gender with experiences, events and activities. Tech companies have trained their models using books, including historical books, textbooks and works of fiction, as well as user-generated content from sites like Reddit — none of which is free of gendered language.
As AI-generated content grows more common, the outputs of biased models are increasingly permeating our lives and the content we absorb.
“The important thing to note, with all aspects of algorithmic bias, is just how subtle it is,” said Sioux McKenna, Professor of Higher Education Research at Rhodes University. A 2025 paper on the topic described how, “among the myriads of harmful biases that AI perpetuates, gender bias is one of the most insidious.”
For example, last month, researchers at the University of Pennsylvania found that AI-generated screenplays were less likely to pass the “Bechdel test” — a classic measure of female representation — than ones generated by humans.
Researchers at the University of Washington also found that when LLMs were tasked with generating stories about animals — with no gender specified — they added masculine characters to 40.6% of stories, and feminine characters to just 2.2%. According to the paper’s authors, this indicates that even when neutrality is emphasized in the prompts, the default underlying mathematics and models might still be biased.
What would help account for the bias in AI models?
One of the outcomes that Wickramanayake would like to see from the Global AI Dialogue in Geneva, “is for it to recognize that gender equality is a foundational principle and should be a foundational principle of AI governance.
“We want governments to commit to integrating gender considerations in national AI governance policies,” she told Straight Arrow. “About 138 countries in the world have a national AI strategy, but only 24% of them even mention women and girls in these strategies. And only about 18 of them actually do something meaningful about it.”
But education is also important, and much more difficult, McKenna told Straight Arrow. When asking an LLM questions, she said, “if you are not sufficiently aware of the topic, you are only going to see the dominant pattern.”
She teaches her students to understand issues from a range of perspectives on different topics. She then guides them to see that LLMs often only offer a single dominant or mainstream perspective when asked to carry out research.
“We get our students to have already become aware of these different positions, and then get them to use large language models to answer the questions,” she said. “And then go: Oh, but hang on, this is only giving the position of those few readings, and it’s completely ignoring” other angles, perspectives or thinkers.
She encourages her students to interrogate the LLMs’ outputs to ensure “that every person in the world understands how large language models work in terms of pattern recognition and pattern creation rather than sort of definitive factual answers. So that they don’t confuse large model outputs for Wikipedia.”
Women’s roles are more vulnerable to AI automation
Algorithmic bias has been shown to influence hiring decisions. In 2018, Amazon had to scrap a hiring platform that was found to discriminate against women. Because the tech industry is male-dominated, men had historically been hired more; so the model taught itself that men were better candidates.
Women are also more vulnerable to industrial change caused by the proliferation of artificial intelligence. And the workers most at risk of being displaced by AI are in industries dominated mainly by women, according to a January 2026 report by the Brookings Institution, a nonpartisan think tank, and the Centre for the Governance of AI.
The study found that 6.1 million U.S. workers were in vulnerable positions because they lacked the capacity to adapt or change jobs — possibly due to a lack of savings, specialized skillsets, their location or their age — and that 86% of those people were women.
In April, the CEO of Palantir suggested that AI would have the most detrimental effects on the jobs of highly educated women. Sometimes, current social factors are at play. Women are more likely than men to take time out from the workplace, and Wired recently reported some women who took maternity leave returned to roles that demanded new skills, as their employers had enthusiastically adopted AI in their absence.
While Wickramanayake worries that hard-won benefits in education and employment for women across the world could be erased by the technical shift to AI, she does also see possibility. She is hopeful that machine learning tools will help adapt educational material into different languages, develop personalized learning and make health care more accessible. She sees promise in examples of AI being used to counter discrimination: During the 2024 Tokyo Olympics, the IOC used AI to take down online abuse directed at female athletes.
She’d like to see more of that.
Round out your reading
- America at 250: Straight Arrow’s week-long look at the pressures testing modern America.
- Ketamine is booming in America. The reasons go beyond drugs.
- Cluster of AI news sites suddenly goes dark after human reporter starts asking questions.
- A 79-year-old, obese, well-connected man gained exclusive access to a new weight loss drug. Is it Trump?
- Newsom pushes national wealth tax after opposing California billionaire tax.
Ella Rae Greene, Editor In Chief
