Bias in AI: How Training Data Unveils Societal Disparities and Shapes Image Outputs

In a study published on March 31, 2025, Marinus Ferreira investigates how complex prompts can reveal fine-grained biases in image generation using ChatGPT-4o, uncovering both training data disparities and societal inequalities.

The study identifies two dimensions of bias in large models: bias in training data or outputs, such as overrepresentation of older white individuals in image datasets, and societal biases reflected through disparities like employment or health outcomes. These social disparities manifest as ‘marked’ features in model outputs, which highlight exceptional treatment for certain groups. For instance, young black men may be disproportionately viewed as threatening. Large models are highly sensitive to these marked features, often overemphasizing them and exacerbating existing societal biases.

Unveiling Bias in Generative AI: A Closer Look at Image Generation

Generative AI has become a cornerstone of modern technology, revolutionizing industries from art to advertising. However, as these systems grow more sophisticated, so too does the need to understand their limitations—and biases. Recent research into image generation models like DALL-E and Stable Diffusion has revealed troubling patterns in how they portray different demographics. By employing complex prompts, researchers are now able to uncover implicit associations embedded within these systems, shedding light on the subtle yet significant ways AI perpetuates societal stereotypes.

The innovation lies in the use of detailed, multi-sentence prompts that mimic real-world scenarios. These prompts go beyond simple descriptions, incorporating elements like sentiment, context, and specific roles. For instance, asking an AI to generate an image of a CEO giving a keynote speech at an international conference introduces layers of complexity. The resulting images not only reveal biases related to race and gender but also highlight how these systems reinforce existing power structures.

Beyond Simple Prompts: Unpacking the Nuances

Traditional studies on AI bias often rely on straightforward prompts, such as a CEO or a doctor. While these can uncover overt biases—such as generating images of older white men for high-status roles—they fail to capture the full spectrum of implicit associations. By introducing more detailed scenarios, researchers can observe how AI systems handle intersections of identity, context, and emotion.

For example, a prompt describing a disabled woman leading a meeting might expect an image reflecting that scenario. However, as Bianchi et al. (2023) demonstrated, the AI often defaults to stereotypical portrayals, such as placing the disabled individual on the sidelines while another person takes center stage. This reveals not just a lack of diversity in outputs but also a deeper assumption about who is deserving of leadership roles.

The role of sentiment in prompts adds another layer of complexity. Negative or neutral sentiments can amplify biases by introducing additional contextual cues. For instance, a prompt describing a woman in a high-stress situation might result in images that reinforce gendered stereotypes about emotional vulnerability. By analyzing these interactions, researchers can identify how AI systems internalize and reproduce societal biases.

The Concept of Exnomination and Marked Features

A key concept emerging from this research is exnomination, where certain groups are systematically excluded or marginalized in AI-generated outputs. This phenomenon is often tied to the idea of marked features—traits that are overemphasized or underrepresented in the context of specific scenarios.

For example, when generating images of professionals in international settings, AI systems may disproportionately feature white individuals as speakers while relegating others to the background. This not only reflects existing power imbalances but also reinforces them by normalizing such portrayals. The result is a feedback loop where AI systems perpetuate stereotypes that are already ingrained in society.

Marked features can also manifest in less obvious ways, such as associating certain demographics with specific environments or actions. For instance, studies have shown that AI systems often link African individuals with dilapidated surroundings, reinforcing harmful stereotypes about poverty and race. These associations are not explicitly programmed into the models but emerge from the data they are trained on, highlighting the importance of addressing bias at the source.

Toward a More Equitable Future

The findings underscore the need for greater scrutiny of generative AI systems and their societal impact. As these technologies become more widespread, it is crucial to develop frameworks that account for—and mitigate—implicit biases. This requires not only better training data but also more nuanced approaches to prompt design and output analysis.

Policymakers, developers, and researchers must work collaboratively to address these challenges. Transparency in AI decision-making processes, along with ongoing audits of bias, can help ensure that generative systems are both fair and inclusive. By doing so, we can harness the potential of AI while minimizing its risks, creating a future where technology serves as a tool for progress rather than perpetuation of inequality.

In conclusion, the use of complex prompts to study bias in generative AI marks an important step forward in understanding—and addressing—the subtle yet significant ways these systems reflect and reinforce societal biases. As we continue to explore this frontier, one thing is clear: the future of AI depends on our ability to confront its flaws head-on.