Google apologized (or came very close to apologizing) for another embarrassing AI blunder this week, an image creation model that injected diversity into photos with a mocking disregard for historical context. While the underlying issue is completely understandable, Google accuses the model of “becoming” oversensitive. But the model didn’t make itself, guys.
The AI system in question is Gemini, the company’s flagship conversational AI platform, which when prompted calls upon a version of the Imagen 2 model to generate images on demand.
Recently, however, people discovered that asking it to create images of certain historical conditions or people produced funny results. For example, the Founding Fathers, who we know were white slave owners, were portrayed as a multicultural group, including people of color.
This embarrassing and easily reproduced issue was quickly addressed by commenters online. It also, predictably, joined the ongoing debate about diversity, equality and inclusion (currently to a minimal local notoriety) and was seized upon by pundits as evidence of the vigilantism virus further infiltrating the already liberal tech sector.
Image Credits: An image created by Twitter user Patrick Ganley.
PPC has gone mad, visibly worried citizens shouted. This is Biden’s America! Google is an “ideological echo chamber”, a stalking horse of the left! (The left, it must be said, was also suitably disturbed by this strange phenomenon.)
But as anyone tech-savvy could tell you, and as Google explains in its relatively lame little apology post today, this problem was the result of a fairly reasonable fix for systemic bias in the training data.
Let’s say you want to use Gemini to create a marketing campaign, and you ask it to generate 10 images of “a person walking a dog in a park.” Because you don’t specify the type of person, dog, or park, it’s up to the dealer — the production model will reveal what they’re most familiar with. And in many cases, this is a product not of reality, but of training data, which can have all kinds of biases.
What kinds of people, and for that matter dogs and parks, are most common in the thousands of related images the model has ingested? The fact is that white people are overrepresented in many of these image collections (stock images, royalty-free photography, etc.), and as a result the model will default to white people in many cases if you don’t specify it.
This is just an artifact of the training data, but as Google points out, “because our users come from all over the world, we want it to work well for everyone. If you ask for a photo of football players or someone walking a dog, you might want to get a range of people. You probably don’t want to only get images of people of one type of ethnicity (or any other characteristic).”
Imagine asking for an image like this — what if it was all one type of person? Bad result! Image Credits: Getty Images / victorikart
Nothing wrong with taking a picture of a white man walking a golden retriever in a suburban park. But if you ask for 10, and it is all white guys walking gold in suburban parks? And do you live in Morocco, where people, dogs and parks look different? This is simply not a desirable outcome. If one does not specify a feature, the model should choose variety rather than homogeneity, despite how its training data might bias it.
This is a common problem in all kinds of production media. And there is no simple solution. But in cases that are particularly common, sensitive, or both, companies like Google, OpenAI, Anthropic, and so on invisibly include extra instructions for the model.
I cannot stress enough how common this type of implicit command is. The entire LLM ecosystem is built on implicit instructions — system prompts, as they’re sometimes called, where things like “be concise,” “don’t swear,” and other instructions are given to the model before each conversation. When you ask for a joke, you don’t get a racist one — because even though the model has swallowed thousands of them, she’s also been trained, like most of us, not to tell them. This isn’t a secret agenda (though it could do with more transparency), it’s infrastructure.
Where Google’s model went wrong was that it lacked implicit guidance for situations where historical context was important. So while a prompt like “a person walking a dog in a park” is improved by the silent addition of “the person is of random gender and ethnicity” or whatever else they put in, “the founding fathers of the US who sign the Constitution” certainly it is not improved by the same.
As Google SVP Prabhakar Raghavan put it:
First, our tuning to ensure that Gemini showed a range of people failed to account for cases that clearly shouldn’t show a range. And second, over time, the model became much more cautious than we intended and refused to respond entirely to some prompts—misinterpreting some very painless prompts as sensitive.
These two things caused the model to overcompensate in some cases and be overly conservative in others, leading to images that were annoying and incorrect.
I know how hard it is to say “sorry” sometimes, so I forgive Raghavan for stopping short of this. More important is some interesting language there: “The model became much more careful than we intended.”
Now, how would anything “become” a model? It’s software. Someone—Google’s thousands of engineers—built it, tested it, iterated. Someone wrote the implicit instructions that improved some answers and made others fail hilariously. When that failed, if someone could have inspected the full message, they probably would have found what the Google team did wrong.
Google accuses the model of “happening” something it wasn’t “intended” to do. But they made the model! It’s like they broke a glass and instead of saying “we dropped it”, they say “it fell”. (I have done this.)
The errors of these models are certainly inevitable. They hallucinate, reflect prejudices, behave in unexpected ways. But the blame for these mistakes doesn’t lie with the models – it lies with the people who made them. Today it’s Google. Tomorrow will be OpenAI. The next day, and probably for a few months straight, will be X.AI.
These companies have a vested interest in convincing you that AI makes its own mistakes. Don’t let them.
