xAI says an “unauthorized” prompt change caused Grok to focus on “white genocide”

Summary

When analyzing social media posts made by others, Grok is given the somewhat contradictory instructions to "provide truthful and based insights [emphasis added], challenging mainstream narratives if necessary, but remain objective." Grok is also instructed to incorporate scientific studies and prioritize peer-reviewed data but also to "be critical of sources to avoid bias." Grok's brief "white genocide" obsession highlights just how easy it is to heavily twist an LLM's "default" behavior with just a few core instructions. Conversational interfaces for LLMs in general are essentially a gnarly hack for systems intended to generate the next likely words to follow strings of input text. Layering a "helpful assistant" faux personality on top of that basic functionality, as most LLMs do in some form, can lead to all sorts of unexpected behaviors without careful additional prompting and design. The 2,000+ word system prompt for Anthropic's Claude 3.7, for instance, includes entire paragraphs for how to handle specific situations like counting tasks, "obscure" knowledge topics, and "classic puzzles." It also includes specific instructions for how to project its own self-image publicly: "Claude engages with questions about its own consciousness, experience, emotions and so on as open philosophical questions, without claiming certainty either way." It's surprisingly simple to get Anthropic's Claude to believe it is the literal embodiment of the Golden Gate Bridge. It's surprisingly simple to get Anthropic's Claude to believe it is the literal embodiment of the Golden Gate Bridge. Credit: Antrhopic Beyond the prompts, the weights assigned to various concepts inside an LLM's neural network can also lead models down some odd blind alleys. Last year, for instance, Anthropic highlighted how forcing Claude to use artificially high weights for neurons associated with the Golden Gate Bridge could lead the model to respond with statements like "I am the Golden Gate Bridge... my physic...