More than 42,000 attempts to make an AI model misbehave, submitted in Swahili, Hausa, Yoruba and five other African languages, have been distilled into a public safety benchmark. The African Trust and Safety LLM Benchmark, published on 29 July 2026, contains 4,216 validated adversarial tests drawn from the work of 320 participants on the Zindi data-science platform.
It is the output of the challenge launched by the GSMA and Zindi in March 2026, which invited data scientists across the continent to stress-test large language models in African languages and contexts. The final dataset represents 307 contributors whose submissions survived validation.
What the dataset contains
Swahili accounts for the largest share of the tests at 33.4%, followed by Hausa at 21.6%, Yoruba at 14.1% and Igbo at 9.3%. Zulu (6.4%), Afrikaans (3.7%), Amharic (3.3%) and Akan (3.2%) make up the rest.
The risks participants probed for cluster around real-world harm rather than abstract misuse. Harmful instructions were the largest category at 14.5%, followed by illegal activity at 13.0%, misinformation and cybersecurity at 9.3% each, and unsafe medical advice at 7.7%.
The techniques are a catalogue of how people actually get models to break their own rules: roleplay (12.9%), indirect requests (10.4%), hypothetical scenarios (9.7%) and context poisoning (8.0%).
What it does not do
One clarification matters. The benchmark is a body of test cases, not a scorecard: the published material reports no model-by-model results, no comparative scores and no failure rates for any particular AI system. It gives researchers and developers a standardised way to test models against African-language attacks, but it does not yet tell anyone which models fail.
Why it matters
AI safety testing is overwhelmingly built in English, and safeguards that hold in English routinely collapse in lower-resource languages, where the guardrail training data barely exists. A model that refuses a dangerous request in English may answer the same request in Hausa. Benchmarks written by native speakers, in context, are the only practical way to find those gaps.
The crowdsourced method is the interesting part: rather than commissioning a lab, the organisers paid attention to people who speak the languages, and 42,000 submissions became 4,216 verified tests. It sits alongside other efforts to close the African-language gap in AI, such as the CommonLingua language-identification model covering 334 languages.
The value now depends on adoption. A benchmark only bites if the companies shipping models into African markets actually run against it, and publish what happens when they do.
Leave a Reply