A major governance crisis has erupted in South Africa after it was revealed that the Department of Communications and Digital Technologies published a national AI policy containing fabricated citations generated by an AI.
The scandal, which broke on Wednesday evening, has sent shockwaves through the continent’s tech community and raised urgent questions about the lack of human oversight in the creation of critical legal frameworks. The draft policy, intended to position South Africa as a regional leader in the fourth industrial revolution, was found to include references to non-existent court cases and academic papers that researchers quickly identified as “hallucinations” typical of large language models. This debacle highlights a dangerous trend where over-reliance on automation is outstripping the capacity for rigorous administrative review.
The Hallucination in the High Office
The discovery was first made by a group of investigative data scientists who noticed that several of the legal precedents cited in the policy’s ethical framework did not exist in the South African Constitutional Court archives. Further investigation revealed that over 12% of the citations in the 150-page document were entirely fictitious. The incident has led to calls for the immediate resignation of the Director-General of Communications, as civil society groups demand a full audit of all government documents published in the last eighteen months.
This is not merely an embarrassing clerical error; it is a fundamental breakdown of the legislative process. When a government uses AI to write the very laws meant to regulate AI, the feedback loop creates a vacuum of accountability. In a region where digital sovereignty is a primary concern, the presence of fabricated data in a foundational policy undermines the credibility of the entire digital transition agenda.
Oversight in the Age of Automation
The South African debacle serves as a cautionary tale for other African nations, including Kenya, which is currently finalizing its own National AI Strategy under the Ministry of Information, Communications and the Digital Economy. The Kenyan approach has, thus far, emphasized a multi-stakeholder model involving academia and the private sector, but the South African incident proves that even the most prestigious institutions are vulnerable to the lure of “lazy automation.”
- Data Integrity: The draft policy cited a “2024 Data Privacy Act” that was never actually passed by the South African Parliament.
- Expert Outcry: Over 400 tech experts have signed an open letter demanding a “human-in-the-loop” mandate for all policy drafting.
- Budgetary Impact: The government has already spent R14 million (approximately KES 102 million) on the development of this flawed document.
- Regulatory Lag: The scandal is expected to delay South Africa’s AI regulatory rollout by at least 14 months.
Technology analyst Arthur Goldstuck of World Wide Worx noted that the “hallucination” likely occurred because the drafting committee used a generative AI tool to “flesh out” the academic and legal justifications for their policy positions without verifying the output. This shortcuts the essential process of research and deliberation that forms the backbone of democratic governance.
A Warning for Digital Sovereignty
The implications of this failure extend beyond South Africa’s borders. As African nations race to capture a share of the global AI market—projected to add $1.2 trillion (KES 156 trillion) to Africa’s GDP by 2030—the quality of local regulation will determine whether the continent is a rule-taker or a rule-maker. If policies are built on the shifting sands of AI-generated fiction, the resulting legal uncertainty will drive away foreign direct investment and stifle local innovation.
Governments must now move to establish rigorous verification protocols. This includes the use of “AI watermarking” for internal documents and the mandatory disclosure of AI involvement in policy drafting. Without these safeguards, the digital divide will only widen as public trust in the state’s ability to manage technology evaporates.
The South African Department of Communications has since withdrawn the document, promising a revised version by the end of the fiscal year. However, the damage to the administration’s reputation as a tech-savvy leader may take much longer to repair. For now, the “hallucinated policy” stands as a stark monument to the dangers of technology used without the anchoring weight of human wisdom.
