A housing association tenant reports damp. The system logs it, scores it low, allocates it as a routine repair. The case is closed before any human sees it.
No one did anything wrong. The system did exactly what it was designed to do, ten years ago, built for asset management, not tenant welfare. The urgency thresholds were set by people who have long since left. No one thought to revisit them when the organisation added an AI-assisted triage layer. Why would they? The system appeared to be working.
The tenant was vulnerable. The system did not know that, because vulnerability is recorded in a free text field, and free text fields do not feed algorithms. So a real person, with a real need, slipped quietly through a gap that nobody knew existed.
This is not a story about a bad policy. This is a story about invisible decisions: decisions being made in the background, in the gap between what governance says on paper and what actually happens in practice.
The Charity Digital Skills Report 2026 found that 79 per cent of UK charities were using AI in some form, most of it informal and ungoverned. The Charity Commission's own Trust in Charities research found that just 3 per cent of trustees said their charity was using AI at all, rising to 8 per cent among larger charities.
That gap, 79 per cent of charities using AI and 3 per cent of trustees knowing it is happening, is not a technology problem. It is a governance problem.
Charity Excellence's Future Charity Report, published in April 2026 and drawing on surveys of over 220 UK charities, found that all three board-level AI governance controls measured were rated Red across the sector. Not amber. Red. Strategic assessment of AI's impact, allocating trustee responsibility for oversight, and ensuring organisation-wide awareness and compliance: none of these are in place for most charities. Ian McLintock of Charity Excellence put it plainly: "AI is already embedded in day-to-day charity work, but much of it is happening quietly, with many boards not really knowing what's in use or why. The risk is not the technology itself. It is using it without clear human control, transparency and accountability."
The problem is not unique to charities. The Public Law Project's Tracking Automated Government (TAG) register, updated to November 2025, records 55 algorithmic tools used by UK public authorities to make or inform decisions, 10 of which make decisions affecting people's legal rights or entitlements. Critically, 83.6 per cent of those tools were only uncovered through Freedom of Information requests. They were not disclosed proactively. The DWP's fraud detection algorithm, for example, is on the register and rated low transparency. The National Audit Office has noted "an inherent risk of bias" in its use, while the DWP's own capacity to test for that bias is constrained by gaps in demographic data.
In housing, data from Demystifying AI for Social Housing (DASH) found that nine in ten housing organisations had experimented with AI during 2025, but fewer than one in five had embedded formal governance or training around its use. AI is already shaping tenant interactions, complaint handling and repairs prioritisation. The Housing Ombudsman's position is unambiguous: the use of automation does not dilute an organisation's duties to tenants. It sharpens them.
The pattern is consistent across sectors. The technology is running. The governance is not keeping pace.
Most AI governance conversations start from the wrong assumption: that the failure space is listable. If we identify the risks, build the register, write the questions, and run the workshop, we will have governed the thing.
This treats AI governance as a tame problem, when it is in fact a wicked one. The failure space of an AI system is not finite. Capabilities emerge that nobody designed. Systems trained on historical data run on in contexts that have changed completely. The people who set the original parameters are long gone. As Stafford Beer observed, the purpose of a system is what it does, not what it was intended to do. And what it is doing is often not what anyone is looking at.
This is why aiming governance at the document level misses the point. You can have a policy, an ethics framework, a risk register and a board paper, and still have no visibility of where the critical decisions in your organisation are actually being made.
The first is legacy systems without review. Most public sector and charity organisations are running technology implemented ten or more years ago, by people who have since left. The logic baked in, the thresholds, the weightings, the rules, was designed for a different context. Adding AI on top does not fix that. It accelerates it.
The second is risk aversion that has become an identity rather than a strategy. Organisations that have lived through serious failures in safeguarding or public trust often build cultures of caution that, while understandable, actively suppress the curiosity and challenge that good governance requires. If questioning the system feels professionally risky, people will not do it.
The third is the secret workaround. The IT director writes a policy prohibiting generative AI use. Staff email work to personal laptops, run it through ChatGPT, and email it back. They are not being reckless: they are trying to do their jobs in an environment that has given them neither sanctioned tools nor real guidance. On paper, governance looks intact. In practice, exposure is increasing and nobody is watching.
None of these are failures of intent. They are failures of visibility.
I use a simple test: if I cannot explain how an AI-influenced decision is different for a specific person doing a specific job on a Tuesday morning, not in theory but in practice, I have not understood the governance problem yet.
Good governance asks: who is the system seeing, and who is it missing? What assumptions are baked in, and when were they last reviewed? Who actually has the power to intervene when something goes wrong, not who the org chart says has the power, but who can genuinely push the stop button? And if someone raises a concern, does it go back through the same system that produced the problem in the first place? Because if it does, you do not have governance. You have a loop.
These questions belong to everyone in the organisation, not just the governance team. Research from MIT CISR, published in early 2026, found that companies with AI-savvy boards averaged a return on equity 10.9 percentage points above their industry average, while companies with non-savvy boards averaged 3.8 points below. The study was of large companies rather than charities or public bodies, but the underlying logic holds: organisations where leadership actively understands and shapes AI governance outperform those where it is delegated away. Governance is a team sport, and it requires a culture in which people feel genuinely able to ask hard questions and expect a real response.
If you are not sure where your organisation stands, the AI Governance Readiness Diagnostic is a practical starting point. It covers 18 questions across four areas: foundations, governance framework, assurance, and maturity. It takes around 15 minutes and generates a personalised report with recommendations.
It works as an individual exercise. It works considerably better when a leadership team or board completes it independently and then compares answers. The gaps between responses reveal more than any single score does.
There is also a five lenses template on the same page, a self-guided tool for walking any specific AI use case or decision through five structured questions. If you have a system you want to pressure-test, this is a practical way to do it without needing a consultant in the room.
For organisations that want to go further, uptakeAI runs an AI governance workshop specifically designed for leadership teams who need to move this from conversation into practice.
The question is not whether your organisation has AI-influenced decisions. It does. The question is whether you can see them clearly enough to govern them safely.
Rachel Jannaway is the founder of Jannaways and an associate with uptakeAI. She works with purpose-led organisations on governance, change, and AI adoption.
This article was co-created through a human-led process using several AI models as thinking partners. It reflects our commitment to ethical, transparent, and accountable use of AI, where human judgement, curiosity, and oversight remain central.