The Governance Challenge Hiding Inside AI Adoption
Artificial intelligence has hit the headlines this month and there is no denying that concerns over control and governance is making me think Terminator might not just be a sci-fi film. But what does this mean for companies who are not building AI but rather using it across their business?
AI is helping teams analyse information, support customers, generate content and automate tasks. There is a hybrid of AI system integration, agentic AI and informal chatbots every employee uses. Increasingly, we are seeing it influence decisions, coordinate workflows and take action on our behalf.
The pace of adoption is accelerating. But there is a growing gap between the speed organisations are adopting AI and their ability to embed it successfully.
PwC's latest CEO research found that 30% of CEOs said AI had increased revenue over the previous 12 months, while 56% reported no financial benefit. At the same time, only 45% said their organisation had a clearly defined roadmap for AI initiatives.
The gap between adoption and impact is clear. Businesses are joining a race without knowing the destination.
For years now everyone has talked about AI as an opportunity: productivity gains, efficiency, innovation and competitive advantage. Today though, there is a growing focus on responsible AI, regulation, ethics and technical controls. And that has only escalated with recent news stories showing fundamental flaws in AI controls and accountability.
As organisations adopt AI, they are changing how decisions are being made, who has accountability and critical controls that protect your business. This is where the governance challenge lies.
AI is moving faster than organisational readiness
We all know leaders (including governments) are not short of ambition when it comes to AI. KPMG found that 93% of leaders believed their investments in generative AI had enhanced their organisation's competitive position and long-term strategic performance. But the same research found that 82% expected risk management to be their biggest challenge as they developed their AI strategies.
The tension is becoming increasingly clear. AI is here to stay and employees use it daily both at work and at home. Business processes and workflows are being transformed. But the structures that protect against AI dependency and reduce exposure to risks have not caught up.
Deloitte's global survey of more than 2,700 senior leaders found that three of the four biggest barriers to successful generative AI deployment were related to risk: concerns about regulatory compliance, difficulty managing risk and a lack of a governance model.
Some might brush this off as a compliance problem, but in reality this is business strategy and protecting long-term resilience.
The biggest issue is that AI adoption is well underway, and governance is now reacting and retrofitting. This blog will help you get ahead faster.
Mapping the operational framework
When we speak to businesses about AI, terms like transformation, continuous improvement, agile and Lean, always come up. In the haste to adopt AI, a clear operating framework is not always in place. And this must be the first step in building governance of AI.
Businesses need to get clarity on:
What is the purpose of AI use in the business?
What positive and negative changes will happen as a result of AI adoption?
What metrics will demonstrate these AI impacts and how do we monitor and report these?
Who is accountable for security and data protection, intellectual property protection, confidentiality and trade secret leakage through AI use?
What AI technology is being used where, when and by who?
Who is designing the learning content and responsiveness of AI, and what quality assurance and bias checks are in place?
How will the workplace change and how do our people adapt to AI enabled work?
Who is accountable for decisions influenced by AI?
What authority has effectively been delegated to an automated system?
Where does human judgement remain essential?
Who can challenge or override an AI-enabled decision?
What happens when something goes wrong?
These are not technology questions, they are questions about decision rights, accountability and organisational control. Organisations have decades of experience governing people, functions and business processes. AI introduces a new layer.
The hidden delegation of authority
One of the most important changes created by AI is the gradual delegation of authority. This often happens incrementally without a conscious decision.
A team begins using AI to prioritise work. Another relies on it to assess risk. Customer services automate responses. A workflow is automated. Individually, each case appears sensible and even natural. However collectively they can change where decisions are being made and how much control you retain.
This is becoming more significant as organisations move towards agentic AI. KPMG research in 2025 found organisations rapidly progressing from exploring AI agents towards piloting and deploying them. By the third quarter of 2025, 42% of surveyed organisations reported deploying at least some AI agents, compared with 11% two quarters earlier.
This is where governance needs to become more explicit. Organisations should understand and document the level of authority they are giving an AI system. Leaders need to be comfortable with where AI sits in the Authority Matrix.
There is a meaningful difference between AI that:
assists by providing information;
recommends a course of action;
acts with approval from a human;
or acts autonomously within defined boundaries.
Governance frameworks, normally through an Authority Matrix, must include all four levels, and then have sufficient controls, metrics and oversight in place for each one. Similar to what you would have in place across other areas of the business, like financial management.
Map decisions, not just technology
We are starting to see many organisations create inventories of AI systems, both officially adopted and those adopted by employees informally. This is an important step but an inventory of technology is not necessarily an inventory of influence and impact.
Knowing where an AI tool exists tells leaders relatively little about how it is affecting the organisation. A more useful approach is to understand:
what decisions AI influences;
what actions it can trigger;
which processes depend on it;
who is accountable for the outcome;
what data it relies on;
which external providers or models it depends on;
and what happens if the capability becomes unavailable.
Dependency on AI is something so many organisations are not talking about. We know that AI outages are not unheard of. Just this year, during heatwaves Google and Oracle struggled to cool their data centres which led to outages. Can your organisation continue when AI is down?
To answer that questions, you need to ask:
Where are we becoming dependent on AI?
What processes will be impacted if AI is down and what impact does this have on our ability to operate?
What happens if the technology is unavailable, unreliable or compromised?
What is the financial implications of an AI outage? (e.g. will invoicing be delayed, service quality reduced)
AI governance must include business continuity planning for business-critical processes that rely solely on AI. The risk of outages as well as cyber attacks can’t be ignored. This is particularly critical when AI impacts quality, revenue and profit margins.
Accountability cannot be delegated
When an AI-enabled process produces a poor outcome, the technology cannot be accountable. Customers, regulators, employees and other stakeholders will still look to the organisation using the AI not the technology itself. That is incredibly important, especially when AI is working with personal data or being used to deliver outputs to customers.
Yet accountability can become blurred when decisions are distributed across people, technology platforms, data sources, suppliers and automated systems. AI is a complex system without one clear owner. With system architects, designers, trainers, end users, everyone may own part of the problem, while no one owns the whole system of consequences.
That is why AI governance is not simply a technology exercise. It is an organisation-wide governance challenge. It is essential that each element of AI integration and use is mapped, with clear accountability. A RACI (Responsible, Accountable, Consulted, Informed) is a really helpful tool to ensure clear roles and responsibilities for every part of AI design and use.
Human oversight needs to be meaningful
"Human in the loop" is one of the most commonly used answers I hear when we talk about AI risk. But human involvement is not the same as meaningful human oversight (as we have seen in recent news stories about agentic AI escaped a test environment and it took humans months to notice!).
Human oversight is critical and it must include:
sufficient understanding to challenge the output.
access to relevant information in real time.
sampling and quality checking outputs.
monitoring AI impact metrics regularly.
genuine authority to override the system.
time to exercise judgement.
Human oversight is only meaningful when we clearly know the purpose of AI, how we monitor impacts, and we have the authority to override and disable the system. Without clear boundaries it becomes difficult to manage something that has no agreed scope. I would also argue that one human having oversight isn’t enough because personal bias, experience and risk appetite may unknowingly create exposure. This is where embedding AI governance into the wider governance framework really has benefits.
Governance should be proportionate
One of the things our clients really value is our focus on proportionality. And that is important here too. Not every use of AI requires the same level of governance. An internal tool that helps summarise meeting notes is different from a system that makes or influences decisions like:
customer outcomes;
employment and hiring decisions (we’ve all seen those LinkedIn posts!);
financial decisions;
access to services;
critical operations;
or decisions involving sensitive data.
The greater the potential consequence of an AI-enabled decision, the greater clarity is needed. Particularly around:
accountability;
oversight;
testing;
transparency;
human judgement;
and intervention.
Proportionality is critical to ensure governance is useful, responsive and valuable. Some leaders complain governance is a barrier or slows decision-making. Ensuring the framework you build is proportionate, agile and useful will increase effectiveness and acceptance from those leaders.
Getting Started
AI is complex and knowing where to start with governance can be challenging, especially when AI is already operational in your business. This is what we would do if you asked ENVOLV to build your AI governance framework:
Map the operational framework.
Detail AI enabled vs AI automated decision-making.
Build a RACI for every element of AI design and use.
Assess AI risk considering every impact on business strategy and stakeholders.
Build an AI risk register with clear impact metrics that should be monitored.
Design AI controls, this will include:
Adapting current organisational policies, employee handbook, supplier code etc.
Designing an AI policy covering business and personal employee use.
Employee upskilling and education.
Establishing business continuity planning.
Embed AI into the governance architecture where human oversight is already established and effective e.g. your Risk and Audit Committee.
Agree timelines, reporting frequency and escalation processes for issues.
Embed AI into the wider governance architecture
Once you understand AI impact, risks and accountability, embedding AI governance into your existing framework is critical. One of the biggest mistakes organisations can make is to treat AI governance as a separate requirement. I have seen this in organisations where sub-committees and splinter groups are set up for AI, when there is already sufficient risk committees and executive leadership committees in place. If your organisation-wide governance structure is still in it’s infancy our guide will be useful.
Without doubt, AI will be an enterprise risk. Either impacting how you operate or the market you serve. It will impact cyber and data protection risks, operational resilience, workforce capability and capacity. It isn’t just about how it’s being used internally, but how it is disrupting your market and impacting how your customers buy. It therefore should show up in multiple parts of your risk register, strategy and operational plan.
The challenge is not simply how to govern AI. It is how to govern an organisation that is increasingly impacted by AI. That requires organisations to think beyond principles, policies and technical controls. It means examining whether decision rights remain clear, accountability is understood and human oversight can be clearly demonstrated. It is ultimately a question of organisational design.
AI is introducing new actors into the organisation's decision-making environment. They aren’t on your organisation chart. The chances are they are owned by a supplier. They hold little to no accountability but they influence information, recommendations, priorities and actions across your organisation. Governance is so critical because all of this results in control moving into a space we haven’t seen before.
The organisations that succeed with AI will not be those that adopt it fastest. They wont have the longest list of AI principles or the most restrictive controls. It will be organisations with clear operating frameworks, accountability maps, impact metrics and continuous human oversight.
At ENVOLV we have deep expertise in building effective governance frameworks that don’t restrict growth but future-proof organisations. We have already built AI governance frameworks and advise clients daily to ensure they are future-fit. To find out more get in touch.