How to structure an AI centre of excellence in your organisation
Most organisations that are serious about AI make the same structural mistake: they create a team, give it a name, and assume the rest will follow. Eighteen months later, that team has produced several proofs of concept, a governance framework nobody reads and a growing backlog of requests it cannot fulfil. The business is frustrated. The AI team is exhausted. And the board is asking why investment has not translated into commercial results.
The problem is not ambition. It is architecture. An AI centre of excellence built without a clear operating model becomes either a bottleneck or a vanity unit - and most organisations discover which one they have built far too late.
This article sets out how to structure an AI centre of excellence that actually delivers. Not a theoretical model, but a practical framework covering mandate, composition, operating model and governance. If you are a business leader preparing to formalise your AI capability - or trying to fix one that is already struggling - this is where to start.
Get the mandate right before you hire anyone
The most common failure point is not talent or tooling. It is the absence of a clear, board-level mandate that defines what the AI centre of excellence is actually for.
There are three legitimate mandates, and they require different structures. The first is a centre of excellence as an internal consultancy - it exists to advise business units, assess opportunities and set standards. The second is a centre of excellence as a delivery function - it builds and deploys AI systems directly. The third is a hybrid model, which combines both but requires explicit boundaries around when the team advises versus when it owns the work.
Without this clarity, you get a team that is pulled in every direction. A consumer goods business in the £800m revenue range - the kind of organisation that has real analytical capability but no formal AI function - will typically see its data science hires spend 60–70% of their time responding to ad hoc requests from commercial teams. Not because those requests lack value, but because nobody decided in advance what the team was there to do.
Before you write a job description, answer three questions at board level:
- Is this team accountable for AI delivery, or for AI enablement?
- Which business outcomes will it be measured against in the first 12 months?
- Who does it report to, and does that person have the authority to protect the team's time?
The answers will shape everything else.
Build for cross-functional reach, not technical depth alone
A common misconception is that an AI centre of excellence is fundamentally a technical team. It is not. Technical capability is necessary but insufficient. The teams that deliver commercial results are structured to bridge data science, engineering, product thinking and domain expertise.
A workable composition for a mid-market organisation (£500m–£1.5bn revenue) at initial build-out looks like this:
- A head of AI or chief AI officer - accountable for strategy and stakeholder relationships
- Two to three applied AI or data science leads - capable of working across model development and business problem framing
- One ML engineer or AI infrastructure specialist - responsible for deployment and integration with existing systems
- One business analyst or product manager embedded in the team - responsible for translating commercial problems into technical briefs and back again
- A data governance or ethics lead - ideally part-time at this stage, but present from day one
The business analyst or product manager role is the one most organisations skip. It is also the role most responsible for whether the team's work gets used. Technical teams that operate without this bridge tend to build things that are technically impressive and commercially irrelevant.
For organisations using third-party AI infrastructure - whether that is a platform like Eclipse for agentic orchestration, or an LLM-enabled reporting layer like Quantsole - the internal team's job is not to rebuild what already exists. It is to configure, govern and adapt those systems to the specific commercial context. That changes the hiring profile considerably.
Choose an operating model that scales without becoming a bottleneck
The operating model determines how the centre of excellence interacts with the rest of the business. There are two dominant models: centralised and federated.
A centralised model places all AI capability in one team. Business units submit requests, and the team prioritises and delivers. This works well in early stages because it concentrates scarce expertise and enforces consistency. The risk is that it cannot scale. As demand grows, the team becomes a queue.
A federated model distributes AI capability into business units, with the centre of excellence setting standards, building shared infrastructure and providing specialist support. This scales better but requires the centre to have genuine authority over standards, or you end up with fragmented approaches and incompatible systems.
The right answer for most mid-market organisations is a phased transition: centralised for the first 12 to 18 months while the team builds credibility and establishes patterns, then federated as embedded capability grows in individual business units.
A private equity-backed retail group, for example, might start with a centralised model under the CDO function, delivering two or three high-visibility use cases - demand forecasting, markdown optimisation, customer churn prediction - and then use those as templates that commercial and operations teams can run themselves with guidance from the centre. The centre shifts from doing to enabling. That transition requires deliberate planning, not organic drift.
Governance is not bureaucracy - it is the mechanism that creates trust
AI governance sits at the intersection of risk management and commercial performance. Organisations that treat it as compliance theatre - producing policies that do not influence decisions - are storing up problems. Organisations that treat it as a barrier to deployment are leaving value on the table.
Effective governance for an AI centre of excellence covers four areas:
- Model risk - how models are validated before deployment, how they are monitored in production and what triggers a review or rollback
- Data governance - who owns which data assets, what can be used to train models and how third-party data is managed
- Use case prioritisation - a transparent process for deciding which projects the team takes on, based on commercial value, feasibility and risk
- Ethics and accountability - clear ownership of AI decisions, particularly where those decisions affect customers, employees or regulatory obligations
The governance structure should be light enough to move at the pace the business needs, but formal enough that decisions are recorded and defensible. A monthly AI steering group with representation from legal, commercial and technology, chaired at director level, is sufficient for most organisations at this stage. The danger is when governance sits entirely inside the AI team - the team ends up judging its own work, which removes the external check that makes the whole system credible.
What good looks like at 12 months
Getting the structure right is a foundation, not a destination. At 12 months, a well-structured AI centre of excellence should have delivered at least two production AI systems that are being used daily by business teams, established a clear pipeline of future work with commercial owners attached to each initiative, and built enough internal credibility that business units are coming to the team with problems rather than waiting to be convinced.
If none of that is true, the issue is almost certainly structural: unclear mandate, missing bridge roles, a governance model that is slowing deployment, or an operating model that has made the team invisible to the people it is supposed to serve.
The cost of getting this wrong is not just wasted headcount. It is the opportunity cost of 18 months during which your competitors are compounding AI capability faster than you are. The gap between organisations that are doing this well and those that are not is widening every quarter.
If you are at the point of building or rebuilding your AI centre of excellence, Rodan offers a structured diagnostic engagement that assesses your current capability, identifies the structural gaps and produces a concrete operating model for your organisation. Speak to us before you hire.
