Jul 1 / James Kavanagh

Building an AI inventory is your first governance intervention

Your first meaningful governance intervention is most likely building an AI inventory. It’s much more than an administrative task before governance starts, and it’s harder than it seems.
Yesterday we ran the second workshop of the AI Governance Practitioner Cohort. This one was about building the AI inventory, and especially about what it takes to define useful use cases. I'm grateful for the privilege to share practices and experiences, and together tackle smart questions from a really engaged group of practitioners mastering their craft. 
But let me back up a little.
Our cohort builds on the learning within the four online courses of the AI Governance Foundation Track, working through one complex case study from start to finish. It's a case of AI in a children's hospital, and like a lot of real organisations right now it's caught between the promise of AI and the risk of moving too fast. The first workshop was about a technique for triaging harms, and an exercise to figure out what the hospital should stand for: a method to map out it's principles and commitments. Yesterday we took the next step of mapping out the most relevant AI use cases.
I won't teach the whole method here, we go through it properly in Course 2: Organising for AI Governance, but here's a quick overview.
I've found useful AI inventory has two distinct views. The business view names the use case (the specific situation where you're pointing AI at a business problem), the capabilities sitting behind it, the system that delivers them, the users who feed it and read its outputs, and the stakeholders, including everyone affected who never lays a finger on the thing. 

Business Inventory of an AI System

Then there is a technical view under-the-hood one: the models doing the work, the datasets feeding and flowing through them, the interfaces where the system meets the world, and any agents acting on its outputs

Technical Inventory of an AI System

Two mindsets show up to build an AI Inventory

You see contrary to what you might see in a textbook, I've never found building an AI inventory is a tidy stocktake you do before the governance starts. Building it is one of the first governance interventions. It's often the first time anyone in the organisation has to sit down and agree, out loud, on what is really there and what it's really for. That is an intervention, and it's harder than it looks, because the people who turn up to do it are not starting from the same place and they come with two very different mindsets
You'll probably recognise both of them.
There's the policy, legal and governance folks. They start from harm, and their first question is who could get hurt, how badly, whether the vulnerable are exposed, whether any of this is even lawful, and who carries the can when it goes sideways. I love this instinct, it keeps the actual affected human being in the room. But on its own it tends to run out of road at naming the harm. It'll tell you a vulnerable teenager might be under-served, and then it goes a bit quiet, because it can't tell more specifically what might need to change.
Then there's the engineering, science and ops crowd. They will commonly start from the hazard. What could fail, under what conditions, which component or capability is to blame, and how do we get to a failure mode we can test and watch and fix. I love this instinct too. It's speceifc, it gets you to something you can actually do. But leave it on its own and it'll happily tune a system to perfection and never once picture the person on the other end of a wrong answer.
Neither of them is wrong. They're facing different directions, and each is strongest exactly where the other gets a bit lost. Get them building the inventory together and you're doing governance whether or not the word ever comes up. And if you're the AI governance practitioner in that room, this is your job. Nobody else is standing in the gap between them. It's a lonely spot some days, and it's also the most useful place to be.

This isn't a personality clash, it's two disciplines

It would be easy to read all this as the careful lawyer versus the can-do engineer, temperament against temperament. It isn't. These are two proper traditions, each with decades of serious thinking behind it, and it's worth knowing whose shoulders you're standing on.
The harm-first side grows out of algorithmic impact assessment and stakeholder-centred work like IEEE's Ethically Aligned Design. Have a read of it if you haven't. It starts with asking "how could someone turn this against a person". It's a disciplined approach to tracing harm of use.
The hazard-first side comes from requirements and safety engineering. Two books I'd put in anyone's hands. Alistair Cockburn's Writing Effective Use Cases is the honest guide to writing one well, and his best point is that a use case is really a prose essay, with all the difficulty that writing decent prose brings. It's craft, not a form you fill in. And Nancy Leveson's Engineering a Safer World (which MIT Press put out as open access), makes the case that the old "what caused the accident" model falls apart for complex, software-heavy, sociotechnical systems. Harm comes out of the whole system, not one broken piece. Precisely the world ofyour AI inventory.
So when the two mindsets pull in different directions, that isn't two people or groups being difficult. It's two schools that spent decades learning to see different things. One learned to see the person. The other learned to see the mechanism. You need both of them in the same picture, and nobody sews them together on your behalf. That's your job as an AI governance practitioner.

The use case is what joins them

Write a use case tightly enough and it becomes the one thing both sides can hold at once. From that single use case, the engineer works down, to the capabilities it leans on, the system and its parts, and all the ways they can break. From that same use case, the lawyer works up, to the users, the stakeholders, the affected others, the harms and the misuses. Same thing but described in two directions at once. That's the join. But it only holds if the use case in the middle is written well, and that has to be done quite deliberately.
And honestly, nobody can hand you a method for doing this. There's no ISO standard for defining an AI use case. The EU AI Act won't do it for you. Writing the use case tightly enough to carry both mindsets is your craft to build, and it's exactly what we started on in the workshop.
The tool we used is a single card, one per use case. A title, short narrative. Then the intended and actual use, and you note any gap between the two. Then the system and who owns it, the capabilities it relies on, the users, the stakeholders including the people who never touch it, and last, two or three plausible misuse cases.

Key elements of a Use Case Card

Misuse cases have a way of surfacing on their own once you've laid capabilities against use cases, and the ones that matter most aren't hypothetical at all. If a system's actual use has already drifted past what it was meant for, you're not dreaming up a risk. You're writing down one that's already here. Watch for it.

From a card on the table to a system that runs

None of this is meant to end its life as a nice picture on a wall or a spreadsheet filed away in some Sharepoint drive.
The reason we map this carefully is that it feeds straight into something live. We take the inventory and the use cases the cohort builds and configure them into VerifyWise, an AI governance platform that's built around exactly this shape: an inventory of AI systems and use cases, with the risk, the controls, the evidence, the approvals and the audit trail all hanging off each one. Every participant in the cohort builds their own AI governance platform out in VerifyWise, seeing how the thinking and discussion turns into configured tooling in practice.
That's when it starts to feel worth it, and feel manageable in a sustainable way. The principles and commitments, the identification of harm, and now the use-case work becomes the backbone of something that's up and running. 

Come and do this with us

If this is the kind of governance you want to practise, the kind that takes safety engineering seriously and refuses to treat an inventory as a box to tick, come and do it with us. The next cohort is where it happens, and the details are here: Governance Practitioner Cohort.
Because AI inventory is never really a technical mapping job with a governance sticker on it. It requires you to understand how very different people think, and then to build the the first artefact that lets them understand each other. 
That's the craft, and I'd love to help you build it.
Text