# AI in a regulated organisation: the questions procurement will ask
> The procurement, security and governance questions a regulated organisation must answer before deploying AI — data handling, model risk, audit trails and accountability.
Author: Matthew Ford
Published: 2026-10-04

<p>An AI proposal for a government, healthcare or financial-services organisation needs to address more than the feature itself. Reviewers will want to understand the information involved, who remains accountable and what happens when the system produces an unsuitable result.</p>

<p>The questions below are a starting point for a procurement conversation, not a complete compliance checklist. Ask your organisation's security, data-protection, legal and relevant domain specialists to identify the requirements for the actual use case.</p>

<h2>1. Where would the data go?</h2>

<p>Record each system the information would pass through: application, model provider, region, subprocessors and any monitoring or logging service. Establish the permitted uses, retention periods, access controls and arrangements at contract end.</p>

<p>Check those answers against the selected service, configuration and contract. Terms such as enterprise, in-region or zero retention need supporting detail; do not assume they describe every request, feature or log. Resolve gaps before using real customer, patient or staff information.</p>

<h2>2. What would the AI decide?</h2>

<p>Distinguish drafting or summarising for a person from taking an action that affects someone. State where human review occurs, what information the reviewer sees and how they can reject or correct an output. A person who lacks the time or authority to challenge the result is not a useful control.</p>

<p>Ask the organisation's advisers which rules and approvals apply to the proposed decision. Agree the permitted actions and the access needed to carry them out, rather than giving the system wider authority for convenience.</p>

<h2>3. What evidence would support release?</h2>

<p>Define representative evaluation cases, success criteria and unacceptable errors with the people responsible for the process. Include difficult inputs, missing information and cases the system should escalate. Record how the test data was selected and which limitations remain.</p>

<p>A <a href="/services/rapid-prototyping/">focused prototype or technical experiment</a> can investigate those questions. Its findings should support a decision about further work; they do not by themselves establish that a live service is ready for every workload.</p>

<h2>4. How would mistakes be detected and handled?</h2>

<p>Agree who investigates an incorrect output, how users report a problem and when the service should fall back to an existing process or be stopped. Plan the evidence needed for an investigation, including relevant changes to models, prompts and source information.</p>

<p>Balance that evidence with data minimisation. Logging every prompt and response without considering its contents can create another sensitive data store. Agree access, redaction and retention as part of the design.</p>

<h2>5. Who would own changes and ongoing operation?</h2>

<p>Name the business and technical owners. Agree who approves changes, who monitors the result and how the feature will be withdrawn if it no longer meets the requirements. Include supplier changes and the continued work needed to maintain evaluation data.</p>

<p>Our internal <a href="/blog/building-watchkeeper-ai-security-checks-iso27001/">Watchkeeper security-evidence tool</a> has a deliberately limited role: it reviews reports and flags items for people to investigate. People remain responsible for server changes and baseline approvals.</p>

<h2>6. What evidence can the supplier provide?</h2>

<p>Ask for relevant delivery examples, secure development practices, access arrangements and the scope of any current certifications. Establish what each example demonstrates and which project-specific checks still need to be carried out.</p>

<p>Our <a href="/technical-assurance/">technical assurance approach</a> explains how we discuss testing, security and operating responsibilities. The requirements for your application need to be agreed alongside that general approach.</p>

<h2>7. What would it cost to operate and to leave?</h2>

<p>Include usage charges, integration, evaluation, human review and ongoing support in the operating budget. Test the assumptions at expected and higher workloads. Ask which costs could change if the provider or model changes.</p>

<p>Agree what your organisation can retain or export, including its data, application code and evaluation records, and what is specific to the selected provider. Document the handover and deletion arrangements before they become an exit problem.</p>

<h2>Use the questions to define the next decision</h2>

<p>Record an answer, evidence and an owner for each relevant question. Some gaps may need investigation or may prevent the proposed use; they should stay visible in the decision to proceed.</p>

<p>Our <a href="/blog/ai-readiness-assessment/">AI readiness assessment guide</a> helps frame the initial investment decision. If you need help with that assessment or an agreed implementation, our <a href="/services/ai-automation/">AI consulting and automation service</a> describes the available starting points.</p>
