NHS AI Lab has released a short buyer’s checklist to guide organisations through procuring AI solutions. It’s been published in light of COVID-19 – the pandemic presents a huge opportunity to accelerate the uptake of innovative digital solutions at scale. In the UK, healthcare organisations are receiving multiple proposals for AI healthcare applications. This could ease the burden on hospitals facing enormous demands on time and clinical resource.
The guide focuses on providing assurance that any AI technology meets the highest standards of safety and efficacy. We recommend all healthcare companies with an AI offering familiarise themselves with the guide, so they’re prepared for likely questions from NHS buyers.
The guide discusses 10 questions a buyer should ask itself during AI procurement:
1. Is AI the right solution for the type of problem that needs to be solved?
- Healthcare organisations are encouraged to work with practitioners and operational staff to define the use case and availability of data for any proposed AI solution. This guide from the Government Digital Service and the Office for Artificial Intelligence may give guidance on developing a viable business case.
2. Can this technology be procured through a transparent, fair and competitive process?
- Like any technology, an AI solution will need to be procured through a recognised public procurement route.
- Guidance from the Office for Artificial Intelligence sets out several specific considerations and innovative approaches to procuring AI.
- The proposed technology may be in use elsewhere in the health and care system so it is worth contacting the NHSX AI Lab to provide connections to other organisations potentially using the same or a similar AI product
3. Can this product do what it claims it can?
The checklist further lists potential questions that a buyer could ask to ensure that the AI can evidently carry out its intended purpose, some of which are:
- What are the product’s performance metrics?
- Has the training data been chosen fairly and is it representative of your patients/service users?
- Has the vendor performed tests on a validation dataset?
4. Are users of this product primed to use it?
- A common failure point for AI adoption is a lack of consideration towards changes in the end user’s workflow.
- Widespread practitioner and operational support from an early procurement stage is critical to successful implementation.
5. Does this product meet regulatory standards?
- Approval for health and social care / community care research in the UK from the NHS is facilitated through the Integrated Research Application System (IRAS). For clinical research projects, the relevant Health Research Authority (HRA) Research Ethics Committee for medical devices research must grant ethical approval.
- If a product directly supports the national COVID-19 response, NHSX AI Lab will simplify the regulatory process by joining up key regulators to create a single gateway for AI products.
6. What data protection policies need to be in place to comply with the organisation’s information governance policy?
- The data access and storage implications of implementing an AI technology will likely require putting in place an information sharing agreement and undertaking a data protection impact assessment.
- A prospective buyer should investigate what anonymised data it may need to supply back to the vendor, to support their plans for ongoing monitoring.
7. What agreements should be put in place to protect any IP generated by your organisation through the use of the AI product?
- Intellectual property may be created that increases the product’s value through to use of the AI product.
- If this is the case, a buyer can take advice and guidance from ailab@nhsx.nhs.uk to ensure that the buyer’s organisation secures an acceptable commercial agreement.
8. Does the organisation have necessary storage and computing requirements?
- The data-heavy nature of AI requires extensive storage and computing power.
- The additional costs associated with support from your organisation’s IT personnel should be considered alongside the costs of the product itself.
9. Will your existing systems work effectively alongside the new technology to ensure a clear and reliable workflow?
- New technology needs to work alongside existing systems, to ensure safety and efficiency.
- Back-end integrations are essential for ensuring a clear and reliable workflow.
10. Can you manage the maintenance burden of this new technology?
- The healthcare organisation making the procurement should have in-house or outsourced technical personnel to manage downtime and future performance.
