Usability Testing
Introduction
Usability refers to the degree to which healthcare technology can be seamlessly adopted within existing clinical workflows, enhancing both user efficiency and patient engagement.
Usability considerations during procurement should inform vendor selection and decision making to ensure acquired products, technology or software meet clinical needs, fit into established workflows, and comply with industry regulations. Therefore, when usability is prioritised, systems and applications are more likely to be efficient, effective, and satisfying for the user to operate, even in high-stress settings.
Incorporating usability in the procurement process
The Plan-Do-Study-Act (PDSA) cycle is an excellent framework to integrate usability testing and accessibility requirements into the procurement process.
This is an example of how it can be structured.
Plan
- Define the usability and accessibility targets of your organisation.
- Ensure to align usability goals with clinical needs and standards (e.g., Web Content Accessibility Guidelines - (WCAG) compliance).
- Collaborate with end-users, Human Centered Design resources, and technical teams to understand detailed requirements.
- Include usability testing and accessibility as non-negotiable "must have" requirements in the Request For Proposal (RFP).
Do
- Conduct usability and accessibility testing with shortlisted vendors, focusing on actual scenarios provided by end users.
- Collect testing results and feedback from users, including clinicians and administrative staff.
Study
- Review results from usability testing.
- Assess vendor's performance based on usability testing.
- Select vendors that meet or exceed usability requirements.
- Discuss identified gaps with the vendor.
Act
- Select vendors that demonstrate strong compliance in usability requirements.
- Include clear usability improvement plans in contract inclusions.
Post Implementation
- Perform ongoing usability testing with users to continuously refine and improve the systems.
- During contract extension, update usability requirements and plans to ensure ongoing compliance.
Conventional usability measurements
Below is a table of common usability measurements which should be included as part of the mandatory requirements in the Request for Quote (RFQ). These metrics are critical for assessing the effectiveness, efficiency, and satisfaction of digital health solutions, ensuring they meet the needs of both healthcare professionals and patients.
| Metric | Definition | Example use |
|---|---|---|
| Task success rate | The percentage of patients or clinicians who complete a health-related task without errors | Measuring if users can successfully generate patient data related reports using Laboratory Information System |
| Time on task | The time taken by users (patients or staff) to complete specific health-related tasks | Evaluating how quickly clinicians can locate patient data in an eMR system |
| Error rate | The frequency of errors made by users during digital health interactions | Tracking mistakes in medication order entry within an Electronic Medication Management system module |
| Satisfaction | Users’ subjective evaluation of their experience | Gathering feedback and surveying patients on the ease of scheduling a telehealth appointment |
| Accessibility compliance | The degree to which health platforms meet WCAG standards, ensuring inclusivity for all | Testing a patient portal's compatibility with screen readers for visually impaired users |
For software procurements it is best practice to engage with the Human Centred Design (HCD), the User Experience (UX) Team or similar experts where possible for guidance and insights. This ensures key aspects are thoroughly reviewed and assessed at each stage of procurement.
Contact us
This content is draft for consultation. To learn more about the Guidelines, the phased publication approach, or if you are interested in being part of future reference groups, please contact us via the form below.