We began by conducting a qualitative analysis of the SCAN participant datasets, including a post-study survey (which was sent to all study participants) and the support emails that were sent to SCAN. The massive trove of data that SCAN provided allowed us to narrow our scope and determine which parts of the procedure warranted further investigation.

After conducting a thorough analysis of the data SCAN provided, the research team identified 3 problem areas and formed corresponding hypotheses to test during the usability study.

10 participants completed our usability test, which were all conducted remotely over Zoom. The first portion of the usability test asked participants to explain their experience up until that point, which included the enrollment process and kit delivery. Then participants were asked to speak their thoughts out-loud as they completed their kits. Finally, the participant was asked follow-up questions about their swab experience and questions about expectations for getting kit results. About 3-4 days after the usability test, participants were called by one member of the research team. During the short (5-10 minute) phone call the participant answered questions providing a retrospective perspective on the test overall and how they found the process of getting their results to be.

We coded each interview and identified insights and key themes in the data. We used affinity diagramming to group together insights and begin generating recommendations. We based all recommendations on data that came from direct observations of participants.

The research team created a User Journey Map to visualize where the pain-points in the procedure were located. We segmented the user journey into 4 stages: recruitment, sign-up, kit completion, and awaiting results. For each action within each stage, we observed the participants' experiences and coded them as being positive, negative, or neutral.

In addition to the User Journey Map, the team created a digital affinity diagram to help organize all of the information that was collected during the interviews. We first pulled significant and noteworthy observations from out notes and then discussed what insights could be synthesized from them. The final step was to brainstorm data-supported recommendations.

The affinity diagram and user journey allowed us to synthesize a list of key findings that could be used to generate solutions to improve the experience for future participants. The following findings are not exhaustive and represent findings we observed in the majority of participants (5+). We had many minor findings that influenced our recommendations but were not common across all of the usability tests we conducted.

In writing our recommendations, we carefully considered all of the data we collected from the usability tests and the datasets we reviewed. We explored many potential solutions and chose to narrow our recommendations to the ones we considered to be the most effective. We also took into account the amount of work necessary for implementation, so as to not burden the SCAN team.

Most participants did not read all of the Quick Start Guide (picture below). We recommend simplifying the content, reducing the word count, increasing white space, and focusing on images. Renaming the guide "Instructions" would also better communicate the guide's purpose.

Participants did not follow the steps on the guide as written, but many did read all of the guides. This was especially true of the kit registration step. We recommend reducing the number of steps and moving the kit registration step off the guide. Sending an email (asking participants to register their kits) after enrollment could help participants know they should complete that step before opening their kit. The remainder of the steps should remain on the guide.

We suggest using the notifications for confirmations of step completion or reminders for patients suspected of dropping off (reminders). Reducing the number of communication channels would also help reduce complexity, receiving information from multiple emails and phone numbers confuse participants.

Participants do not understand the significance of the kit registration step. 3 participants thought they had already completed the step. Changing the name of the step to "Activation" might help participants understand that the step should be done first. Again, sending the link that is necessary for this step via the participant's preferred method of contact or proving the link via QR code could help reduce registration errors. Below is a diagram showing when participants completed this step vs. when they were supposed to.

Promote the resources available for participants. Create an FAQ page specifically for participants. Further, promote the educational video. These two resources could be accessed by a QR code on the swab kit box, which presently only has the SCAN logo.

In writing our recommendations, we carefully considered all of the data we collected from the usability tests and the datasets we reviewed. We explored many potential solutions and chose to narrow our recommendations to the ones we considered to be the most effective. We also took into account the amount of work necessary for implementation work, so as to not burden the SCAN team.