2 min. Read

UX RESEARCH

Evaluating Device For Medication-Free Depression Treatment

DESCRIPTIONThis project aimed to find usability issues with first-time users of Flow, along with product impressions by utilizing mixed methods.

My ROLEI took part as project leader. My tasks included monitor project progress, resolving issues, communicating effectively with team and stakeholders.

Let’s begin

Introduction.

Evaluating Setup Wizard & Initial Product Impression

Flow is a medication-free treatment for depression comprising a brain stimulation headset and therapy app. It is the first approved treatment of its kind in Europe available to buy and use at home. The stakeholders for this project were interested to know how the initial product setup and usage was perceived by target users.

My Role.

Project leader

My role as project leader involved to monitor project progress, resolving issues, and communicating effectively with team members and project stakeholders. Moreover, I had the responsibility to create and manage the survey and booking systems, compile and clean the survey data and participate as test leading in the usability sessions.

Responsibilities:

  • Coordinates the use of development resources for the project

  • Operational responsibility for the implementation of the project

  • Follows up projects continuously with regard to time, finances and work performed to project stakeholders

Method.

Participants

In the study that was conducted on a college campus involving 40 participants, the mean age was 27 years. There were 18 males and 22 females. The recruitment for the study was by flyers on university campus. The participants could scan a QR-code which lead them to our booking system. When booking a time-slot for the usability session, they also had to agree to the ethical considerations that the study involved. They were also encouraged to read about Flow and its usage on the company website.

Test procedure

The participants were introduced to the study and had to agree on the ethical med medical considerations that the study involved. The participant were then into the test room where they had a computer with the instructions and survey questions, a smartphone and the Flow device. The test leaders could observe the participant through a one-way mirror.

Data collection & analysis

The data collected consisted of ten questions before product interaction and five questions after. The survey also contained a multiple-row text input for free writing the experience. The participant answered the questions on a visual analog scale (VAS), ranging from 1 (lowest score) to 5 (highest score). Example questions from the survey:

  • How clear were the instructions for applying the product?
  • How did you experience the simulation?

  • How sensitive do you consider yourself to be to sensory impressions?

  • Based on the information from the website, what is your attitude to the product’s user safety?

Some participants stayed after the test to discuss their experience. These insights were classified as an unstructured interview. The data collected from the survey were exported as a CSV file and later cleaned for qualitative analytics. The free text answers were thematically coded into categories presented in the next section.

Results.

Quantitative Data

The quantitative data were cleaned and analyzed using SPSS statistical software. Our aim were to seek correlation between our dependent variables. We also found some minor correlations between some questionnaire items, for example, item 2.4 and 2.5.

Questionnaire items 2.1 – 2.5 (post stimulation)

2.1 2.2 2.3 2.4 2.5
Mean 2.80 4.40 3.38 3.61 2.91
SD. 0.92 0.61 0.93 0.86 1.17
Minimum 1.20 2.60 1.00 1.50 1.00
Maximum 5.00 5.00 5.00 5.00 5.00

2.1: How did you perceive the stimulation?
2.2: How clear were the instructions for applying the headset?
2.3: How effective was the camera function when applying the headset?
2.4: What was your overall impression of the product?
2.5: How likely would it be to buy this product (for about 5,000sek) for treatment?

Correlation Plot, item 2.4 & 2.5

Qualitative Data

The qualitative data were thematically coded and put into categories of unpleasantness during stimulation, lack of instructions, product impression and willingness to pay. Here we gathered some user quotes. See the example below:

Thematic overview

Theme Code Sample
Count Selected Quote
Stimulation Unpleasantness, Pain, Itching, Comfort, Burning, Mark. 14 “It did feel a stinging and scorching sensation from the pads, especially when the current increased and decreased”.
instructions Sensors, Placement, Misunderstanding, Clarity, Procedure. 9 “The camera function showed the headset hovering way above my head. The positioning was not aligned correctly”.
Product impression Phsyical Sensation, Mental Sensation, Positive Sentiment, Reserved Sentiment. 10 “The presentation and packaging are nice. I feel much refreshed after the stimulation!”
Desire to buy Pricing, Medical Alternatives, Scientific Results, Price [High], Price [Low] 7 “Expensive to buy for myself. Would like to try at my  healthcare center with at a higher strength.

 

“I was surprised that it [the stimulation intensity] went to the maximum level immediately. As depression often coincides with higher anxiety, it can be difficult to get depressed people to expose themselves to an uncomfortable electrical stimulant. Even if they are okay with the stimulant, the fact that you can not control the intensity and that it goes to 100% intensity will immediately increase bad feelings and discomfort. Test leaders should prepare the patient for maximum intensity when the participant / patient is allowed to control the level himself. Controlling the level of intensity yourself will alleviate anxiety about the situation when the patient does not have control over the intensity. ”

Participant #17, on unpleasantness during stimulation

Design Suggestions

Reflection.

What could be improved?

Having unsupervised user testing was beneficial in terms of logistics. Moreover, having an unsupervised user test could arguably be better for ecological validity, as participants may feel intimidated by the present of the experiment leader. However, I am certain that we missed valuable pain points and issues by not including direct observations. These could have provided more nuances to the data.

Special thanks to the project team 🙌

Alexander Edin, Love Zachrisson, Tofte Tjörneryd