Prompted Reflection methods and their flaws when being conducted remotely.
A recent round of user research got me thinking about the advantages of interviews and prompted reflection being conducted remotely… but my latest remote sessions weren’t all plain sailing.
I’m still fresh to the world of user research and UX design, however, this hasn’t stopped me from needing to adjust to the wild, Covid-19 driven ways of 2020. Especially since beginning my MA in User Experience Design. Something that isn’t new to me is conducting user research. And thankfully, remotely conducted user-research and user-testing have been successfully conducted for decades. So, what I imagine to be all of the painfully tedious and repetitive years of trial and error to develop and refine new tools and methods have been tackled already! And then it hit me… remote user research already exists for a reason… it’s undeniably faster and easier to co-ordinate. Or is it?
With moderated remote research, there is no physical location to arrange, no transport costs to cover, or laboratories to set-up. Due to their nature, each session is already contextual- a beneficial element that would have added more layers to the organizational process for non-remote research. As outlined in an article by the Interaction Design Foundation. ‘ …observing users in their natural environment tends to lead to very accurate information’ (Contextual Interviews and How to Handle Them, 2020). Not only are user’s within the environment where the service/product is intended to be used, allowing for any interruptions that might temporarily punctuate their train of thought, environmental factors that might provoke thought and more authentic answers, but most importantly, it allows the user to feel calm and at ease within their natural environment- that’s undisputedly the most important consideration. Contextual advantages aside, remotely conducting interviews has an array of additional benefits over more conventional methods:
1- Participants can be reached from all over the world, diversifying the panel.
2- Tests can be conducted on the participant's own time, leaving them completely in control- that’s always going to go down well with them!
3- Time. Timezones, participants' time, travel time, and your time!
4- Relatively cost-effective.
So, why does remote research only make up 5% of all user research (Asjes, 2014). Well, put it simply, it isn’t all plain sailing. Especially when it comes to prompted reflection methods… as encountered just the other day.
My experience
So, I feel passionately that thorough research cannot be achieved through semi-directed research alone. For this reason, my recent round of interviews involved an element of prompted reflection, in the form of Card Sorting… and that’s where remote research encounters a series of issues.
The form of prompted reflection I used was two rounds of closed, moderated card sorting, which was implemented halfway through each semi-structured interview. I chose this for a number of reasons; I wanted to find out about the participant's perceived values and how they viewed their importance. Something that I felt couldn’t be articulated as clearly through words. As well as a time-line orientated activity that depicted certain emotions and thoughts throughout their routine. I felt this avoided asking repetitive questions, as well as allowed the reflective elements that evoke deeper and more meaningful insights. Card sorting lends itself to a physical interaction… I mean, it’s literally in the name. So where did I run into problems?
The Participant’s Learning Curve
We are spoilt for choice with the abundance of Card Sorting software at our disposal. I investigated a few but just didn’t gel with them. So I stuck with what I knew. Miro.
Miro is a cracking piece of software. Versatile, powerful, and relatively intuitive. I know I’m only just scratching the surface of its capabilities. But it's the relatively intuitive aspect that requires careful consideration when conducting it remotely.
The problem:
Everyone knows how to pick up a post-it note. Hold a pen. Move closer if they can’t quite read something. This knowledge doesn’t quite transfer over to Miro. If your participant is familiar with team collaboration software, Adobe Creative Cloud, or even CAD software, they may adapt quickly. Otherwise, it quickly becomes a barrier.
The barrier isn’t insurmountable. A little bit of careful explaining, patience, and understanding can smoothen the process out. But, it’s still something that wouldn’t occur in person and was an unforeseen hurdle.
It doesn’t end there.
This next problem’s solution will come with time. It’s a problem that I will adjust to as my experience with conducting remote interviews increases. And that is connecting with the participant. Whilst video calls and collaborative software go a considerable distance in creating a personal connection, It’s certainly not a replacement for the real thing. And whilst I once saw myself as being able to read a participant efficiently, remote interviews have changed my perception of this and I can’t help but feel I am falling short.
This is most evident when it comes to predicting the gaps in flow when moving onto the next question. When carrying out the laborious task of transcribing, I am repeatedly cringing at myself for interjecting too soon or interrupting the participant due to wrongly assuming they had come to an end of a response. This is then counteracted with extended breaks in a conversation where I am overcompensating for fear of interrupting, breaking the natural flow, and creating a jarring and unprofessional interview.
Whilst I am certain this will improve over time, going from conducting smooth, flowing interviews to this jarring mess is certainly a challenge. (I’m sure it’s not as bad as I’m making out… nobody likes listening to their own voice on a recording at the best of times!).
What’s next? Ethnographic research. COVID-19 has certainly thrown a spanner in the works for that one…
James
jameshoaredesign.com
Asjes, Kathleen. 2014. The Hidden Benefits of Remote Research. [online] Available at https://www.uxbooth.com/articles/hidden-benefits-remote-research/
The Interaction Design Foundation. 2020. Contextual Interviews And How To Handle Them. [online] Available at: <https://www.interaction-design.org/literature/article/contextual-interviews-and-how-to-handle-them#:~:text=The%20Advantages%20of%20Contextual%20Interviewing&text=The%20detail%20of%20the%20information,out%20wherever%20a%20user%20operates> [Accessed 23 October 2020].
