Usability testing by itself is not an easy task. As a usability expert and a designer, you spend a lot of time to understand the challenge, create hypotheses, and prepare for the study. And during the study, you try to make the participants feel comfortable and at ease, while, in reality, you are trying to get into their deep thoughts and emotions to understand what they really feel about a certain project or a process. It is important to understand the people, their mentality, and mindset in order to create a comfortable setting and get the most valuable outcomes.
So, conducting a usability test in a different country, with participants from different cultural, educational, and even linguistic backgrounds, can turn out to be even more challenging, as it includes much more research into cultural traits, nuances, and etiquette. It also includes some extra steps of preparation. The following tips will guide you through preparing for and conducting a usability test in an international setting, given you have the usability testing basics covered.
1 – Initial Research
The first step (as when approaching any project) is research. It is important to understand the culture of the country where you are planning to conduct a usability test: what is the official language; what are the main cultural traits and general mindset, etc. If you are doing a test for a website, for example, you may want to look into the existing norms in that country, as it will give clarity to the participants’ responses.
Example: Once, our team conducted a usability study for a website in South Korea. The website was quite simple and minimalistic – nothing unusual for western cultures. However, for Koreans it was “foreign,” “unfamiliar,” and even “Google-like,” given that in Korea they don’t use Google but instead Naver, which covers their search, maps, community forums, etc. They are mostly used to websites with a lot of text, images, and information. Knowing this allowed us to understand better why we received such feedback.
2 – Finding a facilitator
It is generally a good idea to find a supporting party from the host country. If you work in an international company, collaborate with the local teams (e.g. market research, experience design), if applicable.
Alternatively, find a market research agency that may help with selecting participants, providing the space, an interpreter, and general facilitation. In addition, having someone the participants can related to, can help to make the interaction smoother.
Example: In one research that our team had to conduct in Poland, a local market research agency supported us by providing the proper space for usability testing (with a double-sided mirror), selecting the participants using our criteria and screener, taking care of the legal paperwork that the participants had to sign, as well as providing a very experienced interpreter. This collaboration greatly facilitated our experience.
3 – Selecting participants
Screener: When designing a screener, one important thing to keep in mind is the cultural trait of providing feedback. In some cultures, people are more submissive and not used to sharing their opinions; so, the usability testing may be more challenging, as their responses will generally be positive or neutral but without constructive feedback. So, in your screener, ask questions that require feedback as an answer.
Another thing to consider is the language. At times, the participants may exaggerate their language skills, if English, say, is required. In this case, test their language skills during the screening process.
Participant selection: Schedule interviews way in advance, as the scheduling will be challenging on distance. Here as well, the local facilitator can greatly help. Schedule extra participants in case of no-shows.
4 – Drafting a discussion and/or observation guide
Draft a discussion guide and make sure to run it by someone familiar with the host country customs.
I don’t believe you need a translation of the guide, unless some of your colleagues in the host country may need that. Make sure to have extra copies of the guide with you to give to the observers and, most, importantly, to the interpreter to review before the interviews.
5 – Selecting an Interpreter
A good interpreter will make the sessions go smoothly and be extremely fruitful; so, from my perspective, it is worth the time and investment to find a good one.
The interpreters can translate the conversation consecutively or simultaneously. In the former, they wait for the participant to provide feedback and translate afterwards; in the latter, they whisper the translation into your ear, as the participant provides feedback. In my experience, the simultaneous translation was better, as it didn’t slow down the conversation, and I was able to follow the participants’ facial and gestural expression along with their feedback.
It is important to make sure the interpreter translates the conversation fully without paraphrasing.
6 – Selecting the Space
You must either find a room to set up face-to-face usability tests or, in case of doing a contextual inquiry, set up the conversations in the participants’ proper environments. In case of the latter, make sure to structure the study in such a way that the participants properly dedicate time to the study and enough space for the moderator and all the observers.
It’s quite a big endeavour to conduct usability testing internationally. However, it is also very rewarding, since not only you gather the essential feedback, but you also open up your perspective to cultural differences that you can always bring into your designs and solutions. Being culturally open, aware, and thoroughly prepared will guarantee that your study will go smoothly with very fruitful outcomes. I would love to hear about your experience of dealing with cultural differences, and how has that affected your perspective as a designer. Leave your comments below.