HOME

Pilot Usability Testing


Introduction

Lessonface, founded in 2013, is a Public Benefit Company whose purpose is to connect students with great teachers for live online music, language, and arts lessons. Their mission is to help students achieve their self-enrichment goals while treating teachers equitably. Lessonface is COPPA (Children's Online Privacy Protection Act) compliant and designed for all ages.

I chose Lessonface as a competitor application because this website displayed many features I envision to include. These include the feature to personally match with a suitable teacher, reviews, and individual pages for teachers with information. However, I would like to focus only on instrument learning, and emphasize more on the idea that everyone is welcome to learn instruments. I would like to use Lessonface as inspiration and to help me decide what to build upon for my final project.

Purpose of Usability Testing

The purpose of a Usability Test (UT) is to evaluate the usablity of the product. A moderator tests and observes a participant (who represents a user) and gathers their feedback. By using a UT, we can learn more about the usability problems on the website we found after conducting a heuristic evaluation. With UT, we can observe in real time a user’s behavior while using the website.

Heuristic Evaluation Recap

Upon conducting heuristic evaluation of Lessonface last week, I found that the three main issues were:

Heuristic Usability Problem
User Control and Freedom Cannot remove instrument category when searching for music lessons, confusion
Consistency and Standards An element that appears to be a button but is not a button
Recognition Rather than Recall Important links under nested menu

I decided to focus on these heuristics for my UT session.

Methodology

Considering these, I used UT and created three tasks to test the usability problems noted above according to the table below:

Heuristic Being Tested Usability problem Task scenario
User control and freedom Cannot remove instrument category when searching for music lessons, confusion Finding a clarinet teacher, then changing to flute teacher
Consistency and standards An element that appears to be a button but is not a button Find the FAQs page and phone number from the top navigation
Recognition rather than recall Important links under nested menu Find the "About" page under the nested menu

Next, I decided on and wrote down the most logical and natural steps to completing these tasks.

Task Scenario 1: Finding a teacher 2: Looking for answers to questions 3: Learning more about Lessonface
Step 1 Search for clarinet teachers Find the FAQs page Find the "About" page
Step 2 Refine your search to Monday, Friday, and Saturday Find the section regarding the cancellation and rescheduling policy Read about the history of Lessonface
Step 3 Refine your search to the Mandarin language Find the phone number Find information about the CEO of Lessonface
Step 4 Change your search from clarinet to flute teachers N/A N/A

The process of the usability test was:

I used Zoom to record the session between me an my participant on my PC, while my participant was sharing his screen through Zoom on his laptop. We were both in quiet places, with myself alone in my room and himself alone in his room. I paused the recording before the demographics and continued after, to thank him for his participation.

UT Materials

UT Survey on Google Forms

Pilot UT Session Video

Findings

Through the UT session, I was able to find out new usability problems and also confirmed that the usability problems I evaluated last week were problematic.

First Task

I mainly focused on the issue I pointed out in last week’s heuristic evaluation:

Problem: Cannot remove category when searching music lessons (Rating: 2) The category filter is designed like a multi-select, with the design of the “X” on the category chosen. However, it is a single-select and you cannot search multiple instruments at once, or remove the category to search all instruments at once.

My participant, when trying to change the selection from “clarinet” to “flute,” tried clicking on the “X” to remove the clarinet selection, but found his knowledge of the design did not match its function. He pointed out that it was confusing.

Second Task

I focused on the phone number’s button-like appearance but found a new problem.

Problem: Not actually a button (Rating: 3) The phone number to contact the establishment is on the top right “navigation bar” (or what is intended to be a navigation bar, but does not include the actual navigation links—will be elaborated upon). However, it appears to the user as a button as that is the web standard, but when pressed, it does not lead to any page. Users may wait for a couple seconds expecting a page to load, but in actuality, it is just information placed in a button design.

Because my participant’s screen was in split screen, the website width was smaller. Because of this, the navigation bar completely disappeared, and he had to complete the tasks through longer steps. The FAQs page and phone number were not visible as opposed to when you are viewing the website in a larger width. I learned of a new usability issue regarding the inaccessible responsive web design.

Third Task

I focused on finding the About page, that was nested under a second-level in the hamburger menu.

Problem: Important links under nested menu (Rating: 3) New users will have to search harder to learn more about Lessonface, as information on the company and how it works is “the least important” in this information hierarchy.

My participant had no issue finding this page, as he already found it when looking for the FAQs page. It did not seem to be an issue for his experience, but would likely be in issue if I were to test more people.

Reflection

Overall, I think my UT session went well, especially in the following aspects. My participant was very vocal when being tested, so it was easy to follow his user flow when performing these tasks. I learned that through usability testing, I can easily find new problems, as every user is different and has different experiences. I also think it was successful despite my participant not knowing what to do for a couple minutes for my second task. I was patient with him and reassured him that there are no wrong answers.

Not counting the demographics, the test lasted 22 minutes, which was faster than I anticipated. It also could be because I did not speak slowly enough, as I did have the requirement of three tasks, with 3-4 listed steps for them. Since I believe the testing session went by quickly, I think it would have been better if I added more tasks that tackled the significant usability problems I noticed. I think I could work on my attitude as well. I think I did not have enough of a welcoming and positive tone in my voice. I did not give as much positive feedback as I wanted and am not sure if my participant felt comfortable on their end. In the future, I will try to build trust at the beginning of the session and have a conversation with the participant before beginning with the test. Considering that this is one of my first UT sessions I critiqued myself in, I think this was good foundation for improving myself and my UT materials in the future.