At NID, Bangalore we have a 2.5 years course for designing. The course has taught me how to go out executing the entire UX process — from research to design, testing, iteration, and then final deliverables. Last summer, I spent my entire vacation on an iPhone app project, and also sleeping a lot. ;) This article talks about how I started with the so-called best practice, in-person testing and then finally gave up on it. What really came to my rescue was remote usability testing.
My design story
Week 1
To start off with my story, I had a month’s time to design, validate and hand-off my designs. But as you can imagine, the first week just flew away. I didn’t do much in the week, except that I slept a lot :P ! The idea was scribbled at the backside of my diary but no progress since then.
Week 2
It was the Sunday night of week 1 when I entered my friend’s room with a bottle of Coke, just to see him work on his designs. “Knock knock! It’s me the reality check!!” I just rushed back to my room, grabbed a pen and started planning the project execution in the three weeks in hand. According to the theoretical plan, things were still under control. *Huh! Sigh of relief*
Next morning, I start with refining my paper sketches. It took around a couple of days to polish my paper sketches and get to a point where I could take feedback from my peers.
By the end of week 2 my first version of PSDs were ready. Next in the process was validating the designs and performing usability testing on the designs. Like any other designer, this is the part that excites me the most. It helps me understand and design for users’ actual experiences.
Week 3
It was time to recruit users for the in-person usability testing. My previous experiences with in-person usability testing have not been quite amazing, but I still wanted to go with the same because it’s really fun to connect with real users, and read their body language and other subtle reactions. The one thing that kept daunting me while recruiting users was — last time when I was performing in-person testing as a class project, we had a month’s time; whereas now I barely that a week’s time. Will I able to do it?
So I decided not to go very far in spotting users. I started looking for friends of friends. My target audience was users in the age group of 18–35, and female gender. That quite fit into my friends of friends category. I approached around 15 of them and tried to find the best day when most of them will be around my college campus. By the end of week 3 I was supposed to be done with the usability testing so that I can iterate on my designs next week and comfortably hand-off my designs.
A day before the testing
I received 4 text messages informing that they wouldn’t be able to make it so far. This is one of those times when you don’t really appreciate your college being in the outskirts. :( I was still hoping that the other 11 would turn up the next day.
The usability testing day
One would really expect that out of the 11 you confirmed, atleast 8–9 will turn up. Can you wildly take a guess as to how many would have come up?? There were just 2 of them. I was totally in a state of panic. I didn’t know what I had to do next. I had these few options in hand —
- Conclude design iterations based on the 2 ladies I interviewed in-person.
- Get my roommates to do this and incentivise them with pizzas maybe..
- Travel 43kms to reach the city, visit a cafe, convince people to spend 20 mins with me..
None of these options though were very convincing for me.
Switching to remote usability testing
Just the other day something happened. We had 3 visitors to our college. These guys were from CanvasFlip. They were in the college campus to talk about “Remote usability testing”. Honestly, this was the need of the hour for me. I rushed to the talk, and I exactly knew what I had to do next.
That night I sent a text to all those on my target audience list, asking them if they could just try the prototype on their mobile phones. What response I got is quite predictable! So just the next morning I grabbed the prototype link on CanvasFlip and sent to to 25 users. By the end of the day, I had the user behaviour data of 22 of them!!!
Seriously??? Was it so simple to validate designs? I was literally jumping with joy. Here’s what all I had in the CanvasFlip collaborative dashboard -
- Session recording of all 22 users. Hits and misses on the prototype
- The time each user took on each design screen
- The drop-off rate and conversion rate of each flow
- The engagement on each UI element
- The quantitative analysis of interaction on the screens in the form of heat maps
- Average time taken by users to complete a flow.
Honestly, it gives me user behaviour data in a much simpler way and helps me take design iteration decisions fasters.
Week 4
The initial couple of days of the week were spent in drawing conclusions from the data in hand and iterating on the designs. I was much more relaxed in the last week. I took my time to iterate on the designs. And an added benefit was that I could repeat the usability test once more before I submitted my project.
My experience with CanvasFlip
It gives me immense confidence that a project can successfully be executed in 3 week’s time. I realised few things with this project.
- In-person usability testing is fun but remote testing is a lot more quick, cost effective and crisp. The benefit being that you can repeat the tests a multiple times.
- Had I used CanvasFlip for my previous class projects, I would have saved s-o-o-o-o much of time and money for a meaty wholesome meal!
- Usability tests are better when its done often.
For fast-paced projects (in college and at work), unmoderated remote testing is a fantastic way to validate usability with minimal time investment. I’m still a proponent of in-person testing for complex prototypes, and for picking up those subtle cues we can catch only in person. But remote participants still provide very useful insights.
