Quora : Usability case study
Does mentioning upvotes in Quora search make it more engaging?
At its most basic, the role of the search engine results is to present items matching a given query. However, behind this simple brief, resides a layer of depth and complexity. There are a lot more thinking and experiments involved in making the search process worthy and engaging for the users.
Searching on Quora is very different from searching for a friend on Whatsapp or Facebook. When I search for a friend on Whatsapp, it clearly shows if he is present or not — It’s binary. But same is not the case with Quora or Google. These applications make sure that you get various results closest to your query. Ideally, it’s a win-win for users, as well as the business — Users read everything related to the topic and thus the time they spend on the app is more. In a nutshell, not being binary ensures higher engagement on the app.
Does that always happen? Not really… Few irrelevant and unsatisfying answers, turns away the reader. But, Quora has a great feature to the rescue — Upvotes for answers that other readers have found useful. It can be used wonderfully to keep users engaged.
My hypothesis
What if the maximum upvote numbers were present on the search results? Wouldn’t it be super useful for users?
Designers might argue, that every answer within a question on Quora will have varying number of upvotes. But I selected the highest number of upvotes to be upfront.
Based on this hypothesis I made a few design changes on the Quora search screen and planned out a usability test around it.
About Quora
Quora’s mission is to share and grow the world’s knowledge. A vast amount of the knowledge that would be valuable to many people is currently only available to a few — either locked in people’s heads, or only accessible to select groups. Quora connects the people who have knowledge to the people who need it, to bring together people with different perspectives so they can understand each other better, and to empower everyone to share their knowledge for the benefit of the rest of the world.
Objective of Usability Test
Understand if a sticky upvote tag by the side of search result improves the engagement on the app.
User research
User persona
Prior to conducting usability tests, I developed a user persona to better understand the target users of Quora’s Android app. This process helped me get into the mindset of the users, thinking in terms of their contexts, needs, and goals
So meet Rahul!
Platform used
I have used the CanvasFlip online tool for creating the prototypes and for UX insights such as session replay (the user videos). conversion funnel and heat maps.
Number of users in the test
I had a user base of 32 users. So, each prototype was tested by 16 users each.
Task given to users
I checked out the Google trends to figure out what people were busy searching late recently. I framed my task accordingly.
I wrote as the task — “Explore about the US presidential elections”
Usability test analysis
All the 32 users took the test and I had all the usability videos, heat maps and all other insights in my account. Before we start with analyzing those results, you would love to have a look at both the prototypes.
Prototypes in action
- Existing design prototype — Open new tab for prototype of existing design
- Design suggestion prototype — Open new tab for prototype of design suggestion
Comparison of UX parameters of both the prototypes
- Watch out the pattern a user follows to check out answers
It was very interesting to find out a specific search pattern in both the cases. For the first prototype (without upvotes listed), users decided on one favorite topic they wanted to read about and coming back o the search page results, they either tried one more or nothing at all.
Check out this session replay of a user-
Whereas the search pattern for the second prototype was quite different. The favorite one obviously goes first. But, users kept coming back to the search page to read the other questions with a good upvote number. The time they stayed on the app was much more than the previous prototype.
Check out this session replay of a user-
- Interaction on the search options
As observed from the heat map, there is almost equal interaction on each of the questions. Which means that almost equal number of user segment interacted with each question. There are not intersecting interactions on any questions as the heat map is nearly blue for every question
On the contradictory, we see a multiple interactions on the design with upvotes upfront. The interactions are towards the shade of red, which means there are multiple interaction for the same questions. It is clear from the heat map that single users have read multiple answers unlike the existing design.
- Session length on each prototype
Session length is the time user spends on the app in one session, performing different actions and then finally exits or session times out. Session length is proportional to the engagement on the app
It is clear from the graph that users have stayed for a longer duration on the second prototype, which has upvotes mentioned upfront. They were probably reading other articles with high upvotes.
Devices used by readers
Since the experience over desktop would be completed different from that on a mobile device, I made sure that the users tested the prototype on a mobile device.
Final Words
The Quora app is crowded with queries and answers. The search results page plays a crucial role in the search experience, conveying to users a response to their information needs, and engaging them in a dialogue that guides them along their information journey. When the user is not sure which question they want to read about the topic, drawing a metric for them to choose proves to be useful and quite engaging for readers.
The idea is to keep them motivated to read the other questions related to the topic as well. This is how the user will spend more and more time on the app.
Agreed? Do let me know what you think about this.
