Background
This project was completed for "Usability II," a course in Kent State University's Master of Science in User Experience Design curriculum. The assignment was to research the Outings app (a Microsoft Garage project) and run usability tests to find pain points. Below is my final report for the Outings app stakeholders. 
Executive Summary 
This test found several usability issues. The most critical being that the main function of this app (helping people explore new locations by providing stories via external links) was not used by any of the participants. Although participants liked discovering new places and saving them to their favorites, they did not want to leave the app to learn more. Another major usability issue was found in the search bar. The search bar was heavily used during testing, but it was unable to find most of the search phrases participants entered. This led to many of the participants becoming frustrated and ready to abandon the app. A list of prioritized recommendations for improving the app’s usability is offered at the end of this report.

About the Study 
The purpose of this study was to find pain points in Microsoft’s Outings app. Each test was moderated, and took an average of 24 minutes to complete. Nine participants completed the usability test. The test included four tasks that highlighted some of the application’s capabilities. The participants consisted of five females and four males, and their age ranged from 25-74 years old.

Tasks
1. It’s a three-day weekend and you want to take a mini vacation with your family. Use the app to find somewhere within 50 miles of you that you would like to go. 
2. Find someplace in your city that you’ve visited before and leave a review of it. 
3. You are saving up for a vacation to New York, New York. Using the app, find three places that you would like to go to and mark them in a way that would allow you to find them again. 
4. You’ve been using the app for a while and want to get better results that align with your interests. Find a way to personalize your account.

Account Setup 
After the participants downloaded the app, they were shown two screens. The first was a login screen that asked them to sign in with either Facebook, Google, or Microsoft. Several participants skipped this step which caused issues later on. 
The second screen asked them to personalize the app by choosing five topics they were interested in. This step was easy, but a few participants didn’t realize that they could scroll to see more topics. 
Task 1 Findings 
Mini Vacation with your Family
When opening the app, several users didn’t understand why the first page presented to them was the Discovery page. The majority of participants used the search bar at least once, but had difficulty finding terms that led to what they wanted or, in some cases, any results at all. One participant searched for camping, but only found hiking trails. He then tried every word combination he could think of to get a result that matched what he wanted, but eventually got frustrated and quit. Participants clicked on the globe, arrow, and magnifying glass icons (highlighted in yellow in images below) when trying to find locations within 50 miles. The only icon which offered this functionality was the arrow. Four out of the nine participants wanted more information, including location descriptions, directions, reviews, hotels, ticket pricing, age restrictions, and the ability to buy tickets. 
Key Takeaways
Users expect search bars to perform as well as popular search engines, such as Google. Search functionality in the app was extremely limited. Navigation icons need labels so users can confidently understand what they are clicking.

Task 1 Quotes
“I would quit using this app right now if I wasn’t testing it.”

“No offense to the app, but it’s not being very helpful.”
“So this actually seems to me more like, you know, a social media thing as opposed to a real functional thing to find places to go.”
Task 2 Findings
Leave a Review
Participants were still unsure of how to navigate the app and clicked on almost every icon trying to find a location so they could leave a review. Once the participants found the location they were looking for, they had little trouble finding the Been There icon to leave a review. Two participants used the map feature to find a location. One of them wished the map pins included numbers that corresponded to a list view. Participants were required to sign in to the app in order to leave a review. The participants who skipped the sign in step when they first opened the app were now prompted to sign in. While there were a few participants who were okay with using their Facebook or Google account to sign in, far more participants either didn’t have their passwords, so couldn’t complete the task, or questioned the purpose of making them sign in at all. One participant refused to sign in because he didn’t know how his information was going to be used by the app. 
Key Takeaways
Most participants prefer not to sign in, and  those who do don’t want to use a third party site to manage their credentials.

Task 2 Quotes
Referring to the linked articles
“So once again this is just a clearing house. This is not an independent self-contained thing”
“Like I said, I’m not very literate but it seems pretty easy to use.”
Referring to logging in
“I just don't care to have my stuff (information) all over the internet. To be honest, I just wouldn’t leave a review, or I’d go make a new Google account.”
Task 3 Findings
Vacation to New York
Participants searched for things to do in New York City by using the magnifying glass icon or the Destinations card. The beautiful photos and “not so touristy” places were appealing to participants. Participants found three things they wanted to share, but again ran into the sign in issue. A few participants were unable to complete the task, or begrudgingly chose to sign in. Once signed in, saving a location to their favorites was easy to accomplish. One participant was especially delighted when he tapped “save” and the location went flying into the user profile icon. He said, “That’s pretty easy. That’s pretty slick. I like that. That feature right there is sweet.” Unfortunately, this animation was only available on the Android app.
Key Takeaways
Participants enjoy high quality images when searching for activities.Participants were unable to complete this task if they didn’t sign in to the app.
Task 4 Findings
Personalize your Account
All the participants tapped the user profile icon and easily found the account page. Once there, they had difficulty proceeding. Participants clicked on the settings icon, the profile photo, the location, or the feedback icon, but none of these were the correct method for personalizing their accounts. The correct way to personalize an account is via the Topics bar. Although some participants clicked on the Topics bar, they were unsure if that was the right way to personalize their accounts due to the lack of minimal personalization options offered. This uncertainty may have been a function of the task or the wording. Once on the Topics page, one participant wondered if changing their topics had any effect on what would appear in their feed. Another participant wanted the ability to drill down within each topic. One other participant commented that there didn’t seem to be any order to the listed topics. Over the nine usability tests, no matter where the participant lived, or the topics they chose, the displayed cards were all incredibly similar.
Key Takeaways
Participants want to personalize their topics according to interest, location and physical capabilities.

Task 4 Quotes
Referring to the Discover page
“It all seems so thrown at you.”
Referring to the entire app experience
“I don’t think they provide enough instructions to help you understand what you are doing here.”
Referring to the Topics page
“I’m not impressed with this kind of hodgepodge of information it’s throwing at me.”
App Issues
Malfunctions and Usability Flaws
The app exhibited the following glitchy behaviors during the usability tests: 
1- The search bar was unable to find common search phrases entered by participants. An example is the search phrase “NYC” offered no results. 
2- Five participants tried to go back to the last page by clicking their phone’s back button; this did not go back, but instead kicked them out of the app. This happened on ten separate occasions. 
3- Every participant clicked on a location card’s text description in an attempt to find more information about a card, but none of the copy was clickable. The app wanted participants to click on the provided link to find more information, but very few of the participants clicked the link to read more. 
4- The keyboard would not consistently pop up when participants where using the search function. This appeared to be an Android OS issue.
5- After searching for the Grand Canyon, four out of the six results were for Havasu Falls. 
6- Some of the locations are out of date. While searching for a mini vacation (Task 1), a participant noted that one of the app’s suggested locations, which the participant was familiar with, was actually no longer accessible due to landslides. 
7- On several occasions, the app couldn’t find a participant’s location with their GPS or via the map. 
8- One participant noted that some of the Family Friendly activities did not seem appropriate for children. Example: Seattle’s Underground Tour.

Recommendations by Priority
1- While running the usability tests, we found that one of the main functions of this app, providing high quality travel stories (links to external sites), was hardly used, and participants were frustrated when they clicked on a link and were pushed out of the app. Most participants wanted to get information about a travel location without leaving the app. There seems to be a major discrepancy between what Outings is trying to provide, and what participants want from it. This should be examined to see if the app’s main purpose needs to be changed. 
2- Create an onboarding process that explains how to use the app, while also showing its value. Make the onboarding skippable but easy to find in case the user wants to review it later. This process should also create a section that includes the user’s local city and nearby attractions. 
3- Include a home page where a user can land after opening the app. The home page should offer users all available app features, allowing them to navigate the app easier, and would include a feature preventing the user from accidentally backing out of the app. 
4- Add an in-app sign in option rather than requiring the user to sign in via a third party (Facebook, Microsoft, Google). 
5- Redesign the search functionality so that it includes activities and locations users want to use, and not just words that are pulled from the linked articles. 
6- Label the navigation icons. This will help users understand how to navigate within the app and be certain of what they are clicking on. 
7- Include additional information such as directions, prices, and ticket cost within the app. 
8- Change the Topics section to allow the user to drill down and personalize their results to include what they want/don’t want to see. 
9- Make any copy with an ellipsis expand within the app.
Back to Top