Effortless Ordering: An In-Depth Evaluation of Airpals Website’s ‘Order Flow’ & User Expectations
Overview
Our team conducted moderated remote user tests on Airpals website’s order flow to gather feedback from participants and provided recommendations to improve the website's usability
Airpals is an online platform that offers B2B courier services to clients in New York City. Our team of experienced researchers from the Center for Digital Experiences at Pratt Institute conducted an evaluation of Airpals' order flow to identify any pain points or areas of improvements for the users. Our objective was to ensure that Airpals provides its customers with an efficient user experience when using the website to place orders. The aim of this study was to assess the usability of the Airpals website, pinpoint areas that require improvement based on user experience, and collect feedback on users' expectations after order placement.
Our testing process involved recruiting a diverse group of participants within the target demographic of creative and business operational professionals and observing their actions and feedback as they completed tasks related to the Airpals website's order flow. By employing a "think-aloud" methodology, our team was able to gain insights into users' experiences and thought processes. After conducting 11 moderated remote user tests and analyzing the collected data, we identified key findings and areas for improvement to enhance the overall user experience on the Airpals website.
A quick deep dive into the study results
Overall, the participants found the Airpals website to be user-friendly and efficient, successfully scheduling a delivery. However, areas of improvement were also identified, including the grouping of parcel details and delivery instructions, more comprehensive error messages, and increased flexibility and control for users in the time, address, and editing sections of the website. As a result, our team provided 6 recommendations for larger issues and 4 adjustments for smaller, but still significant issues.
Streamline the flow for inputting parcel details
1
Reinforce user control when editing order info
METHOD Moderated Remote User Test
PRODUCT TYPE Website (Only Desktop)
CLIENT Airpals
TIMELINE March-May 2023
TEAM Priyanka Jain (me) Sara Sarmiento Mary Haws Anne Kuo Lalita Chavan
MY ROLE UX Designer & Researcher
TOOLS USED Zoom, Figma, Google Spreadsheet & Docs
4
Clarify pick-up & drop-off instructions
2
Increase error salience with more signifiers
5
Improve flexibility and consistency of time selection
3
Provide feedback & flexibility for address selection
6
Project Scope
Target Audience
Setting the Stage: The Airpals Kick-off Meeting
At the project outset, a kick-off meeting with our client was pivotal. We discussed objectives, tasks, target audience, and platform preferences to ensure alignment and set the foundation for a successful usability study.
Meeting the Airpals Team over zoom
Creative or Business operational professionals
Evaluate the usability of the Airpals website's Order flow & gather feedback on user’s expectations after placing an order
Testing the main function of the website: Order flow
10 Moderated Remote Usability Tests
Project Goal
We employed moderated remote usability testing, considered the gold standard, to collect in-depth feedback from our target audience. We recruited participants, guided them through tasks, and conducted debriefing sessions to uncover pain points and areas of confusion, culminating in valuable recommendations
The Process Behind Enhancing Airpals website’s User Experience
Creating Tasks & Questions for the Airpals Usability Test
After defining project goals and testing scope, we crafted a detailed test plan, including a task simulating the common user scenario on Airpals, aimed at evaluating key features and gathering user feedback during the 'Order' flow.
Your task will be to locate and complete the form to send a package to me with delivery on the next business day. Remember to think about the last package you sent or one that you’ve sent before when filling out the form details.
LOCATE AND COMPLETE THE FORM
We designed post-task questions and a Google Form post-test questionnaire to gather specific insights and detailed feedback from participants after usability testing. Additionally, we held debriefing sessions to capture verbal feedback for a comprehensive evaluation.
Who’s behind the data: Participant Demographics
After task setup, we screened participants with a diverse questionnaire to ensure representation across age, gender, industry sizes, and roles, resulting in a well-rounded group that provided varied insights and recommendations for our diverse target audience.
11 Participants recruited & 8 Retained
25% aged 36-45 & 75% aged 18-35
50% Female & 50% Other
50% from 500+ company size & 50% Other
100% based in New York
100% have used courier services before
Understanding User Experience: Airpals Moderated Usability Testing
Each one of us conducted 5 pilot tests to check our testing plan and make changes if needed.
We then conducted 11 moderated user testing sessions via Zoom, each lasting around 30 minutes.
Participants were given an informed consent form and asked a few pre-test questions to gather basic demographic information. Each of us moderated 2 and took notes for 2 tests.
The moderator guided participants through scenarios and tasks while the notetaker recorded feedback via a Google Form.
Post-task and post-test questionnaires were used to gather thoughts on the experience and frustrations.
Participants received a $10 Amazon gift card for their participation.
This phase was essential in identifying usability issues and areas for improvement on the website.
Unlocking User Insights: Analyzing User Test Results
Moderating user tests on zoom
Throughout usability tests, we recorded participant info in a customized Google form, analyzed data using thematic analysis in a digital grouping spreadsheet to categorize feedback based on common issues, identified prevalent issues, and discussed patterns within categories to generate solutions and recommendations
Google form for note-taking
Digital grouping spreadsheet
Delving into the Overall findings of the User Testing
agreed Order form was easy to complete
agreed that they enjoyed using the Airpals interface.
Completed the task of submitting an order
POST-TEST FINDINGS
88%
75%
10/10
I liked, from a design perspective, that it was very clean. It seemed, from the very beginning, pretty clear where I was supposed to go.”
“
“
VISUAL DESIGN IMPACT
I think it was pretty clean.”
Straightforward. I felt it was pretty easy. It definitely does what it needs to do.”
“
USER-FRIENDLINESS OF THE WEBSITE
Our analysis of the data revealed both positive findings and areas for improvement. By categorizing the feedback using the digital grouping spreadsheet, we were able to identify common issues and themes and provide actionable insights or recommendations for enhancing the user experience of the Airpals website. Let’s look at them in detail.
Feedback-driven Recommendations: Key Findings from User Testing
We came up with six recommendations based on the most frequent and severe feedback and observations of issues during the task.
#1 Streamline the flow for inputting parcel details
THE PROBLEM
Challenges in the 'How' page parcel details included the need for clearer package size options, confusion regarding the value field input (with some participants attempting to enter a dollar sign), issues with the single field for package details and handling instructions, and a tendency to overlook item details due to the page's length, leading to errors and omissions
THE SOLUTION
Remove sender, recipient, and pick-up/drop-off notes to simplify to “What” page
Add maximum dimensions to shipment size categories
Separate item description and special handling instructions to two fields
Add a “$” before the value field and clarify the language
“How” page renamed to “What” page
#2 Clarify pick-up & drop-off instructions
THE PROBLEM
Some participants expected sender and recipient information alongside addresses for better clarity. Half of them skipped pick-up and drop-off notes, initially struggled to understand their purpose due to lengthy explanations, and some faced confusion with pop-up messages when skipping notes, even if they'd filled one of them.
THE SOLUTION
- Pop-up should specify which field(s) (pick-up and/or drop-off notes) was left blank
- Collect sender and recipient information alongside their addresses
- Move the pick-up and drop-off notes to the “Where” page
- Simplify the language for pick-up and drop-off instructions
Added ‘notes’ on Where page
I already filled out the pick-up notes, so I’m not sure why this pop-up appeared.”
“
PARTICIPANT QUOTES
#3 Improve flexibility and consistency of time selection
THE PROBLEM
The first feedback from our participants in this regards was that there are only 3 options for the available “deliver by” time, and they couldn’t select some time in between. Aside from that, four of our participants adjusted the time since the default timing is past business hours. Participants also found the pickup drop-downs a bit confusing, not expecting three separate drop-downs in each field.
THE SOLUTION
Use dropdown instead of buttons for delivery deadline to offer more time options
Set the default delivery deadline within working hours
Align the dropdown style to text input fields, and provide specific clicking areas
Added drop downs that are recognizable
Thinking back, it didn't seem to have provided me with too many options for time selection.
“
PARTICIPANT QUOTES
#4 Reinforce user control when editing order info
THE PROBLEM
2 out of the 8 participants tried to edit the order details on the review page and were navigated to the ‘When’ step of the form. Participants who clicked on “Edit” for a particular field on the “Review” page expected to go back to the page on which they could make the edit. Instead, they were redirected to the first page of the form. This form behavior caused confusion for the participants, the next steps that needed to be taken by them were not very clear.
THE SOLUTION
Navigate to the specific step in the order form that needs to be edited instead of the form start page
A mock up showing how it should navigate
When I went back to edit the delivery time, I clicked edit and it brought me back to the very first screen of ‘when’ instead of bringing me back to the ‘time’.”
“
PARTICIPANT QUOTES
#5 Increase error salience with more signifiers
THE PROBLEM
3 out of 8 participants were unable to notice the error messages displayed on the screen. When incorrect data was entered, the error message appeared right below the field that needed to be corrected but it went unnoticed. The participants only noticed the error message when they clicked on continue but nothing happened, their screen froze.
THE SOLUTION
Increase the font size of error messages and highlight the fields to make them more evident.
Highlighted error signifiers
When I couldn’t submit the form because of the apostrophe in my name, if that error was much more visible on my screen that would be helpful because I thought my computer was frozen.”
“
PARTICIPANT QUOTES
#6 Provide feedback & flexibility for address selection
THE PROBLEM
Participants encountered issues with address input, as copy-pasting led to disappearing addresses. Some typed addresses without selecting from the dropdown, resulting in auto-selection of different addresses. Correcting addresses was cumbersome, as participants had to click 'x' instead of being able to directly highlight the text, which was unintuitive
THE SOLUTION
- Clarify that users should select an address from the dropdown
- Allow users to edit the address text directly
Address showing the error and also having drop-down
Perfecting the User Experience: Addressing Minor Issues found in Testing
During testing, our participants uncovered several small yet significant issues that we think should be addressed. Although these issues may seem minor, we believe that addressing them could lead to an overall improvement in the user experience.
#1 Accept special characters in names
Participants faced problems with special characters like apostrophes and hyphens not being accepted in name fields, causing delays in moving to next screen.
As these characters are common in last names, they should be allowed in name fields.
#2 Field for phone extension numbers
One of our participants noted that there was no field for phone number extensions. As this is a B2B business, users are likely to use a work number, which commonly includes an extension.
Add a field for an optional phone number extension.
#3 Date format
Participants pointed out that the review page uses the international date format (day/month/year), while the US convention is month/day/year.
Given it's a New York-based service, aligning with US conventions is advisable.
Participants noted that the text lacked sufficient contrast against the background, with a contrast ratio of 3.67:1, falling short of the recommended 4.5:1 standard for users with visual impairments.
To enhance accessibility, we recommend improving the contrast to meet the recommended standard.
#4 Color contrast
Conclusion: From Study to Success
“There are many useful findings here that I didn’t know. Some of them we noticed but few of them we haven’t noticed before. Thank you so much for that.”
Quote from Eric (UI/UX Designer at Airpals)
In conclusion, our team submitted a comprehensive report and presentation to our client, which outlined the Project goals, scope, Methodology, test details, key usability findings and recommendations for improvement based on the usability testing of the website. Our client expressed their appreciation for the valuable insights provided and recognized the significance of the newly acquired information in enhancing their website's user experience.
Presentation with our clients wherein I presented along with my team
Happy faces after client appreciation
Client Feedback
“I took notes of things we should dig more. I haven’t really noticed about the date format ! I don’t know why.”
Eric (UI/UX Designer)
“You guys have come up with things that we have never, never imagined that would be a problem. So, I’m very grateful for all the work and time and dedication that you’ve put into this”
Carlos
“These little things that skipped from us like the ‘$’ sign. That was super obvious, was in the face but it was so insightful. Those are the things that will helps us optimize our compression rate in the long run. So, good catch on that.”
In conclusion, the client expressed their commitment to implementing most of the recommended changes and requested a follow-up check-in in soon to discuss the progress of the recommendations applied.
We look forward to seeing the positive impact of these improvements on the user experience and the success of the client's website. It was a pleasure working with the client and contributing to the enhancement of their website.
Special thanks to my talented team for making this Usability Testing a huge success and delivering such great insights!
Joshe (Client)