Improving the Usability of a Higher Education Website
CLIENT Pratt Institute Communications
MY ROLE Researcher, Evaluator, Prototyper
TIMELINE October - December 2019
The communications office of the Pratt Institute tasked our team of four usability experts with evaluating the usability of the school’s website, Pratt.edu. Working with me were Archana Ravi, Shivani Ishwar, and Hyerim Hwang.
Our stakeholders ask specifically that we focus on current and prospective students, and to evaluate their desktop and mobile interfaces. This design story is the result of 7 weeks of usability experiments and the analysis and report of our findings.
Preparing the Tests
GOAL: To create a user experiment that was rigorous, ethical, and efficient
What We Did
Deliverables: Test script with tasks, consent form
Elements to Evaluate: Homepage, Admissions, News, Events
Platforms to Evaluate: Desktop, mobile
User Groups: Current students, prospective students
What We Learned
One user test should last under 45 minutes.
To test every element that the stakeholder asked for in this time, each task needed to be simple.
An evaluation of the test was essential before we began recruiting participants to iron out flaws and ensure that our research questions were being answered.
Recruiting Participants
GOAL: To gather 8 test participants who fit into our stakeholder’s desired user groups
What We Did
Determined target participants, methods, and resources
Target Participants: Current Pratt undergraduate students who were considering graduate school (at Pratt or otherwise)
Methods: Online questionnaire to determine eligibility, email correspondence for scheduling
Resources: Mailing list from Pratt Communications of Juniors and Seniors, offer of Amazon Gift Cards as compensation
What We Learned
Putting time estimates on all tasks was crucial in getting potential test subjects to fill out our questionnaire.
It was important to avoid saying which interface we would be using for tests, to prevent users from “coming prepared”.
Word of mouth was often more effective in recruiting participants than a mass email; many of the recruits we scheduled were coworkers or friends with whom we could discuss our work.
Conducting User Research
GOAL: Gather written and audiovisual data from user tests
What We Did
Data collection methods: pre- and post- questionnaires, think-aloud, verbal difficulty ratings, screen and audio recordings with 6 undergraduate students who were considering graduate education
Tools: iMac with QuickTime, user’s personal mobile device, PEVO V4K Ultra High Definition USB Document Camera
Team members: study moderator, note taker
What We Learned
Putting time estimates on all tasks was crucial in getting potential test subjects to fill out our questionnaire.
It was important to avoid saying which interface we would be using for tests, to prevent users from “coming prepared”.
Word of mouth was often more effective in recruiting participants than a mass email; many of the recruits we scheduled were coworkers or friends with whom we could discuss our work.
Analyzing the Results
GOAL: Analyze the data from our user research to create actionable recommendations
What We Did
Tabulated results; tasks were given a score of 2 (completed the task), 1 (completed the task with some difficulty), and 0 (did not complete the task).
Listened to and watched each of the tests to extract quotes and observations from the recordings
What We Learned
7 out of 8 participants ignored the side navigation panel. The one participant who did use it finished their tasks in the fastest time with the least difficulty.
5 out of 8 participants were unable to orient themselves in the site’s architecture, and couldn’t tell if they were looking at graduate or undergraduate pages.
Information presented in paragraph form was difficult to find. All 8 participants skimmed through a page for headings before delving deeper.
All 8 participants expressed frustration towards the site’s mobile presentation, saying that many visual elements were only functional on a larger screen.
Tasks
Find information about how to visit Pratt, and meet with a Pratt representative.
Find out what graduate programs are available at Pratt.
Find the application deadline for the Fall 2020 semester for a graduate program you might be interested in at Pratt.
Find an estimate for the tuition and other costs associated with a graduate program you might be interested in at Pratt.
Find any recent news items about Pratt’s national ranking.
Find an event happening at Pratt on December 4th.
Find an event that you might be interested in attending at Pratt.
Performance ratings for each task by participant. Each task was given a 2 (completed the task), 1 (completed the task with difficulty) or 0 (did not complete the task)
Suggesting Changes to the Interface
GOAL: Present our recommendations to improve the Pratt website
What We Did
Presented our findings and recommendations to our client:
Make better use of navigational features on the website and emphasize the side panel.
Implement a complete separation between Undergraduate and Graduate admissions information
Make improvements to the page layouts, specifically by favoring an image-based layout and consistent heading and list styles
Improve the responsiveness of the mobile interface by fixing the stacking issues caused by a smaller width (like in the calendar prototype below)
NOTE: The images below represent our recommended changes. The current images can be found in our final presentation.
What We Learned
Recommendations that involve removing an element of the site needed to have a suggestion of where that element would be represented elsewhere.
While some recommendation involved previously-templated visual elements, others would take much more design work in the site’s backend. However, most of that new work could be templated for future use.
The process for making visual changes often involved more than one department under the office of communications; all changes were collaborative.
This study was the most recent of many students evaluating the Pratt website.
Remove carousel of news items with a photo of the Pratt campus, as we observed most people ignored the carousel
Add dropdown components to the schools represented so that users can decide if they want to drill down before leaving the page
Implement a complete separation of Undergraduate and Graduate admissions.
Put less information in paragraph form and use images to help break up information
Emphasize the side navigation as a tool for moving through the site
Improve the layout of the calendar so events can be found faster.
Reflection
Before this study, Pratt’s Digital Communication department had collaborated with the School of Information in a similar way in previous semesters. After conducting this user test, I was hired as a Graduate Assistant with in Digital Communications and given the opportunity to synthesize this research. The results of this case study will be a part of a larger redesign of Pratt.edu.