Project Proposal
Background
QuizMaker is a project that was made for the technology company, IBM, in order to showcase the ease of use of IBM's lightweight open framework, Open Liberty. IBM was seeking a way to increase adoption and revenue for Open Liberty, so our team was tasked with a way to demonstrate how to make it work in a product.
Ten people worked on this project within Agile and Scrum methodology. There were four different teams, consisting of Usability, Requirements, Engine, and GUI, although we all worked on the project collectively. We also had three stakeholders and product managers who provided us with requirements, guidelines, and constraints for the project. The purpose of the product, QuizMaker, is for students to enable one another to study by creating multiple choice quizzes, and for teachers to easily take those quizzes to make their own.
Aim
QuizMaker is a web application that will allow students and teachers to create, take, share, and review quizzes made from other students that are stored in a central database.
My Role
My role in QuizMaker was to be the UX Designer for the Usability team, as well as the principle investigator for the usability test that we conducted. I collaborated within all teams to ensure that we are on the same page with deliverables and deadlines. I created deliverables such as, user personas on Figma, user flows on miro.com, sketches, and wireframes and prototypes on Adobe XD.
User Personas and Pain Points
We created personas with the specific target users in mind of students and professors. Being students ourselves, we were able to informally interview our peers by asking questions such as:
1) How comfortable are you with technology?
2) What are some of your wants and goals when it comes to studying/teaching?
3) What frustrates you when it comes to collaborating with other students/teachers when studying/teaching?
4) What are your study/instruction methods?
These questions helped us to formulate six user personas of five teachers and two instructors.
Brainstorming and Ideation
User Flows
Using miro.com, I was responsible for creating user flows to represent mind maps of the user journey for our targeted end-users of students and instructors. The creation of this journey allowed me to understand our overall design and how users may take steps to achieve each goal within the product.
Prototyping in Adobe XD
Hi-Fidelity Wireframes
Usability Testing (with Zoom and Qualtrics)
Purpose
The purpose was to discover how users (students and professors) will navigate the web application by following specific tasks that users will be asked to complete.
6 students and 6 professors at SUNY Oswego were recruited to participate. When they consented, we set up a 3o-minute time slot for each participant to complete the study via Zoom.
Before the test began, participants were briefed and presented with an informed consent form.
Tasks
They were then asked to complete four tasks within our web application as they spoke their thought process aloud.
Tasks for Students and Professors
After Scenario Questionnaire (ASQ)
After each task, the participants were asked to complete an ASQ on the survey software, Qualtrics. The questionnaire consisted of four questions based on a 5-point Likert scale, with 1 being “Strongly Disagree” to 5 being “Strongly Agree”.
After Scenario Questionnaire (ASQ)
System Usability Questionnaire (SUS)
When all tasks were completed, the participants were asked to answer 10 questions on a SUS.
System Usability Scale (SUS)
Debrief
When the study was completed, participants were debriefed and given the opportunity to ask any questions about the study.
Research Goals
1. To understand how SUNY Oswego students and professors interact with QuizMaker.
2. To discover if the implementation of WCAG 2.1 industry standards in our final product are confirmed by user satisfaction.
3. To identify pain points that impact user satisfaction and task completion time.
Results & Analysis
The results from the usability test was calculated using two methods, the System Usability Scale and a t-test.
System Usability Scale
We chose to the SUS because it is widely known reliable tool for testing usability. The System Usability Scale formula was used by calculating the 'participants' and 'instructors' SUS scores and measuring Display Satisfaction for the 'Light Mode' and Dark Mode' of the interface.
Results
Both modes scored high on the System Usability Scale score, as seen in the table below. The scores are rated above the industry standard score, which is 68. Only one score was rated below it this standard.
Results indicate that the light mode (M = 85.500, SD = 10.368) is perceived to be more user friendly than the dark mode (M = 84.000, SD = 20.961).
Therefore, It was found that there was no significant difference between dark mode and light mode, in terms of user satisfaction measures by SUS.
T-test
An independent-samples t-test was used to compare both visualizations by determining their statistical significance. The t-test helped us to determine if our hypothesis was accurate.
Final Product
With the final results and analysis from the usability study, QuizMaker was created. We took the feedback from participants in order to iterate our final website design. After our designs were final, a Usability Assessment report was given to the Project Manager and the development team in order to conduct the final iterations of the product.
Taking a Quiz (Light Mode)
Creating a Quiz (Dark Mode)
Conclusion
IBM was so impressed with our work, along with how successful the engineers were while utilizing Open Liberty, that they are continuing iterations and testings on QuizMaker in order to launch this product in August of 2021.
In all, QuizMaker facilitates collaboration among students and teachers of SUNY Oswego. It gives students as users the opportunity to effectively study with one another, and teachers as users to re-use the study material created by the students.
The product was designed and developed with stakeholder interviews, collecting research, prototyping, and respectfully iterating our designs. We presented our progress to the stakeholders, project managers, professors, and IBM stakeholders during our weekly sprints.
Challenges
One challenge was when a stakeholder created a new requirement of a dark mode option for the interface. We then had to organize a meeting with the GUI team, and re-prioritize our set tasks to ensure that we stayed within our project timeline.
Another obstacle we faced was conducting remote usability testing for the first time. However, we learned that keeping clear objectives for each task and properly scheduling each usability test with enough time to scaffold before and after each participant helped us to mitigate this challenge.
Lessons Learned
1. How to work in an Agile Scrum environment.
2. How to properly interview and communicate with stakeholders and project managers.
3. How to conduct usability studies through Zoom.
The entire project was fully remote, as we used programs such as Discord for collaboration, Monday.com for organizing each sprint, Zoom for usability testing, and Miro for brainstorming and ideation.
I was also given the incredible opportunity to present this product to stakeholders, project managers, and several incredibly talented IBM professionals.
