Remark: A Pen for Student Feedback
This project initially was a class project that I did with three other students from computer science background during my first semester studying HCI at Georgia Institute of Technology. Our design will facilitate communication between high school teachers and students by using a pen attachment and a software.
Role: Research, UI/UX design, Evaluation
Date: Aug. 2015- Dec. 2015
Keywords: Education, Communication, Physical Computing
To explore the current role of educational technology in classrooms and uncover existing problems, we visited two high schools in the greater Atlanta area and observed ten classes in a variety of subjects. Through our observations during class and interviews with teachers, we discovered one of the main problems plaguing high school classrooms is that there is a disconnect in communication between teachers and students, both in the classroom and outside. Outside of the classroom, this includes problems with distributing assignments and asking and answering questions outside of school. In the classroom, this breakdown manifests through students not speaking up in class to clarify course content, which not only negatively affects their own understanding, but that of the entire class as well. Many students at the high school level do not want others to know they do not understand and, therefore, tend to not to ask questions in class. To address this, a simple and intuitive system that helps foster this type of communication is needed in the classroom.
In order to organize and analyze the data we collected from contextual inquiries, we created different levels of data buckets, including users needs and current problems, in the form of an affinity diagram using post-it notes.
After understood users need and current problems, we had three brainstorming sessions, using divergent method first and then convergent method. We narrowed down to the idea that using a device to collect real-time data from students and a software for teachers so that they can view overtime statistics and real-time statistics.
As for the device, firstly we thought about a wearable device, like a wristband. However, a wearable device is just adding another device for students and it is not something that they would use everyday currently. Then we figured out a pen could be an option, since through our observation, each student has a pen on the table and digital devices are not prevalent. Also, a pen is portable between classrooms. The problem with a pen is that students may want to use different pens, or they may run out of ink, or they may lose the pen. Finally, we decided on a pen attachment, which not only has the features that a pen has, but also allows students to use different pens, and change their pens since the attachment is detachable.
Teachers interact with the system on two conceptually different occasions live mode, which is live visualization of the feedback from the students that is displayed for the teacher and offline mode that teacher can also interact with the system after class or when he/she is not teaching.
We assume that the teacher will need this information while teaching, and hence, our visualization needs to be comprehensible from just a quick glance at their screen. This display is extremely minimalistic and can be read even from a distance. It simply shows a dynamic changing pie chart graphing the number of students who are following the class/teaching versus the number of students who are not, for that instant, at any given point in time. For clarity, we also have a numeric representation of the same percentage of students who are following along well, alongside the graph so the teacher can choose to refer to either. This visualization does not include per student details.
Teachers can visualize the class level of understanding and each student performance over time.
Remark pen attachment
We used a potentiometer to collect feedback from students, an Arduino board to transfer the feedback to a computer and the computer can process and visualize the feedback. Swiping up on the potentiometer means students understand the material, and swiping down stands for not understood. On the other side of the pen, there is an LED to indicate students whether the feedback is collected or not. To protect students privacy, the LED uses only one color no matter the students swiped up or down.
We conducted a heuristic evaluation with five students from our program as evaluators and they were asked to participate in a mini-lecture and utilize our Remark pen attachments while listening. Evaluators were given two tasks to perform with Remark: at any time, indicate understanding of the lecture using the device and, if polled, indicate their individual response on the device. We used Nielsen’s 10 Usability Heuristics for evaluation. After the mini-lecture, we asked evaluators to write down their comments and evaluations on a sheet of paper with Nielsen’s Heuristics listed. For each item, we also asked evaluators to give us a grade from 1 (does not meet heuristics) to 4 (completely meets heuristics), and explain their gradings. Although all the evaluators were graduate students and our target users are high school students, evaluators provided usability feedback from students’ perspective. Both qualitative and quantitative data were obtained from this session.
For our heuristic evaluation, we compiled the Likert results from each of our evaluators and visualized them in the graph below. For each heuristic, each bar represents a selection along the Likert scale. The bars’ heights represent the number of users who responded at the level.
Subjects are six high school students and they were first introduced to the device’s background, purpose, and forms of interaction. Next, two users at a time tried out the device, allowing them to experience its form factor and how it could be used in a classroom setting. After all six subjects tested the device, we conducted a focus group to assess users’ opinions and thoughts on the device. We specifically focused on three areas of evaluation: form factor (the physical design of the device), usability (could they use the device without issue or confusion), and utility (would they actually use this device if it was employed in their classroom). We received extremely helpful feedback and suggestions from our users. Customizability is not really a big factor here like we thought it was; the swipe direction mapping is intuitive one learned; it’s not that students don’t want to ask questions in the class, they would rather ask it when they know there is a consensus in the class; and students can even use this to tell the teacher to “move on” when they understand.
- What worked:
The swipe action
Light as a feedback system
The device as an attachment
- What did not work:
Proposal of using vibration instead of light as feedback
Position of the swipe area
Inability to maintain and show status of the comprehension value of a student
error detection & notification
Based on the findings that our group has collected throughout our development process, heuristic evaluation, and user testing session we have decided on a few changes and improvements that we would like to implement to our Remark pen. These redesigns include having an additional user dashboard to go along with the teacher visualization interface, adding a lock and unlock feature to the device, incorporating up and down swipe direction labels, decreasing the size of the feedback light, and adding a function to show a student’s current swipe status. Unlike the prototype, the final product would be Bluetooth enabled, battery powered, and wireless, taking away the concerns of weight added from the wiring.
After the divergent brainstorm session, we generated more than thirty ideas. Since each of them could be finalized and become a solution, we had a hard time to narrow them down to one solid idea. To understand users need more comprehensively, we went back to the data we collected from contextual inquiries and sent out more surveys.
For cost concerns, we prototyped six pen attachments, and everything worked properly, ready for evaluation. Accordingly, we invited six local high school students to come to Gatech and perform the usability testing. However, three hours before the testing, when we were setting up, four devices did not work. We tried to fix them, but it did not help. Since we did not want to cancel the evaluation, we decided to do a focus group instead of usability testing. We spent half an hour and came up with some questions for evaluators. Eventually, the evaluation went smoothly, and we gained lots of constructive feedback.