A product designer and Lego enthusiast
thumbnail remark 1.png

Remark

Remark: A Pen for Student Feedback

 
 

This project initially was a class project that I did with three other students from computer science background during my first semester studying HCI at Georgia Institute of Technology. Our design will facilitate communication between high school teachers and students by using a pen attachment and a software.

Role: Research, UI/UX design, Evaluation

Date: Aug. 2015- Dec. 2015

Keywords: Education, Communication, Physical Computing

 
 
 

CONTEXTUAL INQUIRIES

To explore the current role of educational technology in classrooms and uncover existing problems, we visited two high schools in the greater Atlanta area and observed ten classes in a variety of subjects. Through our observations during class and interviews with teachers, we discovered one of the main problems plaguing high school classrooms is that there is a disconnect in communication between teachers and students, both in the classroom and outside. Outside of the classroom, this includes problems with distributing assignments and asking and answering questions outside of school. In the classroom, this breakdown manifests through students not speaking up in class to clarify course content, which not only negatively affects their own understanding, but that of the entire class as well. Many students at the high school level do not want others to know they do not understand and, therefore, tend to not to ask questions in class. To address this, a simple and intuitive system that helps foster this type of communication is needed in the classroom.

 

AFFINITY DIAGRAMMING

In order to organize and analyze the data we collected from contextual inquiries, we created different levels of data buckets, including users needs and current problems, in the form of an affinity diagram using post-it notes. 

 
 
 
 
 

BRAINSTORMING

After understood users need and current problems, we had three brainstorming sessions, using divergent method first and then convergent method. We narrowed down to the idea that using a device to collect real-time data from students and a software for teachers so that they can view overtime statistics and real-time statistics

As for the device, firstly we thought about a wearable device, like a wristband. However, a wearable device is just adding another device for students and it is not something that they would use everyday currently. Then we figured out a pen could be an option, since through our observation, each student has a pen on the table and digital devices are not prevalent. Also, a pen is portable between classrooms. The problem with a pen is that students may want to use different pens, or they may run out of ink, or they may lose the pen. Finally, we decided on a pen attachment, which not only has the features that a pen has, but also allows students to use different pens, and change their pens since the attachment is detachable

 
 
diagram.png
 
 
 

PROTOTYPING

Remark software

Teachers interact with the system on two conceptually different occasions live mode, which is live visualization of the feedback from the students that is displayed for the teacher and offline mode that teacher can also interact with the system after class or when he/she is not teaching.

Live view:

 
 
 
 

We assume that the teacher will need this information while teaching, and hence, our visualization needs to be comprehensible from just a quick glance at their screen. This display is extremely minimalistic and can be read even from a distance. It simply shows a dynamic changing pie chart graphing the number of students who are following the class/teaching versus the number of students who are not, for that instant, at any given point in time. For clarity, we also have a numeric representation of the same percentage of students who are following along well, alongside the graph so the teacher can choose to refer to either. This visualization does not include per student details.

Offline view:

 
 
 
 

Teachers can visualize the class level of understanding and each student performance over time.

Mockups

 
 
  After the teacher has finished the class, teachers can view more statistics about past classes. They can click on any point on the line chart to view some general information about that point, including when that happened, and what the content was. 

After the teacher has finished the class, teachers can view more statistics about past classes. They can click on any point on the line chart to view some general information about that point, including when that happened, and what the content was. 

 
 
  Clicking on "view details" on My Class page will lead users to a new page where they will be able to view more detailed information, as well as students who gave positive/ negative/ none responses. 

Clicking on "view details" on My Class page will lead users to a new page where they will be able to view more detailed information, as well as students who gave positive/ negative/ none responses. 

 
 
 Besides that, teachers can also view statistics by students. The circles around students profile photos give a quick look about the average percentage of understanding during the past class period. 

Besides that, teachers can also view statistics by students. The circles around students profile photos give a quick look about the average percentage of understanding during the past class period. 

 
 
 Furthermore, teachers can click on each student’s photo or name to see some details. For example, their responses during the given time period, and when those responses happened.

Furthermore, teachers can click on each student’s photo or name to see some details. For example, their responses during the given time period, and when those responses happened.

 
 

 

Remark pen attachment

 
 

We used a potentiometer to collect feedback from students, an Arduino board to transfer the feedback to a computer and the computer can process and visualize the feedback. Swiping up on the potentiometer means students understand the material, and swiping down stands for not understood. On the other side of the pen, there is an LED to indicate students whether the feedback is collected or not. To protect students privacy, the LED uses only one color no matter the students swiped up or down.

 
 

 

EVALUATION

 

Heuristic Evaluation

We conducted a heuristic evaluation with five students from our program as evaluators and they were asked to participate in a mini-lecture and utilize our Remark pen attachments while listening. Evaluators were given two tasks to perform with Remark: at any time, indicate understanding of the lecture using the device and, if polled, indicate their individual response on the device. We used Nielsen’s 10 Usability Heuristics for evaluation. After the mini-lecture, we asked evaluators to write down their comments and evaluations on a sheet of paper with Nielsen’s Heuristics listed. For each item, we also asked evaluators to give us a grade from 1 (does not meet heuristics) to 4 (completely meets heuristics), and explain their gradings. Although all the evaluators were graduate students and our target users are high school students, evaluators provided usability feedback from students’ perspective. Both qualitative and quantitative data were obtained from this session.

For our heuristic evaluation, we compiled the Likert results from each of our evaluators and visualized them in the graph below. For each heuristic, each bar represents a selection along the Likert scale. The bars’ heights represent the number of users who responded at the level.

 
 

 

User Testing  

Subjects are six high school students and they were first introduced to the device’s background, purpose, and forms of interaction. Next, two users at a time tried out the device, allowing them to experience its form factor and how it could be used in a classroom setting. After all six subjects tested the device, we conducted a focus group to assess users’ opinions and thoughts on the device. We specifically focused on three areas of evaluation: form factor (the physical design of the device), usability (could they use the device without issue or confusion), and utility (would they actually use this device if it was employed in their classroom). We received extremely helpful feedback and suggestions from our users. Customizability is not really a big factor here like we thought it was; the swipe direction mapping is intuitive one learned; it’s not that students don’t want to ask questions in the class, they would rather ask it when they know there is a consensus in the class; and students can even use this to tell the teacher to “move on” when they understand. 

- What worked: 

  The swipe action

  Light as a feedback system 

  The device as an attachment

  Audio recording

- What did not work: 

   Proposal of using vibration instead of light as feedback

   Position of the swipe area

   Inability to maintain and show status of the comprehension value of a student

   error detection & notification

 

REDESIGN

Based on the findings that our group has collected throughout our development process, heuristic evaluation, and user testing session we have decided on a few changes and improvements that we would like to implement to our Remark pen. These redesigns include having an additional user dashboard to go along with the teacher visualization interface, adding a lock and unlock feature to the deviceincorporating up and down swipe direction labelsdecreasing the size of the feedback light, and adding a function to show a student’s current swipe status. Unlike the prototype, the final product would be Bluetooth enabledbattery powered, and wireless, taking away the concerns of weight added from the wiring.

 

REFLECTIONS

Challenges

Convergent brainstorm:

After the divergent brainstorm session, we generated more than thirty ideas. Since each of them could be finalized and become a solution, we had a hard time to narrow them down to one solid idea. To understand users need more comprehensively, we went back to the data we collected from contextual inquiries and sent out more surveys.  

 

Evaluation:

For cost concerns, we prototyped six pen attachments, and everything worked properly, ready for evaluation. Accordingly, we invited six local high school students to come to Gatech and perform the usability testing. However, three hours before the testing, when we were setting up, four devices did not work. We tried to fix them, but it did not help. Since we did not want to cancel the evaluation, we decided to do a focus group instead of usability testing. We spent half an hour and came up with some questions for evaluators. Eventually, the evaluation went smoothly, and we gained lots of constructive feedback.