It's hard to go anywhere these days without seeing a laptop, notepad or smart phone. As parents, many of us think our kids spend entirely too much time connected to technology.
What if we told you researchers have developed new computers that respond to your child's boredom or frustration and may actually help them learn in an exciting new way?
It's happening, and one of the developers teaches right here at Notre Dame.
Sidney D'Mello, Ph.D., Professor Of Psychology, Notre Dame, “What we've done with this is, as this dialog is going on, it's now able to sense emotion through facial features, a lot of dynamics of this conversation and body movements. So if you take a really really long time to respond and you are furrowing your brow you're more likely to be confused. That's how it kind of pieces these cues together. So it asks the question and the students give these two or three word answers, but then ii engages in this conversation to kind of eek out what the students know. So they themselves kind of construct the answers. It does that by giving hints, prompts. So this tutor is actually having a conversation based on the answers they get from the student, exactly.”
It may seem like futuristic software, but the future is here.
Notre Dame Psychology Professor Sidney D'Mello started working on the program while he was at the University of Memphis, along with researchers from MIT.
Their focus is a program called Auto Tutor, developed to help students learn complex subjects, like physics, math and sciences.
What it does is amazing, actually teaching complex content by holding conversations and tailoring interaction for individual students.
D’Mello describes how much it can interact with someone, “It's special from all the others because it has conversations with your natural language.”
D'Mello and his colleagues went to great lengths to make sure the auto tutor challenges students, “It asks questions, and it comprehends your responses and its modeled after actual human tutors.”
D’Mello was going to great lengths to get it right, “We started out by videotaping 50 top 100 hours of actual human tutoring, in tutoring programs all over, codes them and actually understood what human tutors do that makes them so effective and then lifted those models and then put them in a computer.”
Emotions that are relevant to learning that you don't often see in one on one learning sessions. Emotions he is now studying at Notre Dame.
D’Mello explains the emotions they are studying now, “The ones we're really focusing on right now are boredom, confusion, frustration and being able to pick them out and distinguish them from a state of neutral, where there's no emotion.”
Using cameras to watch the face for frowning and confusion and cushions that can pick up body language
D’Mello describes what other approaches have focused on, “So a lot of the older approaches have focused on students cognitive states, cognitive abilities and that's really important but they've sort of ignored the emotional aspect.”
Kids who are struggling disengage and D'Mello says this emotion sensing program won't let that happen.
D’Mello explains how the program waits for the perfect moment to give a hint and help the student, “The system can sense when you're confused and then at the right moment come in with a hint. So you can get out of that confusion yourself. If you get frustrated because you are stuck, it can empathize with you to understand you're frustrated with this task, it's not you, you're smart. You just need to process things and think deeper.”
D'Mello says if you're bored it can also keep you motivate on task, “It's adapting to your emotions as well as your cognition. Tracking boredom, this is constantly sensing and sending information to the tutor and that's how its responding.”
It won't replace teachers, but D'Mello says one on one tutoring is the gold standard for which they based their program.
D’Mello explains how the program can relate to real human tutors, “We've compared them directly, equating content, to actual human tutors and we find we can't beat the human tutors but we can do as well as they can.”
Mind versus machine, perhaps is the new gold standard when it comes to learning that involves emotion.
So when might we see these emotion sensing computers in the classrooms?
D'Mello says they are working on the funding needed to get it to students, but he hopes it will be ready for the general population in two to three years.
To read more about Professor D'Mello's research click here.