CS Students Publish Reading Skill Study

Last summer, computer science professor Julie Medero, along with Alfredo Gomez ’21, Alicia Ngo ’20 and Ali Otondo ’20 (aka The A Team), embarked on a project to develop an iOS application that would help children improve their reading skills. The resulting research paper, “Reading KiTTY: Pitch Range as an Indicator of Reading Skill,” has been accepted to the Widening Natural Language Processing (WiNLP) workshop, to be held during the Association for Computational Linguistics conference later this summer in Florence, Italy.

“This paper is about an analysis of the prosody of children’s oral reading,” says Medero (prosody refers to the patterns of rhythm and sound in text, e.g., pitch, reading speed, emotionality). “That means we’re looking at how children of different reading levels use their voices as part of reading out loud. Our lab’s Reading KiTTY project is looking at how elementary-aged children could be guided through the creation of kinetic typography animations that visualize their reading out loud. Kinetic typography is a form of animation that uses size, color and motion of text, along with images, to represent the meaning of a text. It’s popular in music videos and is also commonly used in videos of famous speeches, but we think it has the potential for interesting applications in literacy education, too.”

“The research focuses on two aspects,” says Gomez, who will make a presentation at the workshop, “creating the first iteration of the app and exploring how we could leverage natural language processing and speech processing in order effectively promote creativity. The paper accepted by WiNLP focuses on our work using pitch range as an indicator for reading skill as we apply machine learning and other computational linguistics techniques.”

Ngo explains how the app works: “First, children read aloud for the app. After they finish reading, children can see their words come to life through kinetic typography. For example, if a child reads a question prosodically, the kinetic typography should show a pitch rise at the end of the sentence, perhaps by positioning the high-pitched letters higher (along the vertical axis) than the other letters. Using our results, we intend to provide teachers with unique feedback of each student’s prosody, and therefore, each student’s reading comprehension.”

Ngo continues, “Alfredo and I trained a machine-learning model with scikit-learn to predict the presence of a high pitch in a text, similar to developing text-to-speech software. After realizing that was really difficult (Google hasn’t even figured this one out completely yet), we decided to conduct some more data analyses to understand our dataset. We analyzed and extracted data from 5,000+ audio files of children reading, using Natural Language Toolkit and NLP software (AuToBI and Praat). We found that there is a statistically significant difference in the average pitch range between skilled readers and struggling readers as they read sentences.”

The researchers hope to add more ways to gauge a student’s reading comprehension to include reading speed, emotionality/enthusiasm and the reader’s pause lengths between words.

“This project is important because teachers often don’t have the time to listen to each individual student read,” says Ngo. “Reading KiTTY can provide unique feedback for each student, so that teachers can focus on improving their students’ reading comprehension instead of merely measuring their students’ reading comprehension.”