Solving a math word problem requires understanding the mathematical components of the problem and an ability to decode the text. For some students, lower reading comprehension skills may make engagement with the mathematical content more difficult.
Readability formulas, such as Flesch Reading Ease, are frequently used to assess reading difficulty. However, math word problems are typically shorter than the texts traditional readability formulas were designed to analyze.
To identify metrics relevant to assessing the reading difficulty of math word problems, the authors identified 28 candidate features that may predict math word problem readability.
The authors then assessed the performance of 297,072 middle and high school students completing word problems in an intelligent tutoring system as part of standard educational practice. From this, they identified 4,446, out of 9,421, problems where performance gaps between predicted less- and more-skilled readers were significantly larger than typical gaps between the groups.
Finally, the authors tested how well the readability metrics could identify problems with performance gaps. Of five models tested, a random forest had the best predictive accuracy, AUC = .75.
The findings suggest that readability of the text played some role in decreased performance among less-skilled readers and provide a path toward better understanding how to assess the readability of math word problems and make them more accessible to less-skilled readers.