Showing progress towards a goal is a well-established motivational design tactic. This paper describes how university students designed a new “progress meter” for an online learning game and then evaluated the effects of the design using a controlled online experiment, or A/B test.
Using the UpGrade A/B testing platform, the study randomly assigned 3,200 online players to the original game or to an updated version of the game with progress meters. The authors hypothesized that progress meters would significantly increase student engagement, measured as voluntary time on task and number of items completed.
The new design significantly reduced player engagement by approximately 15%. Rather than treating this finding as evidence against progress meters generally, the authors conclude that the appropriate response is to keep testing new iterations of the game mechanic.
The paper points toward a future where instructional designers and learning engineers can continuously improve online education through rapid cycles of design and A/B testing.