Education is increasingly taking place in technology-mediated learning environments, making it easier to collect student-generated data such as comments in discussion forums and chats.
Although this data can be valuable to researchers, it often contains sensitive information such as names, locations, social media links, and other personally identifying information that must be carefully redacted before being used for research.
Historically, personally identifying information has been redacted by humans. More recently, researchers have also explored regular expressions and supervised machine-learning methods.
This paper assesses GPT-4’s performance in de-identifying data from discussion forums in nine Massive Open Online Courses.
The results show an average recall of 0.958 for identifying personally identifying information that needs to be redacted, suggesting that GPT-4 is an appropriate tool for this purpose. The tool was also successful at identifying cases missed by humans during redaction.
The findings indicate that GPT-4 can increase the efficiency and enhance the quality of the redaction process. However, precision was considerably lower at 0.526, with the tool over-redacting names and locations that did not represent personally identifying information, showing a need for further improvement.