The paper titled “Comparing the quality of human and ChatGPT feedback of students’ writing” delves into the realm of automated writing evaluation (AWE) and examines the effectiveness of generative AI, such as ChatGPT, in providing formative feedback to students. The study compares the quality of feedback provided by human evaluators and ChatGPT on secondary student essays, focusing on criteria-based feedback, clear directions for improvement, accuracy, prioritization of essential features, and a supportive tone.
The primary aim of this paper is to explore the potential of AI, represented by ChatGPT, in offering formative feedback to students, thereby reducing the burden on educators to provide extensive feedback. By assessing the quality of feedback from both human raters and AI, the study aims to shed light on the effectiveness of AI-generated feedback compared to human feedback. This research seeks to inform educational practices by evaluating the feasibility and utility of integrating AI in the feedback process for student writing.
Key Takeaways for Educators:
- Quality of Feedback: The study reveals that well-trained human evaluators outperformed ChatGPT in providing high-quality feedback across various aspects, except for criteria-based feedback. Educators should be aware that while AI like ChatGPT can offer feedback efficiently, human evaluators still excel in providing nuanced and high-quality feedback to students.
- Feedback Customization: Educators should consider the importance of clear and precise feedback that offers specific directions for improvement. Feedback that is accurate, prioritizes essential writing features, and maintains a supportive tone is crucial for guiding students towards enhancing their writing skills effectively.
- AI Integration: Despite the superiority of human evaluators in feedback quality, the study suggests that generative AI like ChatGPT can still be valuable in certain contexts, especially for providing feedback on early drafts or when human resources are limited. Educators should explore the potential of AI tools like ChatGPT as supplementary aids in the writing instruction process.
- Timely Feedback: While educators face challenges in providing timely feedback to students, AI tools like ChatGPT offer the advantage of immediate and iterative feedback. Integrating AI technologies can enhance the timeliness of feedback, supporting students in their writing development more efficiently.
AI Advisory Boards 