Humanities and Social Sciences Communications 2024
What factors will affect the effectiveness of using ChatGPT to solve programming problems? A quasi-experimental study
Corresponding author: Chengliang Wang
DOI 10.1057/s41599-024-02751-w
First published online: 2024-02-26.
Research summary
Using Python graphing tasks, this quasi-experimental study examines why learners differ in their effectiveness when using ChatGPT to solve programming problems. AI literacy, programming knowledge and understanding of ChatGPT are associated with effective use. The study connects the design of AI-supported programming activities with learners' existing capabilities.
Study design.
Quasi-experimental study in a Python graphing and programming problem-solving context.
Abstract
The emergence of ChatGPT has sparked new expectations for AI-empowered educational transformation. However, it remains unknown which factors affect its effectiveness in empowering learners to solve programming problems. Therefore, this study employed a quasi-experimental research design and used Python graphing in programming education as an example to investigate the factors influencing the effectiveness of learners in applying ChatGPT to problem-solving. Findings: AI literacy significantly influences learners’ effectiveness in using ChatGPT to solve problems, with AI awareness and AI usage being key factors. The knowledge base of programming language significantly affects learners’ effectiveness in applying ChatGPT to solve programming problems. Learners’ cognitive level of ChatGPT significantly influences their effectiveness in applying ChatGPT to problem-solving, while usage intention does not have a significant impact. However, learners’ intention to use ChatGPT significantly improves after application. Based on these findings, this study proposes that in the process of empowering education with Artificial Intelligence Generated Content (AIGC) products, the focus on learners should shift from cultivating their AI usage to AI literacy, laying the foundation for empowering learning with AIGC products. It is suggested to shift from mastering specific knowledge to graph-based rules as a method for empowering learning with AIGC products. Additionally, the focus should shift from enhancing learners’ intention to use the technology to strengthen their technological awareness, thereby creating practical pathways for empowering learning with AIGC products.
Research context
What learner characteristics matter when ChatGPT is used to solve programming problems?
Use this paper for learner readiness, AI literacy and ChatGPT-assisted programming. The reported relationships refer to the study's tasks and measured learner characteristics.
ChatGPT · programming problem solving · AI literacy · learner readiness
中文研究介绍
研究以Python绘图任务为情境，通过准实验考察学习者使用ChatGPT解决编程问题的差异，分析AI素养、编程知识和对ChatGPT的认知与使用效果的关系，为根据学习者已有能力设计AI支持的编程活动提供依据。
Read and cite
Jing, Y., Wang, H., Chen, X., & Wang, C. (2024). What factors will affect the effectiveness of using ChatGPT to solve programming problems? A quasi-experimental study. Humanities and Social Sciences Communications, 11(1), 319. https://doi.org/10.1057/s41599-024-02751-w