R3 1.13 July 5, 2023 User Perspectives on ChatGPT and Education
Are you presenting about ChatGPT to faculty audiences? Read this article first.
In this issue of R3, I’m taking a look at a new study on ChatGPT in education, a project designed to uncover major themes and concerns that educators are expressing at this early stage of the game. This article could be an incredibly useful source of quotes, illustrations, and discussion prompts for those presenting about AI to faculty audiences, which many faculty developers will be asked to do this coming academic year if not sooner.
Citation:
Mogavi, R. H., Deng, C., Kim, J. J., Zhou, P., Kwon, Y. D., Metwally, A. H. S., Tlili, A., Bassanelli, S., Bucchiarone, A, Gujar, S., Nacke, L.E., & Hui, P. (2023). Exploring user perspectives on ChatGPT: Applications, perceptions, and implications for AI-integrated education. Preprint/early stage research posted by the authors. Retrieved from http://arxiv.org/abs/2305.13114
Paywall or Open:
Open
Summary:
Social media posts were systematically analyzed to uncover the range and prevalence of opinions about ChatGPT in education, especially among early adopters of the technology. These opinions included uses of ChatGPT for different teaching and learning tasks, positive perceptions about the technology’s potential, and concerns. These themes form the basis for recommendations about how best to incorporate ChatGPT into classes.
Research Questions (From the article):
“• RQ1: What are the primary applications of ChatGPT in education, according to early adopters?
• RQ2: What are the perceptions of early adopters in the education sector regarding ChatGPT? (Is ChatGPT a blessing or a curse?)”
Sample:
Content posted on major social media platforms by users involved with education in some capacity. Most, but not all, of the posts originated in North America (55%) or Northern Europe (21%), although 36% could not be localized.
Method/Design:
Researchers used a systematic method for sampling from the top 25% most liked/viewed posts on 4 social media platforms that included keywords relating to 1) AI and 2) the poster’s status as an early adopter of educational technology. Researchers used inductive coding to explore and identify major themes.
Key Findings:
Users posted a great deal of positive commentary about ChatGPT, praising its uses for their own research and content generation work (e.g., using it to create drafts of presentations, create prompts for writing assignments, or distill large amounts of research materials). Within higher education, social sciences, business/management, and STEM were the disciplines with the highest proportion of early adopters. Negative themes emerged as well, with educators at all levels expressing concern that the technology would hamper the development of critical thinking, encourage students to accept information without verifying it, and demotivate students through the perception of an easy route to generating classwork. There were also comments lamenting the difficulty of defining and managing academic dishonesty involving ChatGPT.
The article concludes with five sets of recommendations for practice: developing instructor digital literacy connected to AI; incorporating appropriate use of ChatGPT into learning objectives, using Bloom’s taxonomy as a guiding framework; developing and communicating class policies about ChatGPT; refining and enhancing how learning is assessed overall; developing prompts for ChatGPT, generating content, and determining how and whether ChatGPT will be cited for that content.
Choice Quote from the Article:
“The purpose of this study is to explore early adopters’ experiences and thoughts about integrating ChatGPT into education by analyzing their social media content on various platforms, i.e., Twitter, Reddit, YouTube, and LinkedIn. Using social media content analysis, we can extract valuable themes and helpful insights from social media data on a larger scale, ultimately leading to the identification of the primary applications (RQ1) and positive and negative perceptions of early adopters’ use of ChatGPT in education (RQ2).”
Why it Matters:
This is one of what will surely be many more articles on the different applications of ChatGPT to education. One useful feature of this particular article is the concise introduction to some key technical concepts, such as deep learning, transformer models, and how different AI systems are trained. There’s also a good summary of the recent advances in natural language processing and how those have come about, along with a more detailed description of the GPT models in particular and how they work. Managing “long-range dependencies” (which I take to mean correlations and links among different elements of text) has been a key advance in creating systems that can interpret and produce human-like communication.
The insights the researchers gathered from early adopters are also a good inspiration for those of us wanting to either incorporate ChatGPT into instruction, build safeguards against unsanctioned uses of it, or some combination of both. I liked the set of word clouds (figure 4) they provided, broken down by educational context (K-12, higher ed, lifelong learning and so on) illustrating the top uses for ChatGPT identified by different groups. I also appreciated the detailed set of recommendations offered at the end of the article.
I think this article could be particularly useful for faculty developers and academic leaders tasked with presenting about ChatGPT to faculty audiences. I say this because it includes so many diverse, relatable examples from ChatGPT users effusing about all its potential uses - everything from creating rubrics to drafting grant proposals. These first-hand experiences, related by fellow educators, could spark new ideas among faculty audiences or at the least, help demystify the ways in which ChatGPT may transform academic work in the future.
Most Relevant For:
Instructional designers preparing to handle faculty questions about generative AI; leaders responsible for policy and practice involving educational technology; teaching and learning center directors; researchers designing their own studies on AI in education
Limitations, Caveats, and Nagging Questions:
My understanding is that this work is still subject to review and revision. Normally I try to limit in-depth review of research to work that’s already made it through peer review, but the pace of ChatGPT’s expansion into education justifies taking a look anyway – especially if the work is going to be used as a source of ideas, examples, and background, rather than as the definitive last word on faculty attitudes toward AI.
Keep in mind that the focus here is on what people are saying and thinking about ChatGPT, not on documenting the impacts or carrying out formal experiments. Those studies will come, and in the meantime, this one offers a good sense of the range of possibility, along with some helpful foundational knowledge about the technology.
One other caveat has to do with the data gathering methodology the team used. At the most general level, the techniques – gathering publicly available posts and other content from popular social media platforms – are not overwhelmingly complicated or technical. But getting into the specifics on how they did what they did was a bit challenging given that it involved several systems (Helium, Atlas.ti) that I don’t have first-hand experience with. Traditional social scientists like myself, and other non-specialists, might have to take some aspects of the methodology on faith.
If you liked this article, you might also appreciate:
Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3). https://doi.org/10.30935/cedtech/13152
Darby, F. (2023, June 27). 4 steps to help you plan for ChatGPT in your classroom. The Chronicle of Higher Education.
Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments. https://doi.org/10.1080/10494820.2023.2209881
File under:
ChatGPT; predictive AI; educational technology; early adopters; faculty development; non-degree/non-traditional learners