R3 3.1 January 23, 2025 What If It Never Happened? Learning Through Counterfactual Reasoning
One way to promote critical thinking in a discipline is to ask students to envision what would have happened if crucial events had unfolded just a bit differently.
Welcome back to the R3 newsletter with this, the first issue of 2025! This coming year, you can expect more of the themes and topics this Substack has focused on from the beginning. There’s thinking, memory and how those two processes complement one another in learning; there’s focus, attention, and how to build these capacities among our often-distracted students. Last, there’s motivation, metacognition, and engaging students in the hard work of learning. I’ll also – no surprise, I’m sure – be continuing to dip into the topic of generative AI when new research comes out, which it is now doing at a rapid clip.
I’ve also become interested in how we in higher ed are approaching AI from a faculty and organizational perspective, and if you are as well, you may want to check out Generative AI in higher education: The good, the bad, and the ugly. This brand-new edited collection covers a range of perspectives and topics, including my own chapter “Generative AI as a challenge to faculty development: Ugly advice at the dawn of generative AI.” In it, I pull together what I see as better and worse ways to engage faculty in the daunting task of incorporating the new AI tools into teaching, research, syllabus policies and more.
This year, I’m also going to be emphasizing an area where I want to deepen and expand on my own knowledge: critical thinking. This is something I’ve written about on and off since discussing it in my first book, but now I want to reconnect with new research that has come out since then. In the talks and workshops I’ve given in the intervening years, I’ve noticed some interesting ways in which faculty approach this aspect of teaching. If you ask a room full of faculty how many of them list “critical thinking” in their syllabus, learning objectives, curriculum maps and so one, a forest of hands will inevitably go up. This is not mere box-ticking, either. Ask faculty what they mean by critical thinking and you will hear a glorious medley of cherished aspirations they have for their students, goals reflecting the core values of their various disciplines. It seems like every field out there, from business to STEM to nursing to art history, prizes this one aspect of learning, and faculty are extraordinarily motivated to make sure their students develop it.
That brings us to the central challenge of the subject: that even though we all want to see our students develop critical thinking, that goal often eludes our grasp. Partly this is because of the difficulty of defining it in an abstract or discipline-independent way, which in turn complicates the task of designing instruction that targets it effectively. I don’t place much stock in the many definitions of critical thinking I’ve seen that are disembodied from a particular disciplinary setting, nor do I think that those kinds of definitions are all that helpful. As I counsel faculty, it is okay to come up with our own descriptions of what critical thinking looks like in the context of one discipline or even a single course. But coming up with that contextualized definition requires some fairly intensive reflection, and although it’s a necessary first step, it’s not sufficient to simply list all the things we want students to be able to do in this arena, not if we want to make significant advances in their ability to do it.
Besides the problem of pinning down what it is, critical thinking involves a number of other barriers identified by experts in the field. It’s extraordinarily effortful, requiring a shift away from easier forms of reasoning such as pointing to catchy examples or relying on what everyone else seems to think. Experts also stress that it’s not just a matter of learning how to think critically, but also having the inclination to decide to do it in a given situation or setting – in other words, it is a disposition as well as a skill. Sometimes I compare this kind of thinking to walking on your hands: a special ability that can be developed through practice and activated intentionally in a given situation but which is always more effortful, and frequently uncomfortable, than the default mode.
Critical thinking is desirable, and difficult, and a moving target that we all still need to center in our sights if students are going to become bona-fide experts in a field. I’ll probably be returning to the topic quite a bit this year, and in this issue, I’m going to start relatively small with a recent article on a creative approach to stimulating discussion, and presumably, deeper thought about concepts in psychology courses. I do think there is cross-disciplinary applicability here as well, and so I wanted to share this at a time of the year when (for those on traditional semester calendars) there’s still plenty of time to build new activities and discussions into spring courses.
Citation:
Innes, J. M., & Morrison, B. W. (2024). Teaching psychology that does not exist: Counterfactuals as catalysts for critical thinking in psychology education. Scholarship of Teaching and Learning in Psychology. Advance online publication.
DOI:
https://dx.doi.org/10.1037/stl0000402
Paywall or Open:
Open
Summary:
This study presents a structured teaching exercise aimed at fostering critical thinking through counterfactual analysis. By encouraging students to consider "what if" scenarios involving important events in the field, the exercise helps them explore causal pathways for psychological phenomena, a key aspect of critical thinking in this discipline. Students work through a structured process involving one event that could have unfolded differently, in which they predict potential outcomes and reflect on their broader implications. The approach emphasizes interdisciplinary thinking and the development of core skills such as critical and creative analysis. Note: This is a short, SOTL-style article without extensive methodology or statistical analysis sections, so my summary and discussion will be a bit different than it typically would be.
Research Questions:
- Can structured counterfactual exercises help students explore causal relationships and interdisciplinary connections within psychological concepts?
- How do students typically respond to the activity as presented in a psychology course?
Sample:
The exercise was initially implemented with third-year undergraduate psychology students who had prior training in statistics and causal analysis. It was later tested with postgraduate psychology students and would also be adaptable for introductory psychology courses.
Method/Design:
The exercise is structured into steps:
Identifying the Event: Choose the event and describe it, with emphasis on its relevance and importance within the discipline. The class can collaborate to identify the event of interest.
Example: the Stanford Prison Experiment, a 1970s-era realistic simulation of a lock-up where “jailers” had nearly unlimited power over “prisoners,” and which has been significantly criticized on ethical grounds.
Proposing Minimal Changes: Propose and describe a relatively small change or set of changes to the event.
Example: What if the Stanford Prison Experiment had followed stricter ethical guidelines of the type we would see today in psychology research?
Predicting Outcomes: Students identify multiple potential outcomes of the hypothetical change, emphasizing the complexity of causal relationships.
In general, students should be able to provide at least 3-4 specific predictions.
Reflecting on Significance: Students summarize the processes and outcomes, addressing the broader implications and exploring the interconnected consequences of the event.
This can be framed as answering the “so what” question. What is different in this counterfactual reality, and why is that important?
Key Findings:
Students had positive reactions overall to the exercise, describing it as fun and engaging. The researchers also observe that the exercise is constructed in a way that promotes psychological safety, given that the focus is on generating and reflecting on ideas rather than identifying one “correct” answer (and risking being wrong in front of one’s teacher and peers).
Choice Quote from the Article:
By requiring individuals to step outside of their current perspective and consider alternative viewpoints, thinking counterfactually enables them to acquire a broader understanding of a given situation. Similarly, exploring alternative solutions to problems and considering how they might have produced different outcomes is fundamental to critical thinking. Speculating about the causes and effects of different scenarios can enhance critical thinking by helping us recognize the complex web of factors that contribute to a particular outcome.
Why it Matters:
I suspect that the counterfactual approach is something that many experienced instructors across disciplines already engage in, even if they don’t use the formal term. I like having the technique formalized and broken down in a more systematic way, as the authors do quite skillfully in this article. I think it’s good to note that the technique as presented goes well beyond just posing what-if questions to the class; although this wouldn’t be a bad way to kick off a class discussion, the systematic steps listed here should help students slow down and engage more deeply in effortful critical thinking.
The authors also mention, almost in passing, something that I think brings a lot of appeal to this exercise: the growing popularity of counterfactual-style narratives in TV, film, and other media. Any class is near-guaranteed to have at least a few fans of shows involving alternate realities, revised histories, multiverses and so on, and their enthusiasm could be contagious – especially if the instructor deliberately points out the connection.
As the authors emphasize, this isn’t just a flight-of-fancy or brainstorming exercise, but one that explicitly directs students to look at cause and effect and be able to explain their reasoning. And although this is not a major focus of the write-up, I particularly like how the exercise itself links factual knowledge (e.g., knowing the details of the Stanford Prison Experiment and how it was conducted, knowing current guidelines for ethical treatment of human research volunteers) and reasoning (explaining how the results would have come out differently with changes to the study protocol and how the wider impacts of the study might have been different). In my writing on the importance of memory in higher education pedagogy, a point I often return to is that rather than seeing the development of knowledge and thinking skills as separate and perhaps opposing goals, we should instead recognize how the two complement one another. Knowing facts about the Stanford Prison Experiment is crucial for thinking critically about how that study could have been done differently, and completing a deep-thought exercise about the experiment reinforces all the factual knowledge that’s associated with it.
It also occurred to me that this exercise could work well online as well as in face-to-face classes. It could perhaps be redesigned as well as an AI-focused exercise, although I would advise expanding on the reflection stage and critique of any AI output, in order to ensure that students are still engaging in the critical thinking process themselves.
Most Relevant For:
Instructors in foundational or lower-division courses; teaching assistants and discussion facilitators
Limitations, Caveats, and Nagging Questions:
Along with the focus on critical thinking, the article alludes to other cognitive processes which are also desirable and worthy of cultivation, but not the same as critical thinking. For example, students might point out the historical impacts of the Stanford Prison Experiment – a type of interdisciplinary thinking noted by the authors. Creativity is involved too, as students generate new combinations of facts and predictions. For me, all of this muddies the water a bit as far as the interpretation, but wouldn’t necessarily limit the usefulness of the exercise itself.
As I read over the article, it struck me that students could be more or less involved in the first two steps of the process – the ones involving choosing an event and proposing the “minimal change” that’s the basis of the counterfactual exercise. The instructor could present the students with a fully-formed scenario instead of having students come up with one, but I’m not sure what the impact on the overall outcomes would be.
Lastly, as I mentioned above, this is a SOTL-style sharing of a practice, not an elaborate set of experiments, so the outcome measures primarily focus on student and instructor impressions. This is, I think, adequate as a basis for adapting the exercise and using it, but doesn’t get into the territory of demonstrating impact on specific aspects of critical thinking skill.
If you liked this article, you might also appreciate:
Bamberger, M. R., Campbell, C., & Smith, T. J. (2023). Priming metacognition and critical thinking to dispel psychological misconceptions. Scholarship of Teaching and Learning in Psychology, 10(4), 471–487. https://doi.org/10.1037/stl0000361
Blessing, J. S. (2023). Busting myths to increase critical thinking in psychology majors. Teaching of Psychology. https://doi.org/10.1177/00986283231166954
Evans, J. S. B. T., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3), 223–241. https://doi.org/10.1177/1745691612460685
Greenwood-Hau, J. (2024). Teaching facts or teaching thinking? The potential of hooks’ ‘engaged pedagogy’ for teaching politics in a ‘post-truth’ moment. Teaching in Higher Education, 29(1), 75–92. https://doi.org/10.1080/13562517.2021.1965567
Halpern, D. F. (1998). Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American Psychologist, 53(4), 449–455. https://doi.org/10.1037/0003-066X.53.4.449
Halpern, D. F. (1999). Teaching for critical thinking: Helping college students develop the skills and dispositions of a critical thinker. New Directions for Teaching and Learning, 80, 69–74. https://doi.org/10.1002/tl.8005
Holmes, N. G., Wieman, C. E., & Bonn, D. A. (2015). Teaching critical thinking. Proceedings of the National Academy of Sciences, USA, 112(36), 11199–11204. https://doi.org/10.1073/pnas.1505329112
Niu, L., Behar-horenstein, L. S., Niu, L., Behar-horenstein, L. S., & Garvan, C. W. (2014). Do instructional interventions influence college students’ critical thinking skills ? A meta-analysis. Educational Research Review, 9, 114–128. https://doi.org/10.1016/j.edurev.2012.12.002
Orhan, A. (2024). Online or in-class problem based learning: Which one is more effective in enhancing learning outcomes and critical thinking in higher education EFL classroom? Journal of Computer Assisted Learning, (June), 2351–2368. https://doi.org/10.1111/jcal.13033
Tiruneh, D. T., Verburgh, A., & Elen, J. (2014). Effectiveness of critical thinking instruction in higher education: A systematic review of intervention studies. Higher Education Studies, 4(1). https://doi.org/10.5539/hes.v4n1p1
Willingham, D. T. (2008). Critical thinking: Why is it so hard to teach? Arts Education Policy Review, 109(4), 21–32. https://doi.org/10.3200/AEPR.109.4.21-32
File under:
Critical thinking; discussion activities; active learning