Better answers, broader thinking: What students gain from ChatGPT and critical-thinking training

What happens when students use ChatGPT on a real-world assignment? Do quality improvements come at the expense of originality?

A new experiment from researchers at Bocconi University, in collaboration with OpenAI Economic Research, found distinct and complementary effects from ChatGPT access and critical-thinking training. Access to ChatGPT improved the quality and coherence of students’ work, while an exercise in causal reasoning—a form of critical thinking—led students to generate more unique ideas. Students who received both ChatGPT access and the training showed both effects.

The results also highlight the importance of a holistic approach in assessing student progress. As AI makes it easier for students to produce polished answers, assignments may need to adapt to measure other desired qualities such as originality. 

AI helped students close an expertise gap

During the experiment, more than 1,000 first-year undergraduate students at Bocconi University worked on a real-world business case developing marketing recommendations for the university’s merchandise store.

Students were randomly assigned by class period into one of four groups that received: access to ChatGPT (GPT‑4o), training in causal reasoning, both, or neither. Causal reasoning is a specific form of critical thinking related to linking cause and effect and explaining why a given solution may or may not work. The training students received was unrelated to AI, instead teaching students causal reasoning concepts through an exercise that involved a game, examples, questions, and feedback. 

Student submissions were evaluated by trained human graders using a five-point rubric. Separately, researchers used automated text analysis to measure each submission’s number and variety of ideas, signs of causal reasoning, and similarity to submissions from three experts. 

Critical-thinking training helped in a different way

AI access can complement thinking skills

Together, they are complementary.