“Compared with students in traditional lectures,
students in active classes perceived that they learned less, while in reality
they learned more.” — Louis Deslauriers, Logan S. McCarty, Kelly Miller,
Kristina Callaghan, and Greg Kestina, Measuring
actual learning versus feeling of learning in response to being actively
engaged in the classroom.

That’s the conclusion of a brand new and
already-attention-getting article just published in the Proceedings of the
National Academy of Sciences, and a close reading suggests that the attention
is well deserved.

It reports a scrupulously conducted study of two
different approaches to teaching physics concepts: a traditional, teacher-focused
lecture, and a student-focused active learning setup involving group work on
problems supported by instructor feedback and guidance.

To most of us with an interest in such things, the idea
that active learning = good is not exactly new. So what is world-rocking about
what Deslauriers and colleagues did?

One is their use of a beyond-gold-standard true
experimental design. Individual students (not whole sections, which is usually
as good as it gets in this kind of research) were randomly assigned to control
and experimental groups corresponding to different teaching approaches. One
class was conducted in each method, then the groups were flipped around so that
each student experienced each method. Same thing for the instructors — each
used each method once, cutting out the issue of whether students were
responding to the instructional method or to the instructors themselves.

The procedures used in the classes were set up carefully
so that neither method — lecture or active learning — had an unfair advantage.
Content was the same, materials and objectives were the same, problem sets were
the same. The only difference was that in the lecture condition, the teacher,
well, lectured, demonstrating how to solve problems while students followed
along on their own worksheets. In the active learning condition, students
tackled the problems in small groups. The instructors circulated as one
typically does during this kind of activity, then at the end, they showed the
class the correct process for the solution.

Researchers gathered data from students on two separate
dimensions, first, mastery of the content (the TOL, or test of learning), and
second, subjective feelings about the lesson (the FOL, or feeling of learning).
The TOL, notably, was prepared independently of the instructors, as a safeguard
against their teaching too closely to the test.

Here’s what the researchers found.

Students in the active learning condition scored higher
on the TOL, and judging from the data distributions laid out in the article, it
wasn’t even close. And, you guessed it — they scored lower on
the FOL.

Those disparities in FOL scores are stunning, especially
on two questions in particular: My instructor was effective at teaching,
and I wish all of my physics classes were taught this way.

If, as an instructor who works hard to create great
pedagogy, that doesn’t make your heart sink, well — you’re a lot tougher than I
am.

As the teaching expert and inclusivity advocate Viji Sathy pointed out in this tweet, this pattern of
findings means that all those student opinion surveys, evaluations and the like
could well be disadvantaging exactly those individuals who are doing more to
promote student learning.

That fact in and of itself would justify circulating
this study like crazy. And while I’ve cautioned against using one study as a blunt instrument to push one’s own pedagogical agenda, this time, a
little bludgeoning might be in order.

Why? There are the pristine methods, analyses, and
interpretations of course, which all read like a master class in how to conduct
scholarship of teaching and learning. There’s also the way that the findings
neatly funnel into a conclusion that many of us have long suspected, that
students — due to being novices in a field, discomfort with the messiness and
effort of active learning exercises, or lack of insight about how learning
works — are poor judges of effective teaching.

There is also the way that the findings fit with themes
from so many other lines of research, amplifying and deepening those without
repeating them. As the authors note, their results follow on a long line of
well-established cognitive principles including desirable difficulty and the development of expertise.

They also remind me of some of my own research from long
ago, when my colleague Laurie Dickson and I were looking at the impacts of assigning practice quizzes in Introduction to
Psychology.

On an end of semester survey, many students said they
thought the quizzes helped raise their exam performance, an impression borne out
by our comparisons across sections that were randomly assigned to do or not do
these exercises as part of their graded work. But we were always puzzled by the
fact that not all of those same students said they would voluntarily do this
sort of quizzing in the future.

It’s strange to think that a student might see value in
a learning exercise but still be unwilling to do it on their own. But it makes
some sense in light of the idea that active learning can be uncomfortable, and
effortful, and seemingly not as worthwhile as sitting and watching an expert
present the material.

What I also like about this article is that it suggests
solutions, not as an afterthought but with some very clear recommendations for
how to improve teaching and learning in higher education.

One is to look at student impressions very cautiously,
or not at all, when judging teaching. This recommendation comes at a time when
reliance on student evaluations is a problem that is currently reaching a
system-blinking-red point. Disadvantaging those who use active learning is just
one of many reasons why.

Note that I said reliance, and not over-reliance.
The longer I work in academia, the more convinced I am that attempting to bring
in “balance” or “context” as a way to appropriately de-emphasize student
evaluations just doesn’t work. Those numbers and comments are a bell you can’t
unring when it comes to forming impressions of others’ work, and hard as it is,
as a profession we need to commit to developing new metrics that reflect
teaching quality, particularly those that privilege evidence-based course
design features.

The article also advocates for something that is dear to
my heart: raising student awareness about why we teach the way we do, and how they can
take the same principles and run with them as they take charge of their own
learning.

It’s easy to overlook, but the authors include an
intriguing final component documenting the positive effects of a short
presentation on the way in which active learning works and why FOL is a bad
indicator of actual learning. Struggle can be good, and teaching students this
fact can be transformative, an idea that also reminds me of the work on
belongingness and normalizing struggle in the early college career.

Of course there are caveats to how we apply and extend
this work to our own teaching practices. I hope that the attention paid to it
doesn’t trigger a wave of teacher-knows-best, adversarial pedagogy the likes of
which we’ve seen aired in the classroom laptop wars over the last few years.

If that were to happen, it could set back the current
trend towards empathic,
truly student-focused pedagogy, and that would be
a tragedy. And a needless one at that, because as I see it, empathic teaching
is perfectly compatible with the idea that our students aren’t always the best
judges of their own learning processes — not because they are students, but
because they are novices, and also because they are human beings.

I’ll be emailing this article to my colleagues and I
suspect many of you will be as well. We’ll also be seeing — I hope — some
spirited discussion about what the findings do and don’t mean for college
teaching. This time, let’s see to it that action also follows.