David Franklin > Quotes

“Dan asks the class whether any of them has ever checked the weather on their phone to see if it’s likely to rain while they’re outside. Everyone has. “What do you do if you see there’s a 30% chance of rain?” “Take an umbrella”, several students all say at once.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“I remember he said something like ‘this should be completely obvious to you’, and to me that was crushing… we had a conversation after the mid-term exam, in which the class had averaged 20 out of 90. It was 25 years ago, and I still remember what he said: ‘Frustration is necessary for learning. This idea that you can enjoy learning is a very American idea’.” Dan pauses, the memory of that conversation etched on his face. “I felt so offended by that claim. My professor felt that to learn, you had to push yourself. One of my dreams was to go back to Venezuela[7] and start a university, and I vowed that I would write in the walls of the university that frustration was not necessary for learning.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“The students are asked to rate their level of surprise on a five-point scale from ‘not surprised at all’ to ‘beyond shock’. Many describe feelings of total shock and numbness: a large majority indicate some level of surprise. He asks them: “Why were you surprised?”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Dan’s use of two-stage exams kills two birds with one stone. Firstly, he maximises learning by ensuring that the exam itself is a learning experience. Second, in doing so, he makes clear that the grade is less important than the learning. Two-stage exams have not yet ‘taken off’ around the world, and grades remain the key outcome of most exams for most students. Dan, though, has taken advantage of his position in a graduate university environment to push the idea forward.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“The failure here is an example of a common flaw in our approach to adaptive challenges: we try to solve them with technical solutions. Terrified of the unknown, we seek to replace problems we have no idea how to address with those we know how to solve. This is a form of ‘work avoidance’, a term coined by Heifetz[x]. If you have ever procrastinated by spending your whole afternoon making a brightly-coloured work plan, and then congratulated yourself on being productive, then you will be familiar with the concept. By focusing on what they know they can measure – the extent to which a lesson went as planned – some professors may avoid the harder work of trying to gauge the learning of their students.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“The inclusive atmosphere of the class was the key driver of allowing the class to be eager to learn. The focus really shifted away from grades and points, and towards understanding concepts and learning.” The key link is between inclusivity and the shift of focus away from test scores. When students are given the responsibility to support each other, their own score falls down the list of priorities.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“On the whiteboard in Dan’s office, he has written in bold black marker the phrase ‘TEACHING ≠ LEARNING’. He believes that most lecturers pay too much attention to what they are doing in a class. Planning is important, and professors are right to spend time in advance thinking about how the class will be structured. One downside, though, is that professors will often evaluate themselves against what they said they would do in the plan. If they covered the material without tripping up, and gave the right explanations at the right time, they will be happy about how the class went.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“There is also something powerful in the concept of an ‘airport idea’. On the surface, it links with Dan’s plea for what students should do if they meet him in an airport in five years’ time. But there are hidden suggestions too: the airport carries an association of being high-level, and fits in with the course’s aim of empowering the students with the tools they need to effect change. Once the course is over, they will disperse to airports across the world and put these ideas into practice in their home countries.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“It turns out that you only need 23 people for it to be more likely to find a shared birthday than not.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“The brain has conflicting goals: to provide you with information, and to reduce your anxiety about the worrying outcome. If you wanted Clinton to win in 2016, your brain achieved both goals by accepting the 30% figure but telling you it would not happen.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Availability bias prevents us from setting our prior beliefs appropriately, whereas confirmation bias stops us from updating them for new information.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“The great economist John Maynard Keynes is supposed to have said “when the facts change, I change my mind. What do you do, sir?”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Exams also give us a concrete example of Dan’s attempts to promote understanding over memorization. He encourages students to bring with them a ‘cheat sheet’, which can include anything they want. He does not want students being tested on whether they can remember the formula for the standard error of a distribution, and does not believe that it will be helpful for them to do so. Allowing cheat sheets means that students can bring more than just formulas with them: if they were struggling with a topic, they could bring all sorts of pre-cooked explanations to help them with the exam questions. In turn, this acts as an extra incentive to Dan to set questions that rely on a deeper understanding.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Dan’s goal, taken from his mentor Richard Zeckhauser, is to get students to think probabilistically about the world. This means engaging with the challenge of understanding and accepting what 29% means in the context of a Trump victory, and fighting the brain’s natural inclination to see things in binary terms.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“With 80 people, the size of this classroom, the probability that there will be no shared birthdays is so tiny as to be virtually impossible.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“So Trump was 29% likely to win. What does 29% mean?” he asks. This is an odd question for many: they’re doing a Master’s degree at Harvard, and they’re being asked what a percentage means. The question is aiming at the gut reaction of the brain to the number. Inevitably, students think it might be a trick question. Dan waits a while, and when no-one raises their hands, he breaks away from the example to tell a story: “You know, when I first started teaching, I was terrified of silence. I thought, ‘oh my god, I’ve got to do something, they’re not saying anything’.” The class laughs: he has eased the tension created by the silence. “The more I taught, the more I realised that silences are important in a class – they give time for people to think. These days, I’m not afraid of silence at all.” After a few seconds, a woman puts up her hand. “Well, obviously, I knew that it meant there was some chance that he would win. But it was still a shock that the 29% happened.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“I’d like anyone who was born on March 22nd to stand up.” Two students stand up. They link eyes from across the room, and smile. The room laughs. “Now, anyone born on June 14th.” Another two stand up. The room laughs again. “Now, anyone born on August 2nd.” Two more: the class is laughing at each pair, and re-examining their original takes on how low the probability was likely to be. They are learning through experience about human fallibility with estimating probabilities, and at the same time learning something about their classmates. “How about December 21st?” This time, three students stand up. This gets a big laugh for two reasons: firstly, it is unexpected, and secondly, it helps to ram home the idea that in a class that size (there are about 3,000 pairs in a class of 80), there are likely to be “coincidences” everywhere.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“In Dan’s case, the interest in policymaking is so central to the students’ presence at Harvard that it would be foolish to run a statistics course that did not acknowledge it. He goes as far as to build this into the purpose of the course: as we heard earlier, his purpose is “not just to maximise learning about statistics, but also to maximise learning of the skills that will be useful to have out there in the world.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Maximising enjoyment All students want to enjoy their classes. This simple truth means that teachers who can make learning fun will, other things equal, be more successful. If the only way students can enjoy themselves is to ignore the class and play on their phones, then student enjoyment is a headwind that slows the boat’s progress. By making learning fun, teachers can adjust the sails to take advantage of the wind and speed the boat up. The progress it makes may not be directly towards our destination, but we can use the sails nonetheless to pick up speed and get closer to it than we would otherwise have done.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Recognising competing purposes Although few would have much trouble identifying ‘maximising learning’ as the purpose of the class, there may be several competing purposes in play, all of which are invisible. For students, maximising enjoyment, minimising effort, maximising test scores, maximising learning about something else, and making friends are all likely to be found across the classroom. For the teacher, we can add to that list maximising evaluation scores, which may be closer linked to student satisfaction than to student learning.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“For these ‘future leaders’ in the classroom, it is more important to understand the psychology of certainty. Often, we believe what we want to believe. Dan starts with an example of the 2016 Presidential Election in the United States. He asks the question: “Relative to what you expected to happen, how surprised were you when you learned that Trump had won the election?”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Acknowledging that students have competing purposes is the first step to managing them.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Now – those who got it right. On the count of three, I’d like you to sit down if you’d seen the question before. 1, 2, 3!”  Everyone who was standing sits down. This gets a huge laugh. The students who got the answer wrong suddenly feel much better about their mistake: in a Harvard class, not a single person got it right unless they had seen it before.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“After pausing, the student continues: “There’s also an element of believing something because you want it to happen.” This betrays a second bias in which we believe what we want to be true. This is a form of confirmation bias, which exacerbates the distorting effect of availability bias. We seek out information that confirms our existing beliefs or desires, and ignore information that refutes them.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“An American student, well-versed in U.S. politics, puts his hand up. He says he had been following Nate Silver’s ‘Five Thirty-Eight’ website, which had said that Trump only had about a 30% chance of winning the election.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“I guess… it was just a shocking event. We’re not used to someone like him being President.” Here, the student reveals an availability bias in which we expect future outcomes to look like what has gone before.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“With these issues in mind, Dan has a unique way of tying exams into the learning process. After attending a workshop run by the Nobel Prize winner Carl Wieman, he was converted to the idea of two-stage exams. The first stage of the exam is the same as any other. The second stage brings the students together in groups of four, and gives them all a subset of the questions they just answered in the first stage. The students are encouraged to discuss their answers in their groups, and they all submit a second version. Groups who manage to improve upon the students’ individual scores get a small boost in the final outcome.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“The Google team at Project Aristotle concluded that there were five main traits of a successful team. Psychological safety was the most important, and their research suggested that it underpinned the other four[xxvi]. The five traits were as follows: (1)   Psychological safety: team members feel safe to take risks and be vulnerable in front of each other. (2)   Dependability: team members get things done on time, and meet Google’s high bar for excellence. (3)   Structure and clarity: team members have clear roles, plans, and goals. (4)   Meaning: work is personally important to team members. (5)   Impact: team members think their work matters and creates change.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“Einstein is supposed to have remarked that “education is what remains when one has forgotten everything they learned at school”.”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course

“What is the probability that at least two people in a group of 20 people will have the same birthday?”
― David Franklin, Invisible Learning: The magic behind Dan Levy's legendary Harvard statistics course