Over the last couple of years I’ve been studying and some of the subjects I have done have presented me with an assessment rubric. This is a kind of checklist which sums up everything that is good and bad about checklists to me. The first is that a checklist only really makes sense for highly repeatable behaviours. There is a really good reason why they work so well when landing planes and performing surgery. Things can go catastrophically wrong in either of these, but mostly they go wrong in somewhat predictable ways. No, that’s not quite right. It is like when I represented people in the trade union – people are infinitely inventive in getting themselves into trouble, but getting them out of trouble again generally follows a very predictable path. Jet engines can stop in all manner of ways, but really, whether they have stopped because of ice or geese, in the end you are still going to want to land the damn plane.
The big lesson here is that checklists can very easily become self-defeating. Checklists aren’t recipes, they are not meant to tell you all of the steps necessary to do any particular thing – if they did that they would be so long as to be completely useless. Instead, the point of an effective checklist is to ensure that you have done all of the things that really must be done – that, if they haven’t been done, then all hell will break loose.
There is an interesting discussion here about fires in operating theatres, for example. They decided not to include a checklist for this in a group of checklists they were working on, not because fires don’t happen or that they are not terrifying, but because they occur too infrequently and a good checklist looks for low-hanging fruit.
Checklists should also start from the assumption that the person using them is an expert. This is why they are not really recipes. The point is not to teach a surgeon how to take out your appendix, it is to make sure they go through the kinds of checks that have been shown surgeons tend to miss ‘some of the time’ and that cause problems when missed ‘most of the time’.
In a sense, and this is a point that is made really well in the book, we all tend to think that checklists are important for other people to follow. We know that we don’t make mistakes – obviously, that goes without saying – but other people are infinitely stupid and so it is a really good idea that they are forced to make doubly sure. If the price we must pay to make sure stupid people don’t make mistakes is for us to have to follow a checklist too, well, so be it. The fact we are deluded about our own abilities should be our primary assumption, but never seems to be…
Checklists need to be comprehensive in the sense that they should be evidence based – what normally goes wrong – and culturally implemented – how do you trip people to make them check before they do what needs to be done? You see, we humans are really terrible at following procedures. We get bored. A computer, on the other hand, is really good at following procedures. In fact, that is pretty much all a computer ever does. This is the reason why we tend to forget the milk we were supposed to pick up on the way home from work – even though we might have reminded ourselves just as we were leaving. We are victims of habit to the extent that something new tends to be forgotten. Unfortunately, we aren’t even all that good at habit and so even things we do over and over again, we tend to mess up now and again as our minds wonder.
The advice here, then, is that checklists can make a huge difference, but that checklists, like poems, are very hard to write. And that is true for much the same reason. In a good checklist you have to leave out what isn’t necessary, but make sure you put in what is. There is an inverse relationship operating here – the easier it sounds to write a checklist that ‘just has important things’ the harder it actually is to do. I believe the best people to write checklists are those who admits to making mistakes – but then, I think people who admit to making mistakes are the best people anyway. They tend to be grossly mistreated (‘she admits to making mistakes, what a loser!’), are infinitely rare – but invariably worth their weight in gold.
If writing a checklist is a remarkably difficult task, following one can be just as hard. As I said, I’ve had to hand in assignments that have required following assessment rubrics. In theory these should be easy – make sure you have answered all of the questions and you win! But what is really called for is a change in culture. You need to remember to go back and check through the checklist before handing in the essay, and really check it, not just glance over it. This is a very hard thing to do once you have convinced yourself you have done ‘everything’ that needs to be done already. On one assignment this year I did not address an entire assessment criteria – it turned out I was not the only one to leave this particular criteria out. It was not in the least surprising to me that this was the criteria that asked me to detail how I would change what had occurred. This was the question that was asking me to be creative – and being creative is the hardest thing and therefore the most likely thing to be left to the end (based on those most the dangerous words, ‘I’ll just come back to that later’). But without having some procedural ‘cultural’ trigger to actually go through the checklist one last time before handing in the assignment, many of us completely missed this question and lost what ought to have been fairly easy marks.
The problem is that checklist are:
Hard to write
Hard to follow
Easy to convince ourselves we’ve followed when we haven’t
Only good for more or less predictable outcomes and processes.
Given these limitations this book does make it clear that they do make substantial improvements to performance if they are well constructed, easy to use and effectively used.
My main problem, though, is that checklists come at the problem from the wrong end of the telescope. Checklists should be used as a ‘when everything else fails’ solution. The first solution we should be looking for is not, how do we restrict the options available by writing more or less optional procedures, we should be redesigning out dangers. Procedures lead to cultures of blame, and as I’ve said, humans aren’t terribly good at following procedures. So a checklist is a kind of time bomb waiting to go off in the face of the overworked and overly stressed person who misses one of its line-items. If you really want to improve things, don’t start by writing lists, start by redesigning hazards out of situations. For example, if you are using a guillotine you can have a checklist that says, ‘make sure your hands are not under the blade before operating’ or you can construct the guillotine so that it is impossible for either hand to be under the blade when it cuts. It is infinitely better to make it impossible for the worker to lose their hands, than it is to smugly tell them they don’t have hands because they didn’t properly follow a checklist.
But, with that said, I think this is a very interesting book and also know that if I was the sort of person who made more lists I would probably be happier and more successful. Just because this book is a clear victory for the anally retentive, doesn’t make it any less worthwhile.