What do you think?
Rate this book
Evaluation in small development non-profits: Deadends, victories, and alternative routes
Research on evaluation shows that low-use and non-use of evaluation is common, yet evaluation is hailed as beneficial and worthwhile. The worth of evaluation is tied to its utilisation, presenting a paradox if evaluation is both revered and underutilised. This book investigates this paradox in the under-researched context of small development non-profit organisations, which have specific resource constraints and ‘bottom up’ community development values that complicate their ability to do and use evaluation in line with top down directives. The book examines the utility, meaningfulness, and purpose of evaluation from small non-profit perspectives, and explores whether evaluation has value for these organisations. For development practice, it presents evaluative alternatives that reconceptualise evaluation as part of the active process of development rather than as an interval-based add-on. For evaluation theory, it highlights a historical preoccupation with improving evaluation without assessing its inherent worth, and considers alternative ways to enhance the value of evaluation for small non-profits.
215 pages, Hardcover
Published October 31, 2020
1 person is currently reading
28 people want to read
About the author
Leanne M. Kelly
7 books12 followersLeanne Kelly has spent nearly 20 years working in community development and social service non-profit organisations across five continents. She has worked for non-profit organisations in a broad range
of disciplines (from housing and emergency services to child protection
and peacebuilding) with the majority of her roles focused on evaluation. Leanne has also worked as an external evaluator for non-profit
organisations, most recently in Myanmar. She has a PhD from Deakin
University on evaluation in small international and community development non-profits and has published 40 scholarly papers and two
books: Evaluation in Small Development Non-Profits and Internal
Evaluation in Non-Profit Organisations (co-authored with Dr Alison
Rogers) She is currently a Postdoctoral Research Fellow at Deakin
University and is the National Evaluation Advisor at the Australian
Red Cross.
of disciplines (from housing and emergency services to child protection
and peacebuilding) with the majority of her roles focused on evaluation. Leanne has also worked as an external evaluator for non-profit
organisations, most recently in Myanmar. She has a PhD from Deakin
University on evaluation in small international and community development non-profits and has published 40 scholarly papers and two
books: Evaluation in Small Development Non-Profits and Internal
Evaluation in Non-Profit Organisations (co-authored with Dr Alison
Rogers) She is currently a Postdoctoral Research Fellow at Deakin
University and is the National Evaluation Advisor at the Australian
Red Cross.
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 4 of 4 reviews
November 17, 2020
Leanne used to sit behind me in the office space at the university where we did our PhDs together, well, not quite together, but basically together – if that makes sense. She recommended I read Planet of Slums, I often remember the books people get me to read. I’m not totally sure I fully grasped what it was she was doing her PhD on at the time, although we must have spoken about it.
I mention all of this for two reasons, one as a kind of disclaimer, and the other is because I used to work as an organiser in a trade union that represented people working in the kinds of small development non-profit organisations she discusses in this book. And evaluation processes are things I am very opinionated about – tragically so. I can, and have, bored people nearly to death on the topic. So, when I noticed that Leanne had published a book on this topic, there was virtually no chance at all that I wasn’t going to read it.
I found this a seriously interesting book – but I do understand that you might not find it quite as interesting as I did. So, let me sell it to you.
Small non-profit organisations do some remarkable work. As is said here, they are not always small because they haven’t worked out how to become big – often they are small because they prefer to remain nimble, and being large makes being nimble difficult. This isn’t to say that large non-profits aren’t good – it is just saying that different organisations are able to achieve different things and respond in different ways. Which then means it might not be a great idea to create a situation where all of the small organisations get eaten up by the large ones or where they are all evaluated in the same way.
When I was reading this, I kept thinking of free-market types and how they generally explain the benefits of capitalism. That is, that small, local companies are better able to understand and meet customer needs.
Except, the world is full of paradoxes. And so rather than allowing small organisations to operate in ways which they decide are best at meeting their clients’ needs, the funding organisations are generally one or two steps removed from the ‘coal face’ of these organisations, but they are in a position to be able to demand ‘value for money’ from their donations despite not necessarily knowing what exactly the non-profits do. Don’t get me wrong – I am certainly not suggesting that there should be no accountability for money donated to small organisations, but the donations shouldn’t destroy the organisation either.
The issue is that two worlds are likely to clash here. In the blue corner is the neoliberal funding organisations. To them, accountability is premised on measurable outcomes. They expect these measurable outcomes to lead to the efficient allocation of resources so as to best meet the objectives of the funding body in being able to say that the client base of the non-profit has been impacted by the funding body’s donation. All of which sounds terribly obvious, that is, ‘evaluation’ should be a quick matter of listing your measurable outcomes and then actually measuring them to ensure they have been met. Simple.
The problem is that this kind of simple accountability doesn’t even work in businesses that produce widgets. The people who say ‘if you can’t measure it, it doesn’t exist’ miss the point that what you measure too often directs (and sometimes misdirects) what you do. If a large part of your organisation’s reason for existing is in you needing to make deep human contact with people, having a ‘measure’ that somehow tries to convert this to a figure like ‘2.757’ probably isn’t going to be much use.
This has been a large part of the lesson of the neoliberal takeover of society over the last four decades. It demands accountability and that involves measures. But how you measure impacts what you do. But if the guiding theory of the non-profit is community development, after a short while certain measures will make that goal impossible.
Leanne begins this book by mentioning an evaluation report she wrote once for such an organisation and as CEO thanked her he put her report into a folder never to be looked at again. This is a key idea here – that if you are going to force a small organisations to do evaluations it really ought to be so as some purpose (other than to merely justify the money you’ve given them) comes from that evaluation. Arse covering is all well and good, but too often ‘evaluations’ can cost nearly as much as the donation itself – which makes for a nice little cottage industry, but it is unlikely to help anyone the organisation was set up to help.
So, if you are going to measure something it really ought to be something that will have the potential to tell you something you don’t already know. And if you are told something you don’t already know, then you should be prepared to act on it. But often this isn’t the impact of evaluations – rather, the reports tell the organisations either exactly what they already know, or the person writing the report gets things so arse-about-face that the report is literally useless.
Then there is the problem that data can be made to sound incredibly positive or incredibly negative depending on how it is presented. Evaluation reports can encourage organisations to ensure they maintain strict control over all of the inputs that are going to be necessary for the report that they will be judged upon. But if the point of a community organisation is to develop a community, so that the community can work to address its own needs – you know, community development – then evaluations that encourage the centralisation of control in the hands of those outside the community are exactly the opposite of what you are looking for.
The point of this book is to show that these problems can be addressed and accounted for – and that evaluations don’t need to be a waste of time for the organisation. In fact, if done by someone who understands the imperatives of the organisation and its client base, evaluation can work to transform both what and how the organisation works, something that too often only comes about once a crisis hits.
Leanne has interviewed many small non-profit organisations to produce this book. She explains how some of these organisations have made evaluation work for them and their clients. The book addresses these issues with considerable care and attention. This really was a very good book, I enjoyed it a lot.
I mention all of this for two reasons, one as a kind of disclaimer, and the other is because I used to work as an organiser in a trade union that represented people working in the kinds of small development non-profit organisations she discusses in this book. And evaluation processes are things I am very opinionated about – tragically so. I can, and have, bored people nearly to death on the topic. So, when I noticed that Leanne had published a book on this topic, there was virtually no chance at all that I wasn’t going to read it.
I found this a seriously interesting book – but I do understand that you might not find it quite as interesting as I did. So, let me sell it to you.
Small non-profit organisations do some remarkable work. As is said here, they are not always small because they haven’t worked out how to become big – often they are small because they prefer to remain nimble, and being large makes being nimble difficult. This isn’t to say that large non-profits aren’t good – it is just saying that different organisations are able to achieve different things and respond in different ways. Which then means it might not be a great idea to create a situation where all of the small organisations get eaten up by the large ones or where they are all evaluated in the same way.
When I was reading this, I kept thinking of free-market types and how they generally explain the benefits of capitalism. That is, that small, local companies are better able to understand and meet customer needs.
Except, the world is full of paradoxes. And so rather than allowing small organisations to operate in ways which they decide are best at meeting their clients’ needs, the funding organisations are generally one or two steps removed from the ‘coal face’ of these organisations, but they are in a position to be able to demand ‘value for money’ from their donations despite not necessarily knowing what exactly the non-profits do. Don’t get me wrong – I am certainly not suggesting that there should be no accountability for money donated to small organisations, but the donations shouldn’t destroy the organisation either.
The issue is that two worlds are likely to clash here. In the blue corner is the neoliberal funding organisations. To them, accountability is premised on measurable outcomes. They expect these measurable outcomes to lead to the efficient allocation of resources so as to best meet the objectives of the funding body in being able to say that the client base of the non-profit has been impacted by the funding body’s donation. All of which sounds terribly obvious, that is, ‘evaluation’ should be a quick matter of listing your measurable outcomes and then actually measuring them to ensure they have been met. Simple.
The problem is that this kind of simple accountability doesn’t even work in businesses that produce widgets. The people who say ‘if you can’t measure it, it doesn’t exist’ miss the point that what you measure too often directs (and sometimes misdirects) what you do. If a large part of your organisation’s reason for existing is in you needing to make deep human contact with people, having a ‘measure’ that somehow tries to convert this to a figure like ‘2.757’ probably isn’t going to be much use.
This has been a large part of the lesson of the neoliberal takeover of society over the last four decades. It demands accountability and that involves measures. But how you measure impacts what you do. But if the guiding theory of the non-profit is community development, after a short while certain measures will make that goal impossible.
Leanne begins this book by mentioning an evaluation report she wrote once for such an organisation and as CEO thanked her he put her report into a folder never to be looked at again. This is a key idea here – that if you are going to force a small organisations to do evaluations it really ought to be so as some purpose (other than to merely justify the money you’ve given them) comes from that evaluation. Arse covering is all well and good, but too often ‘evaluations’ can cost nearly as much as the donation itself – which makes for a nice little cottage industry, but it is unlikely to help anyone the organisation was set up to help.
So, if you are going to measure something it really ought to be something that will have the potential to tell you something you don’t already know. And if you are told something you don’t already know, then you should be prepared to act on it. But often this isn’t the impact of evaluations – rather, the reports tell the organisations either exactly what they already know, or the person writing the report gets things so arse-about-face that the report is literally useless.
Then there is the problem that data can be made to sound incredibly positive or incredibly negative depending on how it is presented. Evaluation reports can encourage organisations to ensure they maintain strict control over all of the inputs that are going to be necessary for the report that they will be judged upon. But if the point of a community organisation is to develop a community, so that the community can work to address its own needs – you know, community development – then evaluations that encourage the centralisation of control in the hands of those outside the community are exactly the opposite of what you are looking for.
The point of this book is to show that these problems can be addressed and accounted for – and that evaluations don’t need to be a waste of time for the organisation. In fact, if done by someone who understands the imperatives of the organisation and its client base, evaluation can work to transform both what and how the organisation works, something that too often only comes about once a crisis hits.
Leanne has interviewed many small non-profit organisations to produce this book. She explains how some of these organisations have made evaluation work for them and their clients. The book addresses these issues with considerable care and attention. This really was a very good book, I enjoyed it a lot.
November 4, 2020
Very good and informative. Helped me understand the value of evaluation and how we could change it to be more useful by looking at the everyday ways people and organisations evaluate their programs and projects.
March 27, 2021
Dr Leanne Kelly presents her information in a clear and concise manner. This book is full of interesting and pertinent information and it’s still funny, sad and engaging.
If you are looking for a book that outlines the uses of the various types of evaluation and their applicability in small NGOs then look no further. This book is for you.
If you are looking for a book that outlines the uses of the various types of evaluation and their applicability in small NGOs then look no further. This book is for you.
April 20, 2022
Might as well give myself a good review. My favourite difference between this book and other evaluation books is that it is based on the lived experience of people working in small NGOs and draws heavily from interviews with 50 such people and observation of their everyday evaluative actions. It's really a book of their views and actions, that I just wrote into a narrative. So thank you again to all of them for sharing their wisdom.
Displaying 1 - 4 of 4 reviews