What do you think?
Rate this book
Safeware: System Safety and Computers
We are building systems -and using computers to control them- that have the potential for large-scale destruction of life and environment. More than ever, software engineers and system developers, as well as their managers, must understand the issues and develop the skills needed to anticipate, prevent and contain accidents.
Addressing this need in her long-awaited book, Nancy Leveson examines what is currently known about building safe systems and kooks at past incidents and accidents to see what practical lessons can be applied to computer-controlled systems.
Safeware demonstrates the importance of integrating software safety efforts with system safety engineering, describes models of accidents and human errors that underlie particular approaches to safety problems and presents the elements of a safeware program, including management, hazard analysis, requirements analysis, design for safety, design of the human-machine interface, and verification.
Addressing this need in her long-awaited book, Nancy Leveson examines what is currently known about building safe systems and kooks at past incidents and accidents to see what practical lessons can be applied to computer-controlled systems.
Safeware demonstrates the importance of integrating software safety efforts with system safety engineering, describes models of accidents and human errors that underlie particular approaches to safety problems and presents the elements of a safeware program, including management, hazard analysis, requirements analysis, design for safety, design of the human-machine interface, and verification.
704 pages, Hardcover
First published April 17, 1995
5 people are currently reading
118 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 3 of 3 reviews
February 11, 2017
I read this book on system safety in 2017, twenty two years after it was published. In 2012, Nancy Levenson published Engineering Safer Systems which should provide a timely update on this extremely important topic.
Levenson considers safety an emergent systems property. In particular, she argues very convincingly, that one cannot talk about software safety divorced from the context it is used in -software is always part of a sociotechnical system and it is in the context of such systems that safety must be considered.
As a software engineer, I found this book engrossing and enlightening and it made me realize why such well-respected figures such as David Parnas have always insisted software engineering education needs to include thorough grounding in more traditional engineering disciplines -this insistence gains particular importance in a world bent on developing self-driving cars, drone fleets, the internet of things and increasingly tighter software-controlled loops binding all sort of devices operating in and on the physical world.
The first part of this book covers topics on the nature of risk. There are chapters covering such key questions as changing attitudes towards risk, the nature of risks in an industrial society, how safe is safe enough, the role of computers and humans in accidents, the incidence of ineffective organizational structures and safety cultures and just what it means to identify root causes of accidents.
After a brief two chapter introduction to system safety from a historical point of view which notes the main contributions made by systems theory, systems engineering and systems analysis, Levenson delves into key definitions, distinguishing clearly between failure and error, accident and incident, hazard and risk, safety and reliability, safety and security. She also overviews accident and error models.
The first three parts lead into the crucial fourth part, Elements of a safeware program, whose importance is underlined by the fact that it constitutes almost half the book. Its chapters include:
Four appendices on historically important safety-related accidents are also included. Appendix A deals with the Therac-25 computer-controlled radiation therapy device which massively overdosed six people between 1985 and 1987. Appendix B treats the approach to safety developed by civil aviation and several specific aerospace cases: Apollo 13 (1970), the DC-10 cargo door blow out which caused 346 casualties in 1974 and the space shuttle Challenger tragedy (1986). Appendix C covers the Hoffman-LaRoche’s subsidiary Givadau’s Seveso (Italy, 1976), Nypro’s Flixborough (UK, 1974) and Union Carbide’s Bhopal (India, 1984) chemical plant disasters. Appendix D covers Windscale, Three Mile Island and Chernobyl nuclear power station accidents and disasters. Appendices B, C and D start with an overview on safety in a specific industry. Each case in the appendix provides sections on background, the nature and state of safety features present, events leading up to the accident or disaster and causal factors contributing to it.
Nancy Levenson is not subtle, she hammers her points home. This leads to some repetition and the book’s exhaustiveness occasionally may also become somewhat exhausting -laying the book aside for a couple of days rest is probably all it takes to keep going. Sometimes the writing seems a little too cobbled together, the list of sixty guidelines for safe human-machine interface design is a case in point -what is the difference between guideline 5 (Distinguish between providing help and taking over) and 19 (Design to aid the operator, not take over).
Since my Ph. D. thesis was on writing specifications, I particularly enjoyed the chapter on software hazard and requirements analysis.
Admittedly without having read much in the field of safety, I feel the book has aged well. Its main lessons are still very, very pertinent. If you plan to use it, at least take a look at Nancy Levenson’s more recent book, which is, at least at the time of writing this review, is still available as an open access title on MIT Press’ website. Should you decide to use this book in class rather than her most recent one, you obviously need to provide some more recent examples of safety-related incidents and accidents.
Levenson considers safety an emergent systems property. In particular, she argues very convincingly, that one cannot talk about software safety divorced from the context it is used in -software is always part of a sociotechnical system and it is in the context of such systems that safety must be considered.
As a software engineer, I found this book engrossing and enlightening and it made me realize why such well-respected figures such as David Parnas have always insisted software engineering education needs to include thorough grounding in more traditional engineering disciplines -this insistence gains particular importance in a world bent on developing self-driving cars, drone fleets, the internet of things and increasingly tighter software-controlled loops binding all sort of devices operating in and on the physical world.
The first part of this book covers topics on the nature of risk. There are chapters covering such key questions as changing attitudes towards risk, the nature of risks in an industrial society, how safe is safe enough, the role of computers and humans in accidents, the incidence of ineffective organizational structures and safety cultures and just what it means to identify root causes of accidents.
After a brief two chapter introduction to system safety from a historical point of view which notes the main contributions made by systems theory, systems engineering and systems analysis, Levenson delves into key definitions, distinguishing clearly between failure and error, accident and incident, hazard and risk, safety and reliability, safety and security. She also overviews accident and error models.
The first three parts lead into the crucial fourth part, Elements of a safeware program, whose importance is underlined by the fact that it constitutes almost half the book. Its chapters include:
11. Managing safetyLevenson does not provide a methodology, as the part’s title puts it, she covers elements which can be used to analyze and design system safety. Some chapters overview a plethora of techniques of varying degrees of effectiveness and practical use.
12. The system and software safety process
13. Hazard analysis
14. Hazard anaysis models and techniques
15. Software hazard and requirements analysis
16. Designing for safety
17. Design of the human-machine interface
18. Verification of safety
Four appendices on historically important safety-related accidents are also included. Appendix A deals with the Therac-25 computer-controlled radiation therapy device which massively overdosed six people between 1985 and 1987. Appendix B treats the approach to safety developed by civil aviation and several specific aerospace cases: Apollo 13 (1970), the DC-10 cargo door blow out which caused 346 casualties in 1974 and the space shuttle Challenger tragedy (1986). Appendix C covers the Hoffman-LaRoche’s subsidiary Givadau’s Seveso (Italy, 1976), Nypro’s Flixborough (UK, 1974) and Union Carbide’s Bhopal (India, 1984) chemical plant disasters. Appendix D covers Windscale, Three Mile Island and Chernobyl nuclear power station accidents and disasters. Appendices B, C and D start with an overview on safety in a specific industry. Each case in the appendix provides sections on background, the nature and state of safety features present, events leading up to the accident or disaster and causal factors contributing to it.
Nancy Levenson is not subtle, she hammers her points home. This leads to some repetition and the book’s exhaustiveness occasionally may also become somewhat exhausting -laying the book aside for a couple of days rest is probably all it takes to keep going. Sometimes the writing seems a little too cobbled together, the list of sixty guidelines for safe human-machine interface design is a case in point -what is the difference between guideline 5 (Distinguish between providing help and taking over) and 19 (Design to aid the operator, not take over).
Since my Ph. D. thesis was on writing specifications, I particularly enjoyed the chapter on software hazard and requirements analysis.
Admittedly without having read much in the field of safety, I feel the book has aged well. Its main lessons are still very, very pertinent. If you plan to use it, at least take a look at Nancy Levenson’s more recent book, which is, at least at the time of writing this review, is still available as an open access title on MIT Press’ website. Should you decide to use this book in class rather than her most recent one, you obviously need to provide some more recent examples of safety-related incidents and accidents.
September 5, 2007
Im reading this for school. I think Safety is important because one of my childhood heroes was Smokey the Bear. Also, I was a Cub Scout.
Unfortunately, this doesnt tell you how to prevent forest fires but talks about safety systems from an ethical point of view (Ford Pinto), environmental, etc.
I havent got to the actual making software safe part yet but I'm sure it will be fabu (Thats fabulous).
Since you asked:
Software safety is important in things like medical systems, see the case study of that machine that irriadiated people to death because the program was bad and gave people wayyyyy too much cancer-fighting relief, robots (except for war robots, you want those to be unsafe), jetplanes and the like.
Unfortunately, this doesnt tell you how to prevent forest fires but talks about safety systems from an ethical point of view (Ford Pinto), environmental, etc.
I havent got to the actual making software safe part yet but I'm sure it will be fabu (Thats fabulous).
Since you asked:
Software safety is important in things like medical systems, see the case study of that machine that irriadiated people to death because the program was bad and gave people wayyyyy too much cancer-fighting relief, robots (except for war robots, you want those to be unsafe), jetplanes and the like.
Read
November 8, 2009safety of critical systems
Displaying 1 - 3 of 3 reviews