The Checklist Manifested

Subscribe to our newsletter


Build them Curiously Strong

If checklists are to be effective as behavior-management tools, you must manage the behavior of using checklists!

checklist

 

Today is the day our family vacation is to begin.  We have so much to remember: Take the pets to their hotel; go to the bank; take out the trash; pack power cords, prescriptions, underwear;  turn down the heat; make sure my 18 year-old brings his ID to the airport this time… the list continues.  We know that once we’re on the road we’ll realize we forgot something.  Wondering what we’re forgetting while we’re forgetting can be maddening.  

Recently, before taking any trip, I’ve adopted a good habit of making a list on my iPhone and checking off each item as soon as it is accomplished.  When I am disciplined enough to do this, I tend to be a happier and more successful traveler.

71CwWiCJhuL

The simple checklist has gotten a lot of press recently.  A couple years back, Atul Gawande put out his popular Checklist Manifesto that described the use of checklists in the operating room and arguing that this marvelous tool can be used to reduce injuries, quality errors, and perhaps even travel forgetfulness.  On the wave of checklist mania also rides a group of former military pilots who call themselves Check Six.  They make a strong case for preparing very precise checklists that allow for quite amazing, yet safe gravity defying feats.  I’d be willing to wager that there are a dozen more groups marketing their particular take on checklists.

 

altoid1

I personally am very much a supporter of the checklist.  My behavioral science colleagues and I have been studying checklists for decades now.  We have been publishing research investigating the checklist’s efficacy in improving quality, sales, sanitation, and, of course, as a tool for promoting safety.  It is no accident that we made the checklist the primary tool of Behavior-Based Safety.    From our behavioral science perspective, the checklist is the Altoids Mint of behavior change techniques… it’s curiously strong … when done right.

Checklists offer a nice mix of antecedents that clarify for users which of their specific behaviors require prompting at the moment.  Progressing through and completing to satisfaction each of the items on the checklist can provide some mildly reinforcing feedback in the form of a job well done.  To heighten their impact, checklists also can be designed to be associated with other more powerful consequences.  Pilots, for example, cannot gain clearance to take off until they complete their checklists.  In other cases employees are required to submit their checklists so supervisors can verify their use.

Creating checklists, however, isn’t necessarily a ticket to success.  In fact sometimes they can prove counterproductive. 

You see, while checklists can be used effectively to manage behavior, we must remain mindful of the fact that the act of using checklists is itself a behavior.  If checklists are to be effectively used as tools for managing behavior, the behavior of using checklists themselves must be managed and reinforced.

ANOTHER CASE OF PENCIL WHIPPING

Once I was engaged in an oil field, attempting to identify and manage critical behaviors that lead to losses such as injury, process safety incidents, or service delivery interruptions.   By reviewing records, we found that a particular piece of equipment was contributing to the greatest loss.  Most of the incidents could have been avoided by using common preventative maintenance procedures.  But that wasn’t happening.

Operators were supposed to conduct the preventative maintenance (PM) routine within a 60-minute interval when the machine was idle between cycles of operation.  Managers had been assuming this PM was ongoing because all operators turned in the required checklist to their support engineers to verify they did the PM.  Piles of these completed checklists could be found on every site.

box2

 I asked to look at them and was handed a stack that had just been turned in.  As I paged through them, I noted that they were all the same… exactly the same.  They literally had been photocopied!  The person who submitted them had been so brash as to mark out the photocopied date and write in a new one!

A critical piece of equipment requiring essential maintenance was causing significant loss because the checklists were being pencil whipped and the “completed checklists” were never verfied. 

I’m sure some engineer spent substantial time working with equipment specs to make the perfect checklist… but to what end?  The behavior of actually completing the checklist accurately had not been considered. Nor had any method been implemented to review the checklist and to take any necessary steps to repair the situation.

Beyond that, the checklist contained over 100 items!  It was not physically possible for anyone to accurately complete all 100 items in less than 60 minutes. The design of the checklist had failed to consider the context of the work.  (I also wondered to myself, who would want to move a paper checklist in and out of their pockets while performing a greasy maintenance job.)

Let’s have a lesson in the behavioral concept of Response Cost.  The more effort it takes to do something, the more punishing it is, and, consequently, the less likely it is to get done.  This is a basic principle that governs nearly everything we do. 

Consider relaxing comfortably on your couch and deciding you want a beer from the kitchen in the next room.  While you’re about to get up to retrieve the suds you remember all the beer is in the basement refrigerator.  You then sigh and settle back in your couch.  The response cost of getting that beer had become greater, to the point it outweighed the potential reward you would attain from the beer.

In the case at hand:

·      Completing a 100-item checklist demanded too much response cost of the operator.

·      Reviewing these 100-item checklists and verifying PM completion demanded too much time (response cost) of the supervisor, especially in light of the more immediate work demands required by the oil-field-operation.

World-Currencies

Consider behaviors as currency - effort to be invested.  Behavior is going to go where it can have the greatest immediate impact during a particular bloc of time.  And “impact” is interpreted on a personal level.  A 100-point checklist, therefore, requires considerable behavioral currency, but provides little return on that investment. 

The use of checklists in surgery or piloting an aircraft probably is worth the investment.  By comparison, skipping the checklists targeting preventive maintenance does not produce immediate consequences nor the catastrophe of a botched surgery or downed plane. If the conditions to support the ongoing practice of checklists are absent then you need to install a performance management system specifically related to that behavior. 

You could increase the reinforcement for doing the checklist through positive reinforcement such as praise or other good things… yeah right.  Managers are more likely to use negative reinforcement with threats and general meanness if the checklists are not completed.  But that would require someone enforcing those consequences.   Think about it, enforcement requires a big response cost for the enforcer … so managers typically avoid enforcement.  Only the most bureaucratic entities actually do such a thing by employing a team of “inspectors” that often fail to add value.

A second path is to reduce the response costs (in terms of time and effort), associated with the checklist.  In the oil fields, we gathered a focus group of employee/operators with some support folks and supervisors responsible for the equipment and guided them through a structured method of identifying and prioritizing those critical behaviors that would lead to the greatest reduction in loss. 

A 100+ item checklist was reduced to 3 items.  The other 97+ tasks were transferred to other points in the operation or to the maintenance yard where they are more likely to be done.

3d-simplify-business

After the employee-designed checklist was implemented in a pilot region, loss attributed to one behavior dropped over 60% and the use of these “checks” (not called a “checklist” any longer) increased to between 75% and 300% depending on the site.

In the longer term our plan was to keep the check dynamic so it could adapt to changing conditions.  Checklists should be living documents.  After our operators demonstrated mastery, by completing the behavior to standard nearly 100% of the time, it would be time to take that behavior off the checklist and replace it with another critical behavior.  Knocking an item off a checklist can be reinforcing.  Managers can sincerely praise these events because this mastery is linked to operational success.  No response cost there, only something worth investing in.

Are your checklists “Manifested”?

       o   Build them smartly by involving your people.

       o   Target critical and readily doable behaviors    

       o   Keep them short and living

And they will be Nimble

                        And Curiously Strong.

                                   

Timothy Ludwig’s website is Safety-Doc.com where you can read more safety culture stories and contribute your own.  Dr. Ludwig serves as a commissioner for Behavioral Safety Accreditation at the non-profit Cambridge Center for Behavioral Studies (CCBS: behavior.org) and teaches behavioral psychology at Appalachian State University, in Boone, NC.  If you want Tim to share his stories at your next safety event you can contact him at TimLudwig@Safety-Doc.com.

 

 

 COMMENTS:

Tim , A good reminder about the pitfalls of using checklists.
 Checklists are useful for directing human behaviour and can help reduce the risk of hazardous events from arising. This web site explains the origin of pilots using checklists.
http://www.atchistory.org/History/checklst.htm
 I often tell people that pilots are highly trained competent people but still use checklists.You may then ask why? That's because they are human (like the rest of us )and can still make mistakes. I'd be interested to learn where else checklists are used to good effect or to hear of examples of mis-use. I know of one instance where an operator was distracted when preparing a batch reactor and missed the addition of a key ingredient. It was concluded that he had ticked the checklist box prior to making the addition ,got distracted and returned to move onto the next step thinking that he had added an ingredient. The reactors ingredients solidified and had to be physically dug out.                                                      

 ******

I am in agreement here - checklists are a tool not an end in themselves. As Tim's article points out they become useless unless used correctly and verified especially in mission critical situations and equipment are involved. Now the whole thing becomes a culture if the users of checklists are "pencil Whipping" and the submitted checklists are not being adequately verified or independently checked through the internal / external audit process then a "she'll be right" culture will grow until an incident (hopefully not fatal) will occur. The ensuring investigation (assuming there is one) should then identify to senior management the wrong culture that has arisen then they have a major job on their hands to bring about a change. unfortunately that change will probably start with job losses.
 Train now for success but verify constantly to ensure compliance.

******

Tim, Nice paper, many thanks for sharing.
One of the associated underlying organisational failures is failure to manage the time/resource model - often multiplied by failing to manage organisational change management.
Similarly, checklists are often seen (as you infer) as 'idiot sheets' for individuals to work through. Simple tools for simple folks? As always, this is far from the truth; used correctly as an aid memoire (and as a team exercise), they stimulate the necessary cognitive processes to jog memories, reset situational violation pressures and assure a risk optimised approach.
Reports have shown that vocalising 'lost items' results in significantly greater 'found' results. Everyone, I'm sure has looked for those car keys (or eyeglasses) and eventually found them exactly where we thought they were and - more importantly - had already looked on a number of occasions! Vocalising 'keys, keys, keys' seems to allow us to 'see' the keys, where before they were 'invisible'.
Checklists similarly allow a 'call and response' approach to ensuring we are REALLY seeing what we are looking at, instead of seeing what we expect to see.
They are obviously not infallable. I remember reading of an instance of a US submarine loss where the Engineer was responsible for ensuring the air inlets to the engines were closed before diving. The submarine was lost and the recovered checklist had this item checked and countersigned...but had not actually been completed, directly leading to the loss. Confirmation bias in action? 

*******

Nice read. If I may, I would like to point a few things that help to make a checklist be effective, they need to have some of the following things:

Where it can be applied, include what the acceptable range may be for the item that is being checked. For example if someone is is checking the differential pressure on a filter word the checklist like this:
Yes No
□ □ Filter differential pressure (normal range 7 to 12 psi)

The other aspect of a checklist is that it is reviewed in a timely manner and follow up discussion occur when items are abnormal to determine and/or demonstrate corrective actions. In the case of the filter differential, if it was to high, communicate to the employee that a work order was turned in to change the filter.

Set up the checklist to record values instead of yes/no.

Organized the checklist so it flows with the work that is taking place. It is awkward to check box one and then have to remember that box 15 is also checked at this same point.

Do not accept or create checklists that promote the same as above response. Use signatures, initals and numbers, and other alphanumeric responses.

Thanks for bringing up such a valuable topic.  

*****

A checklist is normally used to be a record of validation that all steps were followed, It also is the conscience of the users and the process owners. It's what keeps a restaurant going if the chef is off, his replacement can use his recipe's and follow them to the letter and not lose customers. Many times if there is an issue the time line can be compared to the checklist and see what steps were the responsibility of certain people and shifts, Not a gotcha deal, but a way to correct issues to save money and even lives.                    

**********

Great article. So much being written about checklists in general, but very little about how to implement them. My kids elementary school has started using simplest.me and the teachers have been very clear that they will not be punitive about using this tool - but rather use it to support students on following specific steps to help them get from A to Z. I've actually just started using the same tool for some consulting work I do and found that asking users to use it (vs. forcing them to do so) has gone a long way in adoption. Thanks again for the article!


Contact Dr. Ludwig                                 © Timothy Ludwig 2017