Boeing has been in the news lately for a door plug panel falling out during flight on a 737 Max 9 aircraft.
This blog is a guest post by Bob Ashenbrenner and Zafer Mehmood.
- Bob Ashenbrenner served as Director of Dell and other computer companies in the development of servers, PCs and notebooks.
- Zafer Mehmood held Vice President and Director level roles, leading Quality Assurance and Software Development at InvestCloud, Fiserv, AT&T, Bell Laboratories and other companies.
Management Failures Caused Boeing Door Plug Incident
Previously, the Boeing MCAS (Maneuvering Characteristics Augmentation System) issues on 737 Max 8 planes have been implicated in crashes of two aircraft, killing a total of 346 people on the Lion Air Flight 610 on October 29, 2018, and Ethiopian Airlines Flight 302 on March 10, 2019.
Whistleblower reports indicate a culture of poor oversight and questionable use of outside suppliers, whether fuselage manufacturers (door plug) or software contractors (MCAS). More reporting indicates that Boeing was experiencing more than 1 serious defect a day during the manufacture of the 737 Max 9.
These are all examples of senior management failures, failures that all too often are seen in many firms. If one is managing a highly technical endeavor, those making decisions must thoroughly understand the design and manufacture of their products and processes. The reporting of recent failures suggests a breakdown at the senior management level.
Technical staff are judged on the objective results of their work product. It works or it doesn’t. To achieve success, rigorous techniques are used, techniques that are proven to be successful. The engineer responsible for the success of a component will learn and apply these skills and will often have their designs reviewed by other subject matter experts. This is generally true in many professions: marketers have techniques to analyze and address market needs, finance has accounting principles that are followed, and IT is driven by their success metrics of keeping the organization running.
Yet when many move into the upper management and executive ranks, they don’t seek out or accept proven management techniques, instead they rely on how they feel things should be done. They abandon the disciplines of their profession that were the foundation of their success.
Having a major technical issue more than once a day, on products with huge safety implications, is not a problem – it is a crisis. Management should treat this as a serious problem and address it with skill. If managers truly understand their business processes, they will immediately understand that a major fault per day (or more) is a major process failure. Their job isn’t to manage short term costs, it is to maximize the value of their products. First, a process that creates more than one major problem a day is flawed and designed to do so. These are not excursions, they are not anomalies, they are not situations where the staff should be yelled out to do better. This is a process “designed” to create failures daily.
So how should something like this be addressed? By using process and business analytics, determining the capabilities of the process, and developing corrective actions. Too many companies have devolved into regular meetings utilizing Red/Yellow/Green dashboards with executives yelling for better results. These dashboards encourage whack-a-mole types of responses, which does nothing to address the process capabilities or their weaknesses. It is like the dashboard in your car alerting to low oil level. Adding a quart every week can temporarily address the issue, but not until you figure out why the oil level is falling can the car be fixed.
These process and business analytic tools are hard work, but that is the job. Do the work necessary to address issues that you are responsible for. Remember, if a person makes a mistake, it is the process’s fault if that mistake makes it far down the line, or to the end customer. Obviously, it is better/cheaper to catch a problem closer to its source than later in the process.
Boeing designed the 737 Max series with a lower wing. When the need for more fuel efficient but larger engines was needed, their architecture didn’t support them. Thrust moved lower, which risked accidental pitching and a possible stall – a disaster for a plane. Let’s stop here for a moment. All great engineering firms have advanced technology teams that are looking out 10 or more years. Engine manufacturer roadmaps for fuel efficient engines would have been known at major aircraft companies. Yet, Boeing’s upper management (C-suite) decided to go with lower wings.
So, an effort to write software to take over the controls of the aircraft if a stall was imminent was pursued. For a work-around of this magnitude, it required an unusual amount of rigor and review. While using external contractors to write much of this software can result in success at a lower cost, that is NOT the goal. The goal is to get it right. Boeing has an impressive technical staff; they should have been in control of design and acceptance of all parts of the MCAS software. Senior management is required to ensure that every concern is identified and addressed. Subject matter experts should be listened to, and their advice should not be ignored unless the issues are thoroughly understood by the decision makers.
MCAS needs to know the air speed (among other things) to function properly. AOA (angle of attack) sensors are an important component and input into MCAS’s proper operation. Allowing only one sensor to be installed creates a single point of failure. Engineers in almost all industries consider many possible points of failure and ensure the design can gracefully fail. For example, computer designers’ number one priority is to prevent data loss. A failure of the power supply may crash the computer, but back in the day when spinning hard drives were used, the heads had a fail-safe to pop up off the platter to prevent data loss. In a PC, the implication of a failure is smaller in magnitude than an aircraft with hundreds of people on board. This is a case for redundancy; a single point of failure should not cause the MCAS system to fail.
Similarly, a communications failure in the case of the door plug appears to have caused an inspection step to be skipped completely. Based on the preliminary NTSB report issued February 6, 2024, the door plug had been removed at Boeing’s Renton, WA facility to repair 5 defective rivets in the fuselage close to the door plug, and the 4 bolts securing the door plug to the fuselage were likely never reinstalled. Therefore, this Max 9 left Boeing’s factory and was delivered to Alaska Airlines with these bolts missing. According to published reports from a Boeing whistleblower, Spirit AeroSystems – a Boeing supplier that builds the 737 Max fuselages in Wichita, Kansas – has its employees onsite at Boeing’s Renton facility to remedy any quality issues found subsequently by Boeing (warranty work). They worked on fixing the 5 defective rivets but the fact that the door plug was removed was never tagged in their tracking system (a removal requires a mandatory QA inspection step according to Boeing’s process). Therefore, this QA inspection was never performed.
One can have a good quality process, but still may not be sufficient to prevent quality escapes. A process needs to be continually improved, not just for the product lifecycle but also for Change Requests, repairs and rework. Each worker/associate must always be looking beyond mere compliance with the process and ask, “have we built in quality”, and “has something been missed”, especially whenever there is a deviation from standard workflows, i.e., following an alternate workflow as in this case of warranty work being performed by a supplier (Spirit AeroSystems) at a Boeing facility. It is this worker attitude and quality mentality that management needs to instill and encourage in their staff.
A sufficiently competent management team would never allow a single point of failure to ship for as critical a function as MCAS, or a plane to be delivered with fasteners missing. For C-Suite executives, it isn’t good enough to be great at business. For complex products and services, C-Suite executives need expertise in their production and quality processes. There are tools that can be brought to bear, if you don’t know what they are, you need to go find out.