The recent Boeing 737 MAX aircraft crashes have me a bit worried. I fly a lot and would have no trouble getting into a 737 MAX tomorrow. I’m not concerned with the apparent safety exposure. Rather, I’m concerned about something new and unknown.
Boeing is going to take a lot of heat over the crashes, and they should. There were several interrelated issues.
But, the main problem facing the 737 MAX will also impact other industries in transportation, including commercial vehicles: relinquishing control systems to Artificial Intelligence (AI). We should consider this as we continue on the road toward automated vehicles and increasingly “smart” automated vehicle safety technologies.
Issue 1: Accelerated schedule
First off, the accelerated schedule for production and launch of the Boeing 737 MAX aircraft was an issue. We all know that accelerated schedules can sometimes lead people to take short cuts. Boeing was in a heroic battle with their sworn competitor, Airbus, and behind schedule. Maybe Boeing figured they would catch up to Airbus, and then fix those minor issues that their employees were complaining about later.
Take another well-known example: rushing was one of the contributors to the sinking of the RMS Titanic in 1912. The crew attempted to cross the Atlantic Ocean quickly and didn’t slow down, despite reports of icebergs. At a more leisurely pace they could have potentially avoided hitting an iceberg, or limited the amount of damage to the ship.
But an accelerated schedule is not even the main source of my worry.
Issue 2: Conflicts of interest
The second issue is the relationship Boeing officials have with federal regulators. Like the close ties between Boeing officials and the Federal Aviation Administration (FAA) inspectors, this kind of relationship has been the source of many catastrophes in several industries. It seems as if it were human nature for an inspection organization to allow a market leader, like Boeing, to self-certify. After all, who knows the products better than the company that makes them? There is something to be said, though, about having a third party offer their seemingly unbiased analysis and a different perspective.
Even now, in the same news cycle, the recent collapse of a Brazilian dam, controlled by the mining company Vale, has resulted in a massive catastrophe and loss of life. Once again, a special relationship between the inspection organization and the mining company got in the way of proper engineering inspection.
Even so, the sweetheart relationship between the vendor and the regulator isn’t what makes me worry most.
Issue 3: Quality management
The third issue with the Boeing 737 MAX was a breakdown in the systems that ensure quality. W. Edwards Deming, an American engineer best known for the development of his 14 Points of Total Quality Management, said that quality inspections are too late. By the time an inspection is in progress, the damage is already doneQuality control can never replace systems that fail to produce a quality product.
At a company like Boeing, there is so much inspection and re-inspection, but no matter how many inspections a product goes through, if it’s not a quality product to begin with, the inspections won’t do much good. Although, this still isn’t the issue that worries me most
Issue 4: Unanticipated failure modes
Remember the Boeing 787 Dreamliner battery fire issues? They were big news. They were a failure mode, caused by high-current discharge rates, that were not anticipated.
After assessing one instance of these battery fires, which occurred in January of 2013 with a Japan Airlines Boeing 787, the National Transportation Safety Board (NTSB) issued an executive summary on November 21, 2014, and determined "that the probable cause of this incident was an internal short circuit within a cell [cell 5 or cell 6] of the auxiliary power unit (APU) lithium-ion battery, which led to thermal runaway that cascaded to adjacent cells, resulting in the release of smoke and fire. The incident resulted from Boeing’s failure to incorporate design requirements to mitigate the most severe effects of an internal short circuit within an APU battery cell and the Federal Aviation Administration’s failure to identify this design deficiency during the type design certification process."
Like in the case of the 737 MAX, the 787 battery fires could have been related to the complexity of the system. It does not lend itself to a complete inspection since all the failure modes are not yet known. However, even knowing that all of the failure modes are unknown does not worry me.
The main issue: Letting go of control
These automated systems assisting with the operation of vehicles are complex. They are starting to rival the complexity of our brain.
AI software is developed through millions of lines of code, written by thousands of people. No one knows how it all works together. I remember years ago that programmers were rewriting the system for a telephone switch (that still runs our landlines today). They took out a million lines of code that didn’t seem to “do” anything. But when they took them out, the switch stopped working. Unable to find the problem, they reinserted the “useless” code and the switch started to work again. We are moving toward a day in which no one truly understands the systems that run our lives.
In the 737 MAX, the problem was with an artificially intelligent system called the “Maneuvering Characteristics Augmentation System” (MCAS). The MCAS decided that the plane was going to stall (lose lift) and automatically corrected for it. The human pilots, and reality, disagreed. The pilots did everything they could think of to fix the issue, including turning MCAS off. They ended up crashing because the computer took over and caused a catastrophic incident. This set of facts troubles me.
We are rushing willy-nilly into a future where AI systems will be running airplanes, trains, trucks, and passenger cars. Not only do we have no idea how these systems work, the companies that build them don’t either. That worries me most.
Joel Levitt is the president of Springfield Resources, a management consulting firm that services a variety of clients on a wide range of maintenance issues. Levitt has trained more than 17,000 maintenance leaders from more than 3,000 organizations in 38 countries. He is also the creator of Laser-Focused Training, a flexible training program that provides specific, targeted training on your schedule, online to 1-250 people in maintenance management, asset management, and reliability.
