September 2018 - Edition 03, Quality Assurance
Introduction
We are in the third edition, so at this point the format is pretty set unless I get any specific feedback on additions or changes. The topic this week turns towards quality. I’m not going to talk about the minutia of implementing quality programs, or any specific program, but instead focus on the reason quality is important to engineering and how it manifests itself in daily practice.
Demetri's Corner
I’m back at work again, and getting back in the swing is strange, but it’s been a rollercoaster getting everything in place. The work is shaping up as I intended at this point in my schedule, so I need to get back in the saddle and start getting into a pattern so I can move on to more speculative endeavors.
So on to the good stuff!
Today's Subject - Quality Assurance
As an engineer, especially working in the nuclear industry, you hear the term “Quality Assurance” or “Quality Program” almost all of the time. It becomes such a continuous drone, many of us tend to tune it out. That isn’t necessarily bad for engineers that have developed good habits as those habit probably conform with the quality program in effect – at least in intent. But for new engineers, or those with less experience with quality programs, or those that just learned bad habits, tuning out the background hum is downright dangerous.
Let’s start by going back to the beginning of this whole series of newsletters. If you were paying attention, you’ll notice my regular insistence about how an engineer must be prepared to always be wrong and there is no way to ensure “rightness”. That may confuse a lot of people reading this when they think in terms of a quality program. Many have been assured that strict adherence to a quality program means that the result is “right”. Stop reading if you want to maintain that belief as that is not my viewpoint. Correct implementation of a quality program means that there is reasonable assurance that you achieved what you thought was necessary. So there are two places where you can be wrong, even if you follow the program perfectly.
- Reasonable assurance. This is the concept that if enough people look at it in enough different ways, the chance that anything was missed is incredibly slim. You could probably put some sort of probability and confidence based on the type of review, experience of the reviewer, complexity of the problem…but if we did that, we would all be much more depressed about the results of all that effort, so let’s let sleeping dogs lie.
- The result you thought you were trying to achieve. Quality programs are built around the concept that you can define the end product. This is necessary, because if you didn’t define the end product, there would be no way of controlling the process, or measuring success. This goes back to previous discussions about really understanding your problem. You could do an amazingly accurate job of solving the wrong problem. In that case, the quality is outstanding, but the result is useless.
But both of these aspects of quality are critical to understanding engineering and getting the most out of the discipline. When we understand that the results of an engineering endeavor are only as good as having reasonable assurance there are no errors and that we properly defined the problem, then we understand the importance of quality in ensuring the least wrong solution to our problem, or the one with a high probability of “rightness”.
Since I’ve overly simplified quality into two critical factors, I’m going to double down on that bet and spend an extended amount of time on them below. My purpose is not to neglect all of the complexities in implementing and controlling a quality program, but rather address areas where I think engineers often succumb to tunnel vision. The real problem with blind adherence to quality programs is that engineers get so wrapped around the axle on the specific programmatic requirements, they fail to grasp the bigger picture. This is perhaps less troublesome if you do the same sort of task all the time, but if you regularly have to execute a variety of engineering functions, such as manufacturing, design, software development, commercial grade dedication, etc. it is too easy to follow a script vs. staying true to the intent of a quality program.
Reasonable Assurance
Reasonable assurance is a term fraught with confusion and misinterpretation. To make matters worse, it’s really different for different engineering tasks. Let’s take for example a calculation on beam bending vs. the design of the beam.
In the former case, we have a classic example of how quality can really ratchet down on “correctness”. It’s a closed form solution to a problem where a checker can replicate every input, where it was put, each individual calculation, the output, etc. And the reviewer has little more to do other than ensure that the correct equations were used in the correct manner. The room for error to propagate in this situation is extremely small. The onus of the task is on the preparer and checker, which if they are of reasonable intelligence, they will accomplish acceptably. The reviewer in this case is little more than a verifier of basic intelligence.
In contrast, the design of the beam may have many inputs which are either poorly defined or misunderstood, and a complexity which does not allow for a systematic approach to checking. The checker is left with the unenviable task of trying to follow the logic of the preparer in integrating the inputs into a design. And even if this logic is well defined by the preparer, there are decisions that have to be made which are effectively uncheckable, but the basis for making the decision still has to be understood and verified to be properly implemented. In other words, there is no way that a checker could independently come up with the same solution given the same set of inputs (unlike the beam bending problem). And to compound the difficulty, the reviewer has to consider the manner in which the inputs were addressed, the sufficiency of the design outside of the stated inputs, and whether the correct design controls were put in place. In this case, the onus is on the reviewer, but the harder task by far is that of the checker. In this case, the space where things can slip through the cracks is much larger than the first example.
Should would we claim that the first example has more “quality” than the second. Absolutely not, it is just that the definition of reasonable assurance is on a sliding scale. And arguably for important engineering functions, it slides more towards the second example. In both cases, the best protection we have against losing ground is abject honesty and clarity in what we did and why we made decisions so we enable checkers and reviewers to make the cracks as small as possible.
Setting aside the sliding scale, what does “reasonable” mean anyway? This is clearly dependent on the task you are undertaking. If you’re designing a trashcan that will be used only once, reasonable is that it has enough volume and the wall thickness is enough to not collapse. And if you got it wrong, the worst result is you have a mess on your hands. So the design review is simple and the stakes are low. If you’re designing a trashcan that will be made millions of times to hold medical waste, perhaps the design considerations are much greater, and the consequence for failure higher. In that case, what is reasonable for the first trashcan is clearly not reasonable for the second.
Industry has tried to put a button on this with various standards organizations putting together guidance on the appropriate controls for various tasks. Certainly that seems convenient, but often we find ourselves restricted in a program that makes little sense for the task at hand and we end up either force-fitting our process to match, or modifying the standardized approach. We’ll talk about that later.
Defining the Result
This certainly seems mundane and barely a quality function, but if you pay attention closely to all of the most common quality programs throughout industry, the concept of order entry in some form is codified. That seems like an administrative function and is often treated as one, but it’s the first time where engineers write down what problem they are supposed to be solving.
I spoke extensively about defining the problem in a previous newsletter, so I won’t bore you here, but my estimation is that it’s one of the least well performed engineering functions. A correctly defined quality program can turn that around by imposing the necessary reviews at the start of the process to ensure that the right (or as right as possible) thing is being done.
Dependent on the type of product, the way in controlling that the right thing is being done may be a simple as writing a few sentences defining the problem, inputs, and format of solution. It may even be transcribed from a proposal or from the client problem statement itself. The other extreme is where the problem is poorly understood, in which case a defined strategy in determining the problem and finding solutions is necessary. Often this takes the form of a project plan. Though the use of project plans are often looked down upon and seen as an unnecessary burden on the engineer, in many cases, it is at this point where most issues can be headed off in the pass. Therefore, the structure of the project itself is governed by a quality program, and as onerous as that may seem, it is often extremely necessary.
Industry Standardization
As discussed, a number of standards organizations in a variety of industries have put their two cents worth in on how to control quality based on the principles above (as well as others).
Standards that come to mind immediately given my background are those from ISO and ASME, but there are tons of standards for every industry and type of activity. This can put a confusing web over the top of engineering activities. So a single product may be controlled by a variety of quality programs and standards, starting with one that controls design, to another than controls manufacturing, to one that controls testing, and the list goes on.
These standardized approaches are created to simplify the execution, and more importantly, auditing of the processes and procedures put in place to ensure quality. These standardized quality systems contain a wealth of good practices which have stood the test of time in ensuring that a quality result is achieved. As mentioned, different programs are used for different industries due to the large variety of subjects which must be covered, and their relative importance to different engineering fields. Therefore, it is critical to understand the correct quality program to apply for a problem, as well as the rigor at which it should be applied.
Graded Approach
If you’ve been around long enough, you’ve likely heard the term “graded approach” when it comes to applying quality standards. This concept was created to address one of the problems above – how to leverage a standard industry quality program for activities which may not need the full rigor involved. On the surface, this seems like a good idea, and is really no different than writing a project specific quality assurance plan which tailors the standard program to your needs. Unfortunately it is often misused, and considered a way to lessen the burden of the quality program, but still invoke what would be considered the industry standard. This does a great disservice to both the standard referenced, and the engineers trying to implement the program. Too often I have seen procurements where the language is for the supplier to use a “graded approach” to some quality program when delivering goods, with no real guidance on what that means. This results in inconsistent application and understanding of the requirements, frustrating both the supplier and client.
There is a reason that the standards are many pages long, as the application of a quality program is complex and requires consistent application to be effective. Effectively saying, “some parts of this don’t matter for this situation” and leaving it at that, can cause a great deal of difficulty, because without proper guidance, portions of the program which may not seem critical end up being the stumbling block. Had a little more time been put into the thought process, better guidance would have resulted in getting the correct application of quality initially rather than the more common iterative approach.
To properly use a graded approach, the concept of commercial grade dedication must be embraced. This involves determining what aspects of the quality program are necessary to ensure that the end result of the engineering activity results in a product with sufficient rigor and control. I will likely do an entire newsletter on this process, so needless to say, it is complicated. This process is complex for a reason, and if there is no value in going into this process, it would likely be more appropriate to capture the standard program in its entirety, or write a project specific program which explicitly references the applicable portions of the quality program and does not require compliance with any other sections.
In Summary
So what does all of this mean? In my opinion, the concept of “quality” should be ever present in an engineer’s thoughts. The idea that all results of engineering products are controlled by properly defining the end result and providing confidence that it was achieved are inherent in all engineering activities. With this in mind, it is quite easy to live in a world governed by QA programs without getting bogged down in the paperwork. Certainly there will be a lot of paperwork, but allowing it to obscure the basis behind the process does a disservice to both the quality program and engineering. If the principles are engrained in everything you do as an engineer, you will rarely find yourself in a situation where an audit determines there was a consequential lapse in quality. Though if you didn’t do the paperwork, you’ll probably be hit with a bunch of administrative findings…so go ahead and do the paperwork too.
Dose of Aphorisms
The word mash today highlights why I think engineering is so valuable, and will never be replaced by a machine (in my estimation). We can talk about process and procedures all day long, but in the end, it’s the wild leaps of faith based on incomplete information which allows for tangential progress which is key to expanding our horizons.
A computer can replace an analyst, but a level of irrational logic is required to be an engineer.