Burley is recognized worldwide for bicycle trailers that set the standard for safety, durability and design. The company’s products also include multi-functional child carriers and jogging strollers—and in terms of stability, structural integrity, and other safety criteria, they more than live up to their name. In fact, Burley was instrumental in helping to create the American Society for Testing and Materials (ASTM) safety standards for passenger bicycle trailers. “At Burley, our mission is to enable adventure,” says Mary Craighead, quality manager at the company’s headquarters in Eugene, Oregon. “We help parents and families develop the next generation of riders, and a big part of that is our commitment to quality and safety.” Craighead works with in-house engineers and external manufacturers to make sure every item Burley sells meets or exceeds quality and safety standards. That requires collecting and analyzing a great deal of data throughout the entire product lifecycle, and Craighead relies on Minitab Statistical Software to make that part of her job faster and easier.
Over the past 12 months, Craighead has led Burley’s transition to a more proactive quality management system. “We realized that we could gain a lot of benefit from improving the real-time visibility of our production process,” Craighead explains. “We’ve always rigorously tested and inspected products after production and before shipment to customers, of course, but we wanted better insight into what happens during actual production to drive more effective continuous improvement efforts. So, we’ve been using Minitab to establish baseline performance and monitor materials handling and sub-assembly processes; it’s a whole lot easier to manage processes that we can measure. Minitab has helped us get metrics in place for our leading quality indicators very quickly.”
Craighead first encountered Minitab in graduate school, and appreciated the software’s flexibility and ease of use. “For each specialized course I took—like design of experiments, measurement systems analysis, and reliability studies—Minitab had a dedicated component or feature,” she says. “I also see a tremendous amount of value in the ease of use and accessibility of outputs from Minitab as compared to some other programs, which require you to be quite proficient to use many of the tools. Minitab does a great job of offering help and support to make statistics comprehensible and approachable for just about anyone.”
In addition to using it to establish baseline quality metrics across the company, Craighead also has been using Minitab for specialized quality projects. “Most of our trailers feature an aluminum frame and textile covers, and most of the interior components are also textile-based,” she explains. “I’ve done a lot of Pareto charting to focus in on different areas that are possibly more difficult to manufacture and control and may not be running as efficiently as possible; I recently led a Six Sigma project in collaboration with our manufacturer to help improve our sewing and textile performance. Minitab was perfect for a lot of our capability analyses and control charting.”
In the course of that Six Sigma project, Craighead discovered the challenge she’s tackling now—one that Minitab is well suited to help with. “In reviewing historical data, I noticed disparities between what inspectors here at Burley’s headquarters considered a defect versus what inspectors with our manufacturing partner flagged as a defect,” she says. “There were things that we would mark as a fail here at Eugene that the factory inspectors weren’t seeing, and then there were other things that the factory would fail an item for, but that we would consider very minor.”
“For example, inspectors at the factory would immediately fail a cover upon finding a long thread,” Craighead says. “We want threads at the end of a stitch line to be less than a centimeter in length, so if a thread was longer than that, inspectors at the production facility immediately noted it as a failure. It should be recorded as a minor defect—but if that’s the only issue found, all it takes is a quick trim, and it doesn’t make sense to scrap the entire cover!”
Craighead realized that she needed to make sure all of Burley’s inspectors were seeing and treating defects in the same way.
If different inspectors see a potential defect differently, inspection results are biased to the individual auditor, and rooting out variation is a primary target for quality and continuous improvement. Individual evaluators also may be inconsistent in their own judgments, passing a part today they might reject tomorrow.
“We suspected there were specific reasons why we might be seeing disparity between inspection records,” Craighead says. “But before we could do anything to fix it, I needed to understand where exactly different inspectors weren’t in sync, and how far apart they were.”
Craighead used Minitab to set up an attribute agreement analysis, which is a type of measurement system analysis. In contrast to gage R&R studies, which are used to evaluate the precision of continuous measures such as length, width, or weight, an attribute agreement analysis evaluates how consistently appraisers rate items using qualitative classifications. For example, inspectors may be classifying items as pass or fail, or on a 1-5 scale.
Craighead selected 10 sample trailer covers for the analysis, six of which were modified to fail to meet the company standard, and four of which were made to specification and considered passing. Then she entered the number of samples, appraisers, and replicates into Minitab to create a study in which each participant—four from Burley’s headquarters (two experienced inspectors and two brand new), and four from the production facility in China (two experienced inspectors and two brand new)—evaluated each sample cover twice. To ensure the attribute agreement study is performed correctly, Minitab automatically generates a worksheet to facilitate recording and analyzing the data.
An attribute agreement analysis may show that appraisers judge qualitative factors very consistently. Alternatively, it can reveal that some team members make very different judgments than others, or even that individual evaluators do not always rate the same item the same way.
After collecting her data and analyzing it, Craighead was able to verify that the evaluators at Burley’s headquarters and those at the production facility were not all evaluating the same items the same way. “The overall accuracy percentage—how well our inspectors agreed with the standards and with each other—was just under 60%,” she says. “Meanwhile, the parity between experienced inspectors at both Burley and the manufacturing partner were much higher. So the analysis also showed, as we suspected, that the parity between our inspectors just wasn’t what it needs to be.”
Beyond merely confirming Craighead’s suspicions, however, the results of the analysis also pinpointed areas where the team could make improvements through training, developing clearer standards, and other actions. “We were able to identify where the evaluations had great variation, look closely at those types of defects, and determine how to make sure we were all treating the same defects consistently regardless of experience level.”
An attribute agreement analysis often reveals weak spots in the overall evaluation system, rather than problems with individual appraisers, which was certainly true in Craighead’s experience. “We found that newer inspectors were more likely to rate a fail as a pass—but that’s because we weren’t explicitly telling them what to look for, so it was very easy for certain defects or issues to be overlooked.
“Several significant defects did not appear as explicit line items on the inspection sheet, and while inspectors could check off whether an observation was critical, major, or minor, we didn’t spell out how many of each defect type would make an entire unit defective.”
Based on what Craighead and her team learned from the attribute agreement analysis, Burley revised its quality inspection standards and forms and added a color-coded risk and action matrix based on acceptable quality limit (AQL) levels that provides clear direction for inspectors to take specific actions when certain thresholds are met. The company also set up standardized training for inspectors across its operations, leveraging use of a new visual defect guide that not only identifies defects but also classifies their severity, to make sure they all understand the expectations, how to use the forms, and how to evaluate particular defects. Craighead plans to conduct another analysis next quarter to measure the effectiveness of these improvements.
More consistent assessments will help Burley’s bottom line, Craighead says, but even more important, measuring and monitoring process performance will help ensure that Burley’s reputation for outstanding quality and durability remains strong. “By keeping our team focused on the most important contributors to safety and functionality, we’ll be able to deliver long-lasting products that fulfill Burley’s mission—to enable adventure.”
Make sure inspectors at facilities in the United States and China recognize and flag defects appropriately and consistently.