Measurement Systems Analysis
    Is the Soup Really Too Spicy?

    Play Video  

    Measurement Systems Analysis

    Is the Soup Really Too Spicy?

    Mama Bear had reached her limit. The latest debate over the spiciness of her soup had ended in bared teeth, brandished claws, and one very terrified little girl.

    Mama Bear believed that little girl was the problem. Since Goldilocks arrived, nobody agreed on anything. The beds were too soft, or too hard. The porridge was too hot, or too cold. And tonight, the soup was too spicy, according to Goldilocks—or not spicy enough, according to Papa Bear.

    To restore peace to the household, Mama felt the girl needed to leave. But whenever she brought it up, Papa Bear flew to Goldilocks’ defense. Mama needed data to bolster her case, and she knew just how to do it: with a type of measurement system analysis called Attribute Agreement Analysis. With the Assistant in Minitab Statistical Software, Mama knew it would be easy.

    First, she cooked six batches of soup. To three of them, Mama added just enough pepper and spices to make it flavorful, but not too hot. She fortified the remaining three batches with an excessive amount of habanero sauce.

    Follow along as Mama Bear uses Minitab Statistical Software to set up, collect, and analyze the data she hopes will identify why the Bear household is disagreeing so frequently.

    Step 1: Set Up the Attribute Agreement Analysis

    Select Assistant > Measurement Systems Analysis (MSA)…

    The Assistant presents a decision tree, which includes MSA for “Appraisal” data.

    Mama plans to have each member of the household taste six different soups, and identify them as good (“Zesty!”) or bad (“Ow!”). Each evaluator will taste each soup 4 times, so Mama will be able to see not only how well each tester agrees with the others, but also whether they agree with themselves.

    But will her plan for gathering data suffice for this analysis? Clicking on “more…” below Attribute Agreement Worksheet brings up a list of requirements to check. For example, the Assistant recommends that appraisers rate the same number of good and bad items, and that there are at least 3 appraisers, both of which are covered in Mama’s plan.

    Now that she’s confident her plan is suitable, Mama clicks "Back", then on “Attribute Agreement Worksheet” and fills out the dialog box as shown:

    After clicking OK, Minitab generates a set of four randomized trials per appraiser and per soup, and creates a matching data sheet.

    Minitab also lets you print a full set of ready-to-use data collection forms—one form for each trial and appraiser.

    Next Mama heats the six soups, and has each tester taste each soup four times, in the random order specified by Minitab. Then she simply enters the results into the appropriate column of her datasheet:

    You could collect your own data at this point—or download and use the data Mama Bear collected in her test.

    Step 2. Analyze the Data

    With the results entered in the worksheet, Mama can proceed with the analysis. Return to Assistant > Measurement Systems Analysis (MSA)…, but this time, instead of setting up a study, the objective is to analyze data:


    Complete the dialog box as shown, click “OK”, and Minitab produces all of the output Mama needs to make her case.

    The Summary Report shows that, overall, the household correctly identified the spiciness of the soups only 75% of the time. The % Accuracy by Appraiser graph shows how accurately each appraiser identified the soups. Mama and Baby Bear scored highest, with 95.8 and 87.5 percent accuracy. And Papa Bear wasn’t too far off, with 79.2 percent accuracy.

    Goldilocks, however, had a dismal 37.5 percent accuracy rate.

    The Assistant’s Accuracy Report gave Mama more specific details about how each appraiser performed. The % by Appraiser and Standard graphs on the right side of the report showed that each of the three bears were very good at identifying the Zesty soups, but Goldilocks identified them correctly less than 20 percent of the time.

    The Proof Is in the Soup

    The results of the Attribute Agreement Analysis make it clear where most of the disagreements about spicy soup originate…and, in Mama’s opinion, where the disagreements should end. With the straightforward graphs created by the Assistant, even Papa Bear could easily see how Goldilocks’ divergent assessments of the soup could lead to conflicts in the Bear household.

    Whether you prefer your soup mild or extra spicy, you’ve completed this measurement systems analysis successfully. You have seen how to create an attribute agreement analysis worksheet and analyze the data you collect to identify where appraisals agree and disagree using the Assistant.

By using this site you agree to the use of cookies for analytics and personalized content in accordance with our Policy.