Sugar, Spice, and Everything Statistics: Using Design of Experiments to Bake a Better Cookie


That batch of cookies that just came out of your oven looks nothing like the pictures in the cookbook. Talk about a recipe for disappointment! It’s vexing enough when an ordinary batch fails to meet your expectations, but when you’re making cookies for the holidays the disappointment can approach traumatic intensity. Who wants to eat a cutout cookie that looks more like a melting snowball than a snowman?

When the first batch of cookies Bill Howell made last year for family and friends produced…well…mixed results, he applied the power of Minitab to find a solution.

Design of Experiments

Howell, an avid baker and a quality professional at Scheider Electric, used Minitab’s  design of experiments (DOE) tools to get to the bottom of why his cookies failed to hold their shape. A designed experiment consists of a series of runs, or tests, in which you adjust multiple variables—for instance, the proportions of the ingredients used to make a batch of cookie dough.

Many people think that to study multiple factors in an experiment, you must vary one factor at a time while holding all the others constant. In a designed experiment, however, you can change more than one factor at a time, then use statistical analysis to get meaningful results about all of your factors simultaneously. It’s an efficient and economical way to improve almost any process.

“My goal in this experiment was to evaluate the operational space of my standard cutout cookie dough recipe and cooking techniques, quantify the components of variance of the factors under study, then operate the process in the region that produced minimum change in shape and dimension of the finished cookie,” Howell explains.

In other words, Howell planned to design an experiment in Minitab that would let him screen many factors, determine which were most important, then adjust his process to get the results he wanted—in this case, to make cookies that still looked like snowmen when they came out of the oven.

A Fractional Factorial Experiment

Minitab can easily create and analyze many kinds of designed experiments, and includes an extensive Help system to assist in identifying the right experiment for your situation. Howell elected to run a fractional factorial experiment, a class of factorial designs that lets you identify the most important factors in a process quickly and inexpensively.

Howell’s experiment required him to make 8 runs (or batches of cookies) to assess six factors, each of which was tested at two levels:

  • Oven temperature (325 or 375 F)
  • Number of eggs in a batch (1 or 2)
  • Ounces of AP Flour in a batch (9 or 13.5)
  • Baking soda amount (0.5 or 1 teaspoon)
  • Cream of Tartar amount (0.5 or 1 teaspoon)
  • Chilling the dough after rolling and cutout (yes or no)

Howell took extensive steps to ensure a robust process. He used 3 different shape cutters to prepare the cookie dough for the oven, selecting measuring points on each cutter, and measuring them with a 6 inch caliper accurate to .001 inch. Each of the eight experimental batches included stars, snowmen and gingerbread men. To ensure consistent dough thickness, Howell used wood strips to prevent his rolling pin from flattening dough any thinner than ¼ inch. To minimize undue influence or unintentional bias during the baking process, he randomized the placement of the cookies on the baking sheet. He also rotated the baking sheets 180° halfway through baking.

Because two oven temperatures were used in the experiment, baking times varied by trial. The actual cooking times for each trial were recorded on the trial instruction sheet.

Each trial consisted of baking two trays of cookies. When they came out of the oven, Howell measured two samples of each shape from both trays to see if there had been a change in overall height, a selected width measurement, or thickness. These dimensions were recorded on preprinted forms, which identified the trial number, data of trial, cutter shape, width and height. Howell calculated averages and standard deviations for each cutter shape, and used Minitab to analyze the data.

Analysis of DOE Results

An analysis of height and width measurements done in Minitab revealed that flour was the driving factor in spread of the cookie. “In each instance, a higher amount of flour produced less spreading from the original dimension,” Howell says. “Impact on cookie thickness was principally influenced by flour and the number of eggs in the batter. Two eggs produced more rise than one egg.”

Howell also used Minitab to create main effects plots, which examine differences among level means for one or more factors. A main effect exists when different levels of a factor affect the response differently. A main effects plot shows the response mean for each factor level connected by a line. If the line is horizontal, no main effect is present—each level of the factor affects the response in the same way. If the line is not horizontal, then there is a main effect, and different levels of the factor affect the response differently. The steeper the line’s slope, the greater the magnitude of the effect.

Howell’s main effects plots reinforced the findings of the analysis, and also revealed that cutter type had an effect. “The width measurement for the star-shaped cutter moved an average of .35 inches, but the Snowman moved an average .95 inches and the Gingerbread Man moved an average of .55 inch,” he says. “This indicates that the shape of the cutter affects the flow of the cookie dough as it bakes.”

Howell originally planned to replicate the experiment two times, to confirm his results and give his analysis greater statistical power. “But I was awash in cookies and had to limit myself to one confirmation trial,” he observes. “In the confirmation trial I pushed the envelope and raised the amount of flour by an additional 2 ounces. These cookies held their shape extremely well, but that level of flour made them too dry.”

Howell is confident the experiment he designed and analyzed using Minitab will result in better cookies in the holiday seasons to come.  “Cutout cookie batches this holiday season will follow the methods and levels that worked best in the experiment—with maybe just another ½ oz of flour thrown in—as these held their shape nicely, and the people who sampled from this trial liked their taste.”

But that’s not all—doing this experiment also gave Howell new training material and a little peace of mind. “This was a fun project, and I use it as an introduction to Design of Experiments when teaching Green Belt classes,” he says. “Better still, it resolved a pesky baking problem that had been nagging at me for years.”

Recipe for Success

If you'll be baking this year and you'd like to try Bill Howell's optimized cookie recipe, here it is.


13 oz AP Flour (measured)
½ level teaspoon Baking Soda
½ level teaspoon Cream of Tartar

2 sticks unsalted Butter, pliable or room temperature
6.4 oz Powdered Sugar (measured)

1 large egg
1 teaspoon Vanilla Extract
1 teaspoon Almond Extract

1 oz Colored Sugar


  1. Set oven temperature to 375ºF.
  2. In a separate bowl, mix AP Flour, Baking Soda and Cream of Tartar.
  3. In a stand mixer, cream unsalted Butter and Powdered Sugar for a measured 3 minutes.
  4. Add in Egg, Vanilla Extract and Almond Extract, mix 30 seconds.
  5. Slowly add materials from Step 2, mix until incorporated.
  6. Wrap in wax paper and chill in refrigerator, minimum 2 hours to overnight.
  7. Remove dough from refrigerator and use microwave to make pliable (normally 15-20 seconds) or allow to sit out to come to room temperature.
  8. Roll out dough into 3/16″ thickness. Use stick guides to control thickness.
  9. Cut into shapes; sprinkle with colored sugar.
  10. Bake 4 minutes on rack in center of oven, on Parchment Paper lined baking sheet.
  11. Rotate baking sheet 180 degrees.
  12. Bake another 3 minutes or until done.
  13. Remove from oven, cool 5 minutes, and then place on cooling racks.

By using this site you agree to the use of cookies for analytics and personalized content in accordance with our Policy.