Blog 3: Design of Experiments🧪

Feb 1, 2025
7 min read

Updated: Feb 2, 2025

Welcome back!

It's been over 2 months

since the last blog 😪

Blog 3 is gonna be ̶a̶n̶o̶t̶h̶e̶r̶ ̶l̶o̶n̶g̶ ̶o̶n̶e̶ the longest one yet 💀

so let's jump right in ⤵️

Design of Experiments (DOE)

What is DOE? 🤷🏼

"A methodology to obtain knowledge of a complex, multi-variable process with the fewest trials possible" --- basically optimizing the experimental process itself 🤩

it is used to investigate effects of single factors, and how they interact with one another🧐

There are many types of DOE, but for this blog I'll be showing you how to use the

Full Factorial and Fractional Factorial designs🤓📊

Let's begin with the case study:

Now imagine this🤔💭

you've just sat down😌, bowl of popcorn in one hand

ready to watch your favorite show/movie🎞️🎥

You reach into the bowl and grab a handful🫳🏼🍿

*CRUNCH*🔊

and now your tooth is broken🦷😣

why? because you just bit into a "bullet" (un-popped kernel🌽)

..but fret not!

Because using DOE,

we can run experiments to find out how to minimize the number of "bullets",

and save your teeth🦷 from another painful surprise!😬

using Full Factorial design, we will investigate these 3 factors:

A: Diameter of bowl🥣 (10cm and 15cm)

B: Microwaving time⏲️ (4min and 6min)

C: Power setting of microwave🔥 (75% and 100%)

for Full Factorial, the number of runs (N) was determined from this formula:

..and 8 runs were performed using 100 grams of corn each time, giving us this table:

Table 1: Amount of "bullets" for each run (Full Factorial)

Run Order	A	B	C	Bullets (grams)
1	➕	➖	➖	3.75
2	➖	➕	➖	2.75
3	➖	➖	➕	0.74
4	➕	➕	➖	1.75
5	➕	➖	➕	0.95
6	➕	➕	➕	0.32
7	➖	➕	➕	0.75
8	➖	➖	➖	3.12

Legend:

➖: low level

➕: high level

e.g. For A (diameter), high level (➕) is 15cm while low level (➖) is 10cm

Since Full Factorial design includes all combinations for the high and low levels of the different factors, it lets us perform a complete analysis on how factors influence the outcome

From Table 1,

we can calculate the average amount of "bullets" at the high and low levels of each factor:

and taking the difference between the average for high and low gives us the effect on the outcome

e.g. For power setting, C:

At low C, there is 2.8425g of "bullets" remaining

At high C, the average drops to 0.69g

There is a decrease of 2.1525 which is the total effect

by plotting a graph using the averages for each factor:

Fig. 1: Effect of factors on the amount of "bullets" (Full Factorial)

We can rank the factors based on their effect on the amount of 'bullets', from largest to smallest, by comparing the gradients📉📈

From Fig. 1,

Power setting has the steepest gradient, as well as the largest effect🏆

followed by microwaving time⏲️

and lastly, diameter of the bowl🥣

Giving us the ranking🏅:

C > B > A

Now we're not done yet, with Full Factorial design, we can also find out how the factors interact with one another🫱🏼‍🫲🏼

Here, let me show you how:

Let's say we want to examine the interaction between A and B,

we start by calculating the effect of A at low and high levels of B, then using the averages,

we can plot a graph similar to before, where the gradient is the effect ⤵️

From the graph in Fig. 2, since the gradients are both different (one is positive, one is negative),

it tells us that there is significant interaction between factors A and B☝🏼🤓

Now we do the same for A and C:

This time, both lines are nearly parallel with just a slight difference in gradients,

meaning there is still interaction between A and C, but it is small🤏🏼

If the lines were parallel, that would mean no interaction at all!🤯

Lastly, for factors B and C:

Over here, the gradients are both negative and of different values, hence there is also significant interaction between B and C🫱🏼‍🫲🏼

Now, we can summarize our results from the Full Factorial data analysis📊:

Starting with the effect of single factors,

All 3 factors reduce the number of "bullets" when set to high
Power setting🥇 has the biggest impact, followed by microwaving time🥈, and finally bowl diameter🥉

followed by the interaction effects,

A x B (🥣 x ⏲️): opposite gradients = significant interaction
A x C (🥣 x 🔥): almost parallel = small interaction
B x C (⏲️ x 🔥): different values in gradients = significant interaction

and in conclusion,

to achieve the least amount of "bullets" in your bowl of popcorn🍿, you'd want to leave it in the microwave for a longer time, at a higher power setting, and preferably with a bigger bowl(though it doesn’t make as much of a difference🫤)

..with that, we are done with Full Factorial design✅

Although it provides us with a thorough analysis of all factor combinations🔄️, it can become impractical as the number of runs increase exponentially with more factors⤴️, leading to increased time and cost😢💸

Even with just 8 runs, if instead of microwaving popcorn, we had to build a prototype, it would also take way too long😫

which brings us to..Fractional Factorial design➗

This time, 4 runs are selected from the Full Factorial design based on statistical orthogonality.

In the context of this experiment,

a design where all factors occur the same number of times, at both ➕ and ➖,

is said to be orthogonal.

which gives us these 4 runs:

Table 2: Amount of "bullets" for each run (Fractional Factorial)

Run Order	A	B	C	Bullets (grams)
1	➕	➖	➖	3.75
2	➖	➕	➖	2.75
3	➖	➖	➕	0.74
6	➕	➕	➕	0.32

As shown in Table 2, the high and low levels of each factor are tested twice✌🏼

The next few steps for Fractional Factorial are the same as Full Factorial, just that we have fewer values this time:

from these calculations, we can plot another graph📉📈:

Fig. 4: Effect of factors on the amount of "bullets" (Fractional Factorial)

now, let us rank the factors by comparing the gradients:

From Fig. 4,

Power setting takes the lead yet again with the largest effect & steepest gradient📉,

followed by microwaving time⏱️, and lastly

diameter of the bowl, with the gentlest gradient🤏🏼

..and that's all for Fractional Factorial design, to summarize the results:

For the effect of single factors,

Similar to Full Factorial, the ranking is:🔥>⏲️>🥣
When B⏲️ and C🔥 are set to high, the number of "bullets" decreases
However, when A🥣 is set to high, there are more "bullets" remaining

Using Fractional Factorial design, the best way to reduce the number of "bullets" is to leave it in the microwave for a longer period, at a higher setting, using a smaller bowl...🤔

The difference in results compared to Full Factorial tells us that Fractional Factorial design is less comprehensive because, though it is more efficient⌛& resource-effective🤑, it carries the risk of missing important information😧 Which is why we ended up with a different conclusion🤯

onto the next part, Practical 3!😆

Practical 3 DOE

For this practical, we used both Full and Fractional Factorial designs

to run experiments using a catapult🟤↗️

The goal was to determine how arm length, projectile weight,

and stop angle affects the distance traveled by the ball🤔

This time, we had to replicate each run 8 times, which meant a total of 64 runs😵

and at the end, we had a group challenge😮

Using the data we had collected, each group had 8 tries to hit 5 targets

our group went and knocked down 3 targets with 5 shots🎯, leaving us with 3 more attempts

and the last 2 targets were surprisingly aligned from a certain angle..

so of course we tried hitting both at once✌🏼, here's what happened:

https://video.wixstatic.com/video/7af17b_e80e99f3140c4ed69b36bdff485a81a3/1080p/mp4/file.mp4

..anyways, time for another learning reflection🙃

more pictures from the practical here👈🏼

Learning Reflection

c'mon i can do this

Learning about DOE

Initially during tutorial lessons, I was completely lost because there were so many graphs and charts to look at, that I think my brain just gave up understanding halfway😖

"why's there so many numbers"

"what's that graph for"

"what's orthogonal"

So many questions that left me confused,

looking back now, maybe I should've paid more attention to the lessons😓

⏩Fast forward to after mid-term break,

we had to do pre-practical work for our next practical about DOE.

I start panicking because I didn't understand anything in the DOE template provided,

so I decided to look through the DOE lesson slides multiple times.

Finally, I understand it now

the lesson slides were really useful in helping me understand what DOE is,

as well as the different steps for Full and Fractional Factorial design.

On top of that, I realized that there are certain trade-offs when it comes to using Fractional Factorial design. Since it makes use of less data,

it makes sense that there is a possibility of missing information in the results.

Next, I begin to apply what I've learnt in the DOE practical.

Using DOE in Practical 3

With the data that we had collected, we were able to identify the optimum settings

to use for our catapult during the group challenge,

and we ended up hitting 3/5 targets with 3 shots remaining.

I believe that we could have hit all the targets if we hadn't tried to go for the 2-in-1😬,

but overall it was still a fun experience and we got to apply what we learnt about DOE,

such as how to determine the effects of single factors.

What lies ahead

I hope to continue learning more about the topic of DOE,

especially since there are more than just 2 designs.

e.g. Response Surface Design, typically used in the latter stages of experimentations,

requires at least 3 levels for each factor, meaning instead of just high and low,

it could be high, medium and low🤯

I also look forward to applying what I've learnt in the future when I work on my capstone project, or even in my upcoming internship where I'll be supporting activities such as optimisation and development of projects to improve the unit process.