Test Design
Change more than one thing in a test without losing track of which one helped: every combination of your settings, in a shuffled order you can repeat.
Inputs
One row for each thing you’ll change, such as Compression with settings0.25 in, 0.5 in. Up to 4 things, 6 settings each, and 96 runs in all.
More runs of each show how much a setting varies on its own.
Any whole number. Write it down: the same number always gives the same order.
Results
Runs in all
—
Combinations
—
The run order
Run them from the top. The result column is for you to fill in.
Repeat each combination
Every combination of the settings you are changing, as a block, in the shuffled order the table gives. Adding a factor multiplies the blocks rather than adding to them, which is why three things with three settings each is already twenty-seven runs. Repeating each combination multiplies them again, and that is the price of knowing how much a run varies on its own.
How this is calculated
Trying every combination is called a full factorial design. With two settings of compression and two of speed there are 2 × 2 = 4 combinations, so each setting of one thing is tried with each setting of the other. That’s what lets you tell their effects apart.
combinations = settings of the first × settings of the second × … runs = combinations × runs of eachWhy shuffle
Things drift during a session: the battery drains, motors warm up, a tile wears. If you ran every run of one setting first, that drift would look like a difference between settings. Shuffling spreads it across all of them. The seed makes the shuffle repeatable, so the order on your notebook page can be checked.
What this doesn’t cover
It doesn’t say which setting is better. Record a result for each run, then compare the settings with Compare Test Versions or Descriptive Statistics.
Make a notebook page
Turns this calculation into a dated engineering notebook page: the setup, data, results, figures, working and sources, numbered and cited. You add the goal, the decision and the reflection in your own words, plus photos and sketches. Print it, paste it into Google Docs, or download a Word file or slides.
Save this run, and compare
Keeps what is on screen so you can change something and see both sides of the change. Saved in this browser only, never uploaded.
Sources & assumptions
No VEX figures are used. The factors, settings, repeats and seed areyours; the order is a seeded shuffle of every combination.
Where this goes next
Explore further on the full mapWhere the answers on this page can be used, and which tools could have given you the numbers it asks for.
Comes after
A starting point. Nothing on the site needs to come first.
You are here
Test Design
a number carries across the next question it raises