← All tools

PID Comparison

Whether one tune is really better than another, or whether the difference is the same controller having a good day.

Inputs

What you measured

Whatever you measured on each run. Keep it the same for both tunes.

Seconds, degrees, inches: whatever fits.

The tunes

Recorded so the export says what was compared.

The runs

One row per pair of runs. They do not have to be paired in any real sense (the columns are compared as two groups), so leave a cell blank if you have more runs of one tune than the other.

1
2
3
4
5
6

Results

Is the difference real?

—

Tune A averaged

—

Tune B averaged

—

Difference

—

95% interval on that difference

—

Size of the change

—× the scatter

Which is more consistent

—

Runs each to settle it

—

How far the interval sits from zero—

past 1.0 the interval clears zero entirely

Show the numbers behind this chart

Every run as entered, with each tune's average and how far that run sat from it.

Add runs one at a time

Add runs one at a timeAB
2

Every run of each tune, with the average of the ones counted so far and the interval on the difference between them. While that interval covers zero, the two tunes have not been told apart, whatever the averages say. Adding runs narrows it; if it clears zero and stays clear, the difference is real.

How this is calculated

A controller on a real robot does not do the same thing twice. The battery sags, the mechanism warms up, the arm starts a few degrees off where it did last time. So a single run of each tune gives you two numbers containing both the change you made and the noise you did not, with no way to tell which is which.

Repeating is what separates them. The scatter within one tune tells you how much the robot varies on its own; the gap between the tunes has to be bigger than that to mean anything.

The interval is the answer

The averages are the least interesting output on this page. What matters is the range the true difference plausibly sits in, because that carries the uncertainty with it instead of hiding it behind a single number.

If that range sits entirely one side of zero, the difference is real at the level you tested. If it straddles zero, the honest reading is not "they are the same". It is "these runs cannot tell them apart", which is a statement about your testing rather than about the tunes.

Real is not the same as worth having

With enough runs a difference of a thousandth of a second becomes statistically real and remains completely useless. That is why the size of the change is reported separately, measured against how much the robot scatters on its own.

Below about half a scatter, the change is smaller than the run-to-run noise and no driver will feel it. Above two, it is obvious without statistics. The middle is where this page earns its keep.

Consistency often matters more than the average

A tune that settles in 0.6 s every single time is usually better than one averaging 0.55 s but ranging from 0.3 to 0.9, because autonomous routines are built on what the mechanism does reliably rather than on its best day. The spread of each tune is reported for that reason.

How many runs

The last result estimates how many runs of each tune it would take to settle a difference of the size you are seeing. It is an approximation and it assumes the difference you measured is close to the real one, so treat it as an order of magnitude: the useful distinction is between "three more runs" and "eighty".

What this cannot see

  • Order effects. If you ran all of A and then all of B, anything that drifted over the session (a draining battery, a warming motor) is now sitting inside your result wearing the label of a tuning change. Alternate them.
  • Whether you measured the right thing. A tune can settle faster and overshoot more. One number per run only sees one of those.
  • Anything about kP, kI or kD. The constants are recorded for the export and are not used in the arithmetic. This compares outcomes, not settings.

Make a notebook page

Turns this calculation into a dated engineering notebook page: the setup, data, results, figures, working and sources, numbered and cited. You add the goal, the decision and the reflection in your own words, plus photos and sketches. Print it, paste it into Google Docs, or download a Word file or slides.

My notebook

Notebook page

More
Open My notebook

    This page

    Our own layouts

    A layout remembers how a page is set up: its type, its system, which parts print and which of your tables it uses. It never remembers a word, a number or a photo.

    Your team kept on this device
    Add someone to the team list
    Your words never written for you

    These are the parts a judge or teacher reads to see your thinking, so the page leaves them to you. Leave a box empty to get lined space for writing by hand. Start lines with "- " for a list. The notes under each box only count and point things out; they never change what you wrote.

    Rough notes

    Rough notes

    Rough notes

    Rough notes

    Rough notes

    Rough notes

    Rough notes

    Rough notes

    Rough notes

    The word counts can be measured against a target your team sets on My notebook. The reading level is Flesch-Kincaid grade level = 0.39 × (words ÷ sentences) + 11.8 × (syllables ÷ words) − 15.59. Syllables are counted by vowel groups, so the grade is an estimate of the school year the writing reads at. It says nothing about whether the writing is any good.

    One part at a time, with a clock. At the end of practice, while you remember it.
    How you tested your details

    Fill in what applies. These print as a small table under the method.

    Before and after

    Compare this calculation with a run you saved earlier on this tool (under “Saved runs”). The page shows what changed and by how much.

    A line through the data from your recorded table

    Pick two measured columns from the same table. The page plots them and draws the least-squares line, only across the range you measured.

    Robot log a CSV from the robot's SD card

    Load a CSV your robot saved. Pick the time column and up to three columns to plot, and type their units. Only those columns go on the page; the file stays on this device.

    Two versions' trials your data, compared

    Pick a column of recorded trials for each version, from this page or another saved page. The page shows both averages and a 95% range for the difference. The numbers are copied in when you pick them, so later changes to the other page don't change this one.

    Options compared your scores

    Score each option 1 to 5 on each criterion, and weight each criterion 1 to 5 by how much it matters. The page only multiplies and adds.

    Photos and sketches stay on this device

    Up to 4. Paste a screenshot with Ctrl+V, drop pictures here, or pick them. Photos are shrunk before they’re kept, and an unsaved page keeps them in its draft. Use Mark up to draw arrows, labels, dimensions and numbered callouts on a photo. Mark one photo Before and one After to print them side by side.

      Test plan written before testing

      Write this before you test: the question, what you expect with a number, and what result would make you drop the idea.

        Build steps our build

        One row per step, in order, so someone could rebuild from this page.

          Parts used our parts list

          Lengths in holes are also shown in inches between the end holes. Weights are only added up where you type them.

            Plan the cuts: packs every piece of one part (sizes in holes, or inches on the hole grid) onto whole sticks of the stock length you type.

            Ways it could fail our scores

            List the ways this could fail. Score how likely (1 to 5) and how bad (1 to 5); the page multiplies them and puts the biggest risks first.

              Designed against built our measurements

              What the CAD or the plan said, and what you measured on the robot. The tolerance is yours.

                Changes from the last version our changes

                What changed since the last version, why, and which result it was meant to move.

                  That page’s settings, results and first photo are copied in and set beside this page’s. The result each change should move is looked up on both pages.

                  Tasks our plan

                  What needs doing, who, and by when. Open tasks carry to the next meeting page until someone ticks them done.

                    The pages behind this idea our evidence

                    The pages in this notebook that show the idea being tried, measured or changed. A judge reading the claim can turn straight to them.

                      Time spent our hours

                      Who worked on this and for how long.

                        Session log what we did, as we did it

                        Notes taken during a session. The time comes from the clock when the note was taken; every word is yours.

                          Size limit does it fit?

                          Type the starting size limit from the current game manual, and the robot's size as designed or measured, in inches. The site doesn't store the limit, because it changes with the game.

                          What broke from your breakage log

                          The breakage log is on My notebook. Copy it onto this page as a table, as it stands now; later entries don't change this page.

                          CAD model which version this page is about

                          Paste a link to the model (Onshape, Fusion or anything else) and the name you use for this version. For Onshape, the page reads from the link whether it points at a saved version or a workspace that keeps changing. Nothing is fetched.

                          Your code
                          Comments notes for the team, not the page

                          A teammate or coach can leave a note here. Comments stay beside the page, travel with it in exports and links, and don't print unless you tick them under Include.

                            Sources you used besides this tool

                            Videos, other teams, manuals, people. The tool’s own sources are cited for you.

                              Sign-off
                              Include

                              Press a ? to see what an option is for, or read them all on How the notebook tools work.

                              What “The calculator’s figures” is for

                              The calculator’s own diagrams and charts, numbered as figures and drawn from your numbers. Judges look for pictures that explain a design.

                              What “How it was calculated” is for

                              The formulas and steps the calculator used, so a reader can check the arithmetic for themselves.

                              What “The calculator’s warnings” is for

                              Things the calculator flagged, such as a motor near stall. They’re quoted and credited to the calculator, so they never read as your conclusion.

                              What “The calculator’s notes” is for

                              The calculator’s plain-language notes on the result. Off at first, because they read like conclusions, and the conclusions on a page should be yours.

                              What “Lined space” is for

                              Ruled lines under any part you haven’t written yet, so a printed page can be finished by hand.

                              What “Writing space as” is for

                              The blank space for handwriting as ruled lines, a dot grid or quarter-inch squares. Dots and squares suit sketches and hand-drawn graphs.

                              What “A summary under recorded data” is for

                              The number of readings, the average, the spread, the lowest and the highest under each table you recorded, worked out from your numbers. It shows every run counted, not just the best one.

                              What “Words used” is for

                              A small box with the site’s one-line definitions of glossary terms the calculator used, for readers who don’t know them. It never looks at your own writing.

                              What “Robot at the time” is for

                              The Robot Profile as it was when the page was first saved, so weeks later you can tell which version of the robot a test was run on.

                              What “Long tables in full” is for

                              Long calculated tables printed complete on an extra page, instead of every few rows on the page itself.

                              What “Fit on one sheet” is for

                              Smaller type and shorter tables, aiming for one printed sheet. Use it when every entry in your notebook should be a single page.

                              What “Wider left margin” is for

                              A wider left edge, so the page can be hole-punched or bound without losing any words.

                              What “Larger type, for reading aloud” is for

                              Bigger print with more room between lines, for the team member who reads the page to a judge. It overrides Fit on one sheet.

                              What “Our rubric scores” is for

                              Your own scores against the rubric, on a sheet of their own, for checking a page before a judge does.

                              What “When this page was first saved” is for

                              The day the page was first saved, printed beside its date. It shows the notebook was kept while the work happened.

                              What “Teammates’ comments” is for

                              Prints the comments teammates left on this page at the end, marked as not part of the page. Off at first, because comments are notes for the team, not for judges.

                              What “Show every table and block” is for

                              Shows every table and block in the editor, not only the ones this type of page usually uses. It changes the editor, not the printed page.

                              Rubric check

                                This checks what is on the page, not how good it is. Clear, specific writing in your own words is what earns the top marks.

                                Preview, letter size

                                Write it now

                                Save this run, and compare

                                Keeps what is on screen so you can change something and see both sides of the change. Saved in this browser only, never uploaded.

                                Sources & assumptions

                                No VEX data. Every number comes from runs you measured yourself.

                                The comparison uses Welch's form of the two-sample interval, which does not assume both tunes scatter by the same amount, since a change that moves the average usually moves the consistency too. Critical values come from a t table rather than the normal 1.96: at five runs a side the normal figure is about a quarter too confident, and five a side is what people actually run.

                                The runs-needed estimate is the standard approximation for 80% power at 95% confidence, and is an approximation rather than a guarantee.

                                  Every source the site uses, and when it was checked

                                  Where this goes next

                                  Explore further on the full map

                                  Where the answers on this page can be used, and which tools could have given you the numbers it asks for.

                                  Comes after

                                  PID Error Analyzergives you your trial resultsSample-Size Planner

                                  You are here

                                  PID Comparison

                                  Where your answer goes

                                  Compare Test Versionscheck the better tune also makes the routine succeed more oftenuses your trial results againPID Error Analyzerrecord a run with the winning gains and look for what to tune nextuses your trial results againAutonomous Valuesee if a more reliable routine is now worth more pointsSample-Size Plannerfind out how many runs each tune needs before trusting the winner

                                  a number carries across the next question it raises

                                  Keyboard shortcuts

                                  Anywhere

                                  Ctrl K or /
                                  Search
                                  T
                                  Switch color theme
                                  B
                                  Bigger text, for a projector
                                  F
                                  Readable font, on or off
                                  ?
                                  This list

                                  On a tool

                                  S
                                  Star or unstar it
                                  C
                                  Copy the results
                                  L
                                  Copy a link to these exact numbers
                                  P
                                  Print or save as PDF

                                  Letter keys do nothing while you're typing in a box.