How we measure.
If you are going to use our answer to move a budget, you should know why it holds up. These are the principles every Yardstick campaign follows. The full method is something we walk your team through on a call.
A taste test, not a split test.
You could give half your engineers one agent and half the other, then compare. But someone would have to define "better code" in advance, and half your team would spend the study with the weaker tool. Instead, every engineer sees both results side by side and judges the code directly, the way a blind taste test works.
Why the answer holds up.
Blind and fair
Both setups start from the same code, with the same instructions and time limit. Results appear as A and B, with nothing to tell them apart except the work itself.
Rules fixed before the first comparison
What the campaign is testing and what would count as an answer are settled before any results exist. The verdict comes once, at the end, so nobody can stop early when the numbers look good.
The test checks itself
Built-in checks watch the measurement as well as the agents. If something about a campaign is off, Yardstick flags it before anyone trusts the result.
Honest about "no difference"
Often the answer is that the cheaper setup is no worse, and that is a real answer worth acting on. If a campaign can't reach an answer, it says inconclusive rather than guessing.
Engineers are the judges, not the subjects.
Engineers' picks are used to compare setups and to check that the measurement is working. They are never used to score or rank the people who made them.
Bring us your hardest objection.
If you have run experiments on engineering teams before, you probably have a specific reason to think this is harder than we've made it sound. Tell us, and we'll walk your team through the full method on a call.