After reading the title of this post, I’m sure many of you are asking “What does baseball statistics have to do with standardized tests?” The subtitle of Keith Law’s book 
Smart Baseball provides a clue to the answer: The story behind the old stats that are ruining the game, the new ones that are running it, and the right way to think about baseball.
Law’s book is an in-English explanation of the new statistics that are being used by baseball teams and increasingly the media to measure baseball players. But the book starts with an explanation of why the old statistics needed to be replaced. The bottom line: they do a lousy job of measuring the contribution each player makes to winning baseball games. So you had for nearly a century a set of stable statistics that as the subtitle says were “ruining the game” because the statistics were not aligned with the outcome every team wanted: winning.
Law argues that the adherence to traditional measures of player value leads to bad personnel decisions––what players are hired and how much they are paid––and, maybe even more importantly, how the game is played. And both reduce the chance of a team winning. Not smart!
Anyone who watches baseball today can see that better understanding of what contributes most to winning and having measures that are aligned with what contributes most to winning is fundamentally changing how the game is played. From using pitchers far differently; to how players are positioned on defense; to what hitters are trying to accomplish at the plate; to which players get to play; and which players are paid the most.
As I was reading the book it struck me this is the same reason why we at Michigan Future have turned against standardized tests as the measure of school, educator and student performance. We have been unable to find any evidence that standardized test scores are good predictors of the outcomes we want from K-12 education: improved life outcomes of students.
As we have explored previously there is compelling research that concludes that standardized tests scores are not good predictors of either college success or life outcomes. Pat Cooney wrote in that post:
The book Crossing the Finish Line used a large, nationally representative set of student data to analyze what truly predicts success in college. And what they found was that a student’s high school GPA – made up of grades given by individual teachers across four years of high school – was far more predictive of eventual college graduation than her SAT/ACT score. And the student’s GPA was predictive regardless of high school attended; whether the student went to a “good” high school or a “bad” high school, a good GPA predicted college success.
Why? Because while test scores measure a student’s ability on a narrow band of math and reading skills, GPA measures a diverse set of capacities, encompassing academic habits, content knowledge, and non-cognitive skills, exhibited day after day across four-years of high school.
Research from Northwestern economist C. Kirabo Jackson came to a similar conclusion. Using a large set of student data from North Carolina, Jackson found that a “non-cognitive” index of grades, attendance, and disciplinary records was more predictive of long-term success than test scores. And he also found that the set of teachers that were able to improve this index was an entirely different set of teachers than those that were adept at raising test scores.
The message from both cases: when we focus only on test scores, we miss the really important stuff.
Just like in baseball, the adherence to the wrong high-stakes measures in schools leads to bad design. Both curriculum and pedagogy. Both what is taught and how it is taught. Narrowing curriculum to the content-specific skills that are on the test rather than a focus on broad 6Cs skills that includes content, but far more: collaboration, communication, content, critical thinking, creativity and confidence. And making it harder for schools to move away from teachers imparting content to learning through projects.
What baseball has figured out––and so far education policymakers have not––is that you can do real harm by an adherence to traditional measures. That the right way to construct measures is by working back from the outcomes you want. The lesson Smart Baseball teaches us is if the measure does not help predict achieving the desired outcome, get rid of them and find measures that do.
