Whose Horse in the Race?

Among the many pitfalls of high stakes testing, the one that troubled me the most when I was teaching high school (in Kentucky) was misplaced accountability. My students’ performance on annual Commonwealth Accountability Tests (CATS) had real consequences for me, but not for my students. Teachers and schools were assessed, and actions taken,  based on student scores. What difference did the score make for the student? Scores weren’t used to determine advancement or placement. We couldn’t have used the scores to determine grades, the test scores weren’t released in time. Teachers and schools may have created ways to incentivize performance, but there weren’t any natural consequences to students built into the testing program. Begun in 1999, the explicit purpose of the CATS testing was to hold all schools accountable to achieve at least a pre-designated proficiency target score by 2014.

Are they going to make it? Well, if you set a goal and see you aren’t going to make it, it’s a lot simpler to change the goal than to change what you are doing. The CATS tests were completely overhauled in 2007, making it impossible to compare scores before and after that change. CATS was dropped completely for 2011. And so goes the trajectory of every major reform that moves accountability away from students. I’m reminded of a cartoon I saw when I was teaching that stuck with me. An angry father is looking at a low test score his son has brought home and says: “Son, I am very disappointed... in your teacher.” I’m sure I chuckled when I first saw it, but the effect of removing responsibility is serious -- moral -- stuff. When the community in which a child is developing says, in effect, someone else is responsible, then he will feel relieved of that obligation and his efforts will reflect that.

Is that true? Do student scores on tests correlate with the consequences of the test for the student? In a previous post I cited a recent study by ETS called Motivation Matters in which a proficiency test was given to three groups of college students each with a different consent form. The group given a consent form that said, in effect, your score on this test will affect your future employment did significantly better than the group given a consent form that said scores would be kept confidential. It makes sense: if the score matters to the student she will work harder. Of course I felt this teaching school. My job became about motivating students to do well on the test -- because there were no intrinsic reasons for students to do well. How I would have liked to merely help students see the direct relationship between their performance and their own future goal -- if only there had been such a relationship!

Most frustrating of all, the more I pushed as a teacher, the less reason my students had for taking responsibility themselves. If it was my horse in the race, why should my students worry over it? This brings me to another very recently published study. This large, well conducted study at representative colleges found what, at first blush, is a surprising relationship. Students whose parents paid more toward college had lower GPAs than those whose parents paid less. The careful and thorough analysis came to this conclusion: students who had to pay more had a horse in the race. OK, those are my words, but the conclusion boiled down to the student having a greater or lesser stake in the outcome of each class she took. Now to be fair, the study revealed a seemingly opposite relationship between graduation rates and parental investment. You can sort through the fine print in the article itself. Suffice to say I don’t think this finding undermines the conclusion that students who have a greater stake in the outcome will perform better. In my previous post I went so far as to suggest that we might more accurately think of tests as measuring motivation than as measuring skills or knowledge. The same, I think, goes for GPAs. If we look closely I think we’ll find that student GPA tracks more closely with motivation than with skills and knowledge. If only I was sufficiently motivated to do the study!