A letter to an old student

Here is part of a conversation I had with a former student - His question is in italics, my response is in normal font.  This student has a great interest in education but is probably going to go to med school upon graduation.

Hi there!
I wanted to check in and see how you are doing and how the family is. How many kids now? 9? 10? 
I also wanted to message you to ask for a bit of advice (as if I didn't ask for enough 8 years ago). This Fall I will begin teaching Biology at a high school (I cannot wait; living the dream) and I'm currently taking a course that's about Science pedagogy and education design, etc. The course has made me think a lot about my experiences with great teachers and terrible teachers, and I really want to hone it on what exactly it is that made teachers like you so effective and influential, and some like one of my college chem teachers so ineffective. So I figured you might be of some help. Do you have any tips from your experience? One thing I'm really interested in is how to create, control and encourage the right type of classroom environment. How do you maintain interest from your students? How do you gain their trust and respect so quickly? I'm spending a lot of time on lesson planning, but I could really use some help with the other aspects of teaching, and I thought you might have a few pointers.
Sorry this message is so disorganized. Don't rush with a response. I don't begin teaching for months.
Also, I wanted to mention that it would be really good to see you in person sometime soon. It's been far too long. I was thinking I could make the drive up to Glade Spring (that's where facebook says you are, is that right?) at some point if that would be convenient for you.
Hope to hear from you soon, and that all is well.
Best,

(name omitted)

Hey (name omitted)! This is just like you - you get to where I am in 1/20th the time:)  I am right where you are right now - doing some serious questioning, not so much of myself, but of the system that ... almost encourages... bad teaching. I am now beginning an education of sorts for myself - to try to do some good reading on what is 1) done right in other areas and 2) what is done wrong in so many schools around here. Me and a colleague are going to try to get published and then really become advocates for a change... oddly enough, guess who is also interested in such stuff and might even help us out? (name omitted - another former student, classmate of this one).... and now I might be able to hook you in too:) Nice circle of life thing. Maybe we should all write a book someday.

I am so proud of you for starting where you are starting. Not with the garbage of how to do lesson plans, etc... like there is some formula of how to be a good teacher... but you are thinking of what it is that actually makes a good teacher. That is a great place to start.  Step 2 would be for you to think about what makes a teacher ineffective.  This is all subjective, though - because you see the world through your own eyes and brain - however, there is still truth to be found if you look.

Here is what I think is good and bad about me as a teacher.

Bad: 1) I'm unorganized, 2) my lesson planning is kind of abysmal, 3) classroom management is my own style which involves a lenience bordering on chaotic at times, 4) I hate grading and give really poor feedback, and 5) I don't stick to schedules well

Good: 1) I like working with kids, 2) I believe in them, 3) I can see them the way they will be (you are a perfect example here - you are mr. ultra-successful now... but I never saw you as any different, just younger), 4) I can see boredom creeping in and I can switch direction in an instant to try to win them back (one moment we are talking about mitosis, then next we talk a bit about phantom limbs or something, then back to mitosis - this drives some students crazy, however - but its usually just the good students:), and 5) my classroom management is my own style which involves a lenience bordering on chaos.

Notice how all my flaws (in my mind) are in areas that most people think are critical to good teaching and all the good stuff is just basic human stuff?

They can't make you a great teacher. They can give you tools that will help. You already are a great teacher... just BE.  Just be ok with who you are - the good and the not so good (if not, they will definitely expose it:) Truly care about your students - some will need a great biology teacher, some will need to play ping pong with you. You only think I am something special because we connected. And you can't connect with someone if you don't care for them.

So find out about your students - create situations where you can just talk to them in small groups or individually (like labs) - don't make it about biology all the time. My main goal when I teach a class is that they don't hate it when they are done. I am only laying the groundwork so they can keep learning throughout their lives.  So I don't get all bent out of shape if some biology concepts don't resonate with them now.

And by god, have some fun. Never forget that those kids have to be in your class - you don't... even worse you get paid and they get nothing.

Ok - things to consider. 

1) You are young and cool and some of those HS girls will throw themselves at you. Society does not look kindly upon that even though the difference between you and you student's ages is probably less than many of your students parents. be aware, be honest. For instance, it would have been trickier for me to have helped you if you had been female. I've done it but its a precarious line.  Basically don't abuse or take advantage of your students... keep a clean conscience.  Step back if you need to.

2) Many teachers are burnt out because the profession is hard.. and if you aren't enjoying it... then its going to beat you down. Don't hang around those teachers too much... suck the good ideas from them and discard the rest.

3) Continually evaluate yourself and your class - every day - ask yourself what went right and what went wrong

4) DON'T JUDGE YOURSELF. You should absolutely suck this first year. embrace it, laugh at it... let it make you a little bit better next year. And when you have 17 years in like I do, you will only just kind of suck... and you will still embrace it and still laugh at it.

5) Accept yourself.  The good and the bad.  Students respond to genuine... and they are repelled by fakeness and hypocrisy.  It takes a long time to be ok with who you are - just make that the goal and keep striving for it.  And hey, if the teaching thing doesn't work out - at least you have accepted yourself:)

Ok, that should be good for now - I would love to talk with you anytime - its difficult to get away, but if you can make it to Glade anytime - you are more than welcome here! Keep in touch - as you do lesson plans, feel free to shoot me questions about what I do for certain topics - shoot me an email and I will share my google folder that has all my biology stuff in it

Good luck, old friend. Keep me posted

Steve

Preparing for Tests Undermines Long-Term Growth

The pressure is on in Virginia schools to prepare for the high stakes "Standards of Learning" tests that are coming up in a couple months. The teach-to-the-test mentality adopted by many (or most) teachers not only makes schooling unpleasant, it likely stunts the long-term academic growth of our students. I wrote a personal anecdote relating the experience of my 10 year old daughter intending to include it in this post. I've decided to save that, and many other things I've written, for a book on motivation. More about that later. In the meantime I am counting on you to contribute your own anecdotes, interpretations, and/or comments to keep this conversation interesting!

In my last post I cited a recently published study of German students who were followed from fifth to tenth grade in order to identify the factors that led to growth in math achievement over those five years. [http://onlinelibrary.wiley.com/doi/10.1111/cdev.12036/abstract] The researchers measured intelligence, motivation, study strategies, and math skills. They concluded that growth in math skills is predicted by motivation and study strategies, not intelligence. The results were unsurprising to the researchers, whose initial hypotheses were confirmed. My post was critical, but more about the perceived implications of the study than of the study itself. I expressed my concern that motivation may not be as distinct from intelligence as we tend to think. Educators efforts to edify young people are similarly constrained by both.


An important finding of the German study that I haven't yet mentioned will help us understand one of the ways our efforts are unwittingly sabotaged. Recall that growth in math skill was predicted by intrinsic motivation, i.e., students who scored highly for intrinsic motivation showed greater gains over several years of math tests. [Intrinsic motivation was measured by self-reported questionnaire items like "I invest a lot of effort in math because I am interested in the subject."] That was expected. What was unexpected -- and most interesting -- was that these intrinsically motivated students tended to score lower on any given year. They were outperformed in the short-term by extrinsically motivated students, but they showed greater gains in the long-term. [Extrinsic motivation was measured by self-reported questionnaire items like "In math I worked hard because I wanted to get good grades."] I will quote the analysis of the researchers at length:

Students with high intrinsic motivation are less concerned about how well they perform on upcoming achievement tests. Accordingly, although intrinsic motivation should provide long-term benefits, such a non instrumental approach to learning may not add much to current performance. As for deep, elaborative learning strategies, previous studies indicated that elaborative learning may not be an efficient means of dealing with an upcoming achievement test because semantic elaboration is a relatively slow learning process and therefore costly if time is limited.

Learning is a slow, gradual, and ultimately personal process of who we become. Students who respond to short-term extrinsic motivators like grades and scores on tests may be favored by school systems under pressure to perform on annual exams. Students who are motivated to make sense of material according to their own interests may not only underperform on annual exams, but they may be disfavored by the school systems which, paradoxically, will undermine the interests of everyone in the long term.

Motivation vs Intelligence

Quick armchair experiment: Imagine you have a 12 year old boy named Johnny. You have requested a conference with his 7th grade math teacher, Mr. Smith. When you ask Mr. Smith to explain Johnny’s relatively low report card grade for math, he says that Johnny is less intelligent than others in the class. How does that make you feel?

Now imagine a nearly identical scenario with only 1 difference: substitute “motivated” for “intelligent” so that Mr. Smith’s explanation for the low grade is that Johnny is less motivated than others in the class. How does that make you feel?

If you are like me you have had plenty of parents inquire about kids’ grades, and so have already put yourself in the place of this imaginary parent. And you likely never have nor ever will attribute poor performance to lack of intelligence -- that just seems cruel! But you probably have offered something like the second explanation: that little Johnny just hasn’t put in the effort required. If he just works harder his grades will improve. That sounds so much nicer.

The difference between the two explanations -- the reason the first seems cruel and the second nice -- may seem obvious, but only given an intuitive model of intelligence and motivation that I think needs revision. According to the model intelligence describes something about a person’s biology while motivation describes something about a person’s spirit. With this model intelligence limits a person, something like a person’s height, except even more broad in the scope of things a person won’t be able to do. Motivation, on the other hand, potentially expands the person. It feels like the immaterial medium through which we transcend our physical boundaries.

I'd like to point out two good reasons to get over this model. First, any conceptual model that relies on non-natural phenomena puts itself beyond the scope of science. What research program can you design to look for the sources of motivation if it is, in essence, a non-physical trait? Second, as new data emerge they must be reconciled with our models for understanding the underlying processes at work. Brain imaging studies, for example, may produce data about motivation that need explanation. Models built on non-physical attributes will not provide the foundation needed to accommodate these data and something will be forfeited -- either the best interpretation of the data or the model that we have come to rely on.

Most of the non-scientific community is ready to forfeit the best interpretation of the data. Social-scientific studies, particularly in education, also seem determined to follow human intuition and attempt to fit the data with outdated models of human agency. The rest of this post will illustrate with a case in point.

A small buzz began to hum in the blogosphere following the December publication of a new study about the sources of improvement in understanding math. [see http://onlinelibrary.wiley.com/doi/10.1111/cdev.12036/abstract] The website of the British Pschological Society led with: "Being motivated could be of greater importance than intelligence when it comes to academic achievement, new research has suggested." The post continued, "the study found a young person's growth in mathematics achievement is driven more by their determination and how they study rather than how smart they are." [http://www.bps.org.uk/news/motivation-more-important-intelligence]

US News reported that "Innate intelligence -- as defined by IQ tests -- may provide a head start, but it's learning skills and determination that ultimately add up to success, according to the new research." [http://health.usnews.com/health-news/news/articles/2012/12/20/motivation-and-study-not-iq-are-keys-to-kids-math-success]

Most tellingly,Time.com posted, "You don’t have to be born with math skills; solving problems is a matter of studying and motivation." This is a welcome finding, the author contends, because "...the findings provide reassuring confirmation that academic success is not governed by a student’s cognitive abilities alone." [emphasis mine] [http://healthland.time.com/2012/12/26/motivation-not-iq-matters-most-for-learning-new-math-skills/]

Thank goodness, these posts imply, math performance depends on something besides the brain -- even if ultimately the brain has to do the work. You are born with the brain (or at least the instructions for growing it) but motivation comes from somewhere else. Someone demanding a fully naturalistic account of math performance might like to know where to find all the non-cognitive human math abilities.

The appetite for this kind of news is abundant and calls out for an explanation. One reason is that it seems to support the intuitive model that motivation and intelligence are different. This feeds into the desire that parents and teachers feel to help young people. It's hard to imagine ways to intervene to increase intelligence but easier to imagine ways to intervene to increase motivation. It likely goes even deeper, though, to the sense of personal efficacy that correlates with perception of control. When we feel that something is outside of our control -- then it is outside of our control. Conversely, to the extent we are convinced that something is within our control, we actually gain a concomitant measure of control over it.

The intuitive model of human motivation requires that it works something like Dumbo's magic feather. Both posit the need for magical source of power. The point of the story, though, was that the feather wasn't actually magic. And for Dumbo to think that flying depended on magic made his flying susceptible to what nature would ultimately reveal -- that the feather was just a feather, and what mattered was Dumbo's belief about himself.

The intuition about motivation is like Dumbo's belief about the feather. It may have served to foster a sense of self-efficacy, but we have to grow past the belief that it's source is other-worldly if our powers are to continue. Continued research in neuroscience will likely reveal that motivation is a cognitive ability just like intelligence. It is built into the brain, varies from person to person, and worst of all, levels of motivation probably remain stable within one person. I think it is likely we will learn we have no more power to intervene over intrinsic motivation than over intelligence.

That intelligence and motivation are both real and physically mediated in the brain does not need to imply that our involvement as teachers and parents is wasted. Understanding how someone processes information and how they are self-motivated gives us the tools -- perhaps the only tools -- for appropriate intervention. Discussing the intelligence of the children we work with should not be any more taboo than discussing relative levels of motivation.

Lance, Grit, and Dope(amine)

Put aside your grief for a moment and you can likely bring to mind an iconic image of Lance Armstrong driving himself toward the finish line of yet another long and grueling race. We are compelled by great athletic achievement because it shows us what is possible. Your sympathy for Armstrong may be low right now, but only obstinate denial will keep you from remembering that there was a point when he was bald, bedridden, and facing death. You may decide to cast your stones now, but who had the stones then? The greatness of a successful athlete is not due to having a true moral compass -- as we might like.  What is required of our athletes is the true grit to accomplish personal goals. Lance doped because that’s what it took to win the Tour De France. The real difference between Lance and the other riders on tour wasn’t the dope, it was the dopamine -- a chemical in the brain now implicated in motivation.

Let me suggest that what we all want -- what we need, really, to participate as persons in the world -- is the leverage to move ourselves toward what is of value. The term for that leverage is agency. Further, despite wishful thinking to the contrary, agency isn’t a birthright. We have to work for it ourselves and demand it of others. Agency is the parent of twins: freedom and responsibility. We get traction for our agency from grit. Athletes show us what grit looks like so that we can use it, in some measure, for ourselves.

Athletic pursuit operates in a specially designed microcosm so that goals are readily defined and success in easily measured. All the work that athletes put in -- all the planning, training, dieting, managing, and competing -- is rationalized by a set of explicit performance goals that generalize to “winning.” To that extent you cannot dismiss the importance of winning because it defines the index used to measure the success of all that work. We still mean it when we say “winning isn’t everything” though, because the pursuit of winning is itself in service to the greater purpose of athletics: to showcase our capacity to set and meet goals -- to prove our agency in the world.

One reason you are grief-stricken over the Lance Armstrong scandal is because the means he used to reach his goals were against the rules. Rules, however, like all other defining aspects of sports contests, are a means for establishing what competitors have to do in order to win. To a non-competitor the rule may look simple: “don’t take EPO.” That’s about as simple in bike racing as the rule: “don’t foul” is in basketball. Just like the real goal for basketball players becomes to make the greatest possible use of personal contact without getting called for a foul, the real goal for endurance cyclists is to get the most possible red blood cells without failing a drug test. Although those outside the sport are upset, my guess is that most of Armstrong’s immediate competitors at the time accepted that Lance had won at the game they had all tacitly agreed to play. Unspoken rules among athletes are subject to interpretation and change -- and of course pressure from outside. I suspect that is what is going on now.

Lance Armstrong won an unprecedented 7 Tours De France. Two questions emerge for me now that he is poised to confess his use of performance enhancing drugs. First, should these drugs be against the rules? He had to successfully take and cover up PEDs in secret for his whole career. Think what he could have accomplished if he could have used PEDs to maximum effect with the full benefit of doctors and specialists working openly! That interesting question is not the subject of this post, however. The question I’d like to consider, and I’m going to assume that most of Armstrong’s main rivals were not above doping themselves, is what set Armstrong apart? For that matter, what sets any of us apart? Even if you remain indignant and hold that Armstrong deserves no credit, will you at least concede that he was highly motivated? Lance saw where he wanted to be even when he was a long way from it. And he got there, taking large risks along the way. That’s motivation, right? I think it’s fair to say that Armstrong was sufficiently motivated to do what it took to win the Tour De France.

If you ever took a psychology class you can probably bring to mind the experiments in which rats greedily press a lever to get a morsel of food. A very general finding was that the neurotransmitter dopamine was released when the rats got the food. Dopamine is known to be integral to human experience of pleasure as well. New research continues to refine our understanding of the role of dopamine, however, and it goes a lot deeper than mediating our desires for food and sex. According to a recent study by John Salamone and Mercè Correa [The Mysterious Motivational Functions of Mesolimbic Dopamine. Neuron, 2012; 76 (3): 470 DOI:10.1016/j.neuron.2012.10.021] the neurons activated by dopamine are tasked with “behavioral activation, exertion of effort, approach behavior, sustained task engagement, Pavlovian processes, and instrumental learning.” If that sounds a little technical, I can paraphrase. Dopamine is not the chemical of reward, washing over us after the work is done, it’s the chemical of motivation -- jacking us up to do the work required. The cells that kick in with dopamine cause us to get going, to work hard, to go toward the thing we want, to stick with it, to do more of the things we find working, and to do more of the things that help us to achieve our goals. Experimental evidence suggests that dopamine circuits don’t cause us to find the destination more rewarding, they increase our willingness to do more work en route to getting there.

The neurochemistry of dopamine is important because we seek interventions for people suffering from a range of conditions from depression to addiction. At root these conditions are about motivation -- too little in the case of depression and too much in the case of addiction. A stark and unsettling implication of this research is that one could literally take the measure of the brain chemistry of any group of individuals and quantify a range of motivational states. Some people are more motivated than others. I’m sure one could similarly take the measure of a single individual over time and find a similar range of motivational states. We are more motivated at some times than at others. Biological determinism rears its ugly head. Are we just puppets pulled by the firings of nerve cells that have too much or too little of the chemical needed for motivation?

That is not the kind of question broached in the standard fare of education literature. Michael Horn authored the book Disrupting Class to describe his system for motivating young learners. In his blog on Forbes he posts this synopsis with the title: Building Motivation, Instilling Grit.” [See: http://www.forbes.com/sites/michaelhorn/2013/01/10/building-motivation-instilling-grit-the-necessity-of-mastery-based-digital-learning/] In it he argues that technology now provides the means for all students to work toward mastery individually at the appropriate level and pace. All that is required is the intrinsic motivation to get there. I was right with him until he went on to describe (as the title suggests) how we should instill the grit needed into seemingly unmotivated students. How, I wanted to know, can you “instill” something and it also be intrinsic to the person? Never mind the chemistry -- whose motivation are we talking about here? If it is intrinsic, it has to be the student’s own. If the educator does the work, it was the educator who was motivated. Can a teacher work to get students to do something? Of course! Call and order pizzas for students who finish. But if that is what moved the student it was extrinsic. Remove the promise of a pizza and the student will stop working.

And before you correct me, I realize that in a real classroom the dynamic is considerably more complicated. Maybe I ordered pizzas just because I’m a nice guy and I like my students. Students like me because I’m a nice guy and because I like them. Kids do the assignment, therefore, because they like me, even if I don’t always have a pizza waiting for them. Psychoanalyzing a group of kids is fraught with peril in any case -- you can’t always know why they do what they do! The point I hoped to make is more definitional. Behavior that is motivated intrinsically doesn’t depend on someone else making you do it.

More perplexing still, Horn insists that all young people are equally motivated to pursue what is of value. The reason some students appear more motivated than others, he says,  is that schools haven’t made the case that all students should buy what schools are selling. Few things are as aggravating to me as the assumption that differences in student performance are best explained by differences in the effort of teachers and schools. That implies that improving the performance of students requires teachers and schools to be more motivated. Grit doesn’t work that way. Grit is the real traction given a person to pursue what is of value -- to him. It takes work: stepping toward the problem, confronting it, and sticking with it until one has achieved the thing of value. No one can do it for you.

That the chemical transactions in your brain likely differ from those in other brains might cause you to view your own agency as limited or absent. With that view you are as doomed as the student waiting to be sufficiently motivated by his teacher. In the meantime I suggest you continue to watch and revere those who do not shrink back in the face of apparent obstacles to their own success.

Motivation and Testing

If The Graduate was remade for today I’d have the “one word” for Dustin Hoffman’s character: testing. If it isn’t already, testing will soon be as ubiquitous as plastics. If your interest is in making money you should consider investing in this high growth industry. Like plastics, though, testing clutters the landscape like so much cheap refuse.

I’m a teacher-educator at Emory & Henry College, a small private liberal arts college in SW Virginia. In order to enter our program students must take and pass the Praxis I test in mathematics, reading, and writing (or it’s equivalent.)  For many this is just another $135 hoop. For a few it is a major roadblock. And then there are those for whom the Praxis I is a whole series of $135 stepping stones. These students are making the yacht payments for the well-placed at ETS.

OK, so I’m revealing my bias against testing as big business. So much of our schooling is centered on testing, let’s hope it isn’t all bad. I am prepared to at least provisionally accept that well constructed tests measure something of value. I think we are wrong about what tests measure, though, and I will use the data from a recent study by (who else?) ETS to prove it.

The explicit assumption most of us make is that tests measure knowledge and/or skills. Many criticisms may be leveled at tests, but they would assert something like this test isn’t good because it doesn’t accurately measure what the test-taker knows or can do. In other words we take for granted that tests --  by definition -- are at least meant to measure knowledge and skills. For example let’s say Johnny takes the Praxis I and gets a combined score (across the 3 subtests) of 532. The Virginia legislature has deemed that this score indicates Johnny has the minimal knowledge and skills to enter into a teacher preparation program. Let’s set aside (for now) the problem that if Johnny had, because of some very minor perturbation, scored a minutely different 531, his score would have indicated he did NOT have the minimal knowledge and skills to enter into a teacher preparation program. Apart from that, I’m guessing that my description so far is uncontroversial. I can probably even safely add that the reason that Johnny scored better than Jerry -- who took the same test on the same day and scored a distinctly lower 515 -- is because Johnny knows and/or can do more than Jerry. Getting agreement beyond this is going to be more difficult -- but let’s try. And fair warning: I’m trying to describe how most people think about this so I can then criticize it and suggest a better alternative.

Why is it that Johnny knows and/or can do more than Jerry? Some will say Johnny was better prepared by his prior experiences, some will say Johnny has higher native ability (at least for the sort of knowledge and skills measured on the Praxis test), and I think most will agree to a generic statement like: a combination of differences in ability and experiences led to the difference in knowledge and/or skills between Johnny and Jerry. I think this level of generality makes this statement as strong as it can be while keeping as many subscribers as possible. There may be few strict “nativists” who claim that all the difference can be explained by innate ability, and there may be a few strict developmentalists who claim that all the difference can be explained by experience, but most people will accept that both play a role, and indeed, may only make sense taken together. And we don’t know anything else about Johnny and Jerry to prejudice us about the potential explanations of the difference.

It may be an interesting aside to consider what question would you ask first if you sought an explanation for the difference between Johnny and Jerry. I suspect it would be something like: what were the differences in childhood environments between Johnny and Jerry? Any question that smacks of biological determinism has such negative connotations that you will likely try to avoid thinking about those questions. Instead you will ask: Did the caretakers encourage reading early? Did Johnny have better teachers in school? Did Jerry’s friends distract him from his studies? Like biology, though, these environmental factors are also “deterministic.” Early childhood environments are not chosen any more that one’s genes. I suppose we feel more like we can intervene in the fluid-seeming realm of the environment. I want you to see why that feeling is fallacious. More broadly I want you to see that these explanations are so brittle and prone to being misconstrued because they are built on a false assumption about what tests really measure.

If you’ve read this far you are probably a sincere educator, so I’d like to ask you a question that might clear up the fallacy over biological determinism. Given the choice, who would you prefer to help, a child with a difficult home life or a child with a learning disability? Silly question? I know, you don’t choose who comes in the door. And I also know you are exasperated by some of what you hear about what goes on at home. And you are frustrated by the difficulties some children have -- and the time it takes them to learn what others pick up quickly. Most importantly, the life work you have chosen is to help both. You will show the child with a difficult home life the security of a consistent and supportive classroom environment. You will help the child with a learning disability develop the specific reading strategy that will help him progress. We have equal purchase to intervene for a child regardless of the source of difference in that child’s performance.

It shouldn’t be about us, though. The child with a rough home life will go to that home at the end of the day. She will always be the child of her parents -- and the parent of her own children. The child with the learning disability will go to other classes and on to places of employment where he will have to rely on his own assimilated strategies to process information. We have to be careful not to overstep the boundaries of our responsibility -- because only one person can ultimately be responsible.

That brings me back to testing. Who is ultimately responsible for the score of a test-taker? I don’t ask this to start a buck-passing game. The answer tells us something important about what is really being tested. Here is the title (and conclusion) of the study conducted by ETS that I referenced near the beginning of this post: Motivation Matters. [full title and abstract at http://edr.sagepub.com/content/41/9/352.abstract]

The test that was tested (leave it to ETS!) was the Proficiency Profile, one of 3 major assessments offered by ETS to colleges and universities to measure the “value added” by higher education. The idea of these tests generally has been to give the assessment to freshman and seniors and use the difference as a measure of the value added at the institution. Colleges who bought into the assessment -- hoping to use it for accreditation or similar purposes -- have been torpedoed by the results that have now been widely publicized and discussed in the form of the book Academically Adrift, by Richard Arum and Josipa Roksa. If ETS wants to continue to sell this test to institutions of higher education it desperately needs to explain why the results of the assessments so far show little or no value added. Why wouldn’t the gains in knowledge and skills that surely come with years of college-level work be evidenced by a test designed to measure knowledge and skills? Maybe (the test-makers at ETS think to themselves) the test-takers are not sufficiently motivated! Let’s do a study to try and prove our hypothesis!

The study compared the results of 3 randomized groups each of which was given a slightly different consent form. The control group was given a standard consent form that indicated the test results would be kept confidential and only used by the research team. The personal condition group was given the same consent form with the addition of the clause: your test scores may be released to faculty in your college or to potential employers to evaluate your academic ability. The third group with the institutional condition was given the standard consent form with the addition of this clause: your test scores will be averaged with all other students taking the test at your college. Only this average will be reported to your college. This average may be used by employers and others to evaluate the quality of instruction at your college. This may affect how your institution is viewed and therefore affect the value of your diploma.

Check it out for yourself [full article at http://edr.sagepub.com/content/41/9/352.full#sec-4] but in my view the study was well designed and carried out, and the results are robust. Rather than follow the evidence to its logical conclusion, however, the researchers clung to their initial assumptions and then bent their analysis to fit it.

The researchers found that students in the personal condition group reported higher motivation (as measured by the Student Opinion Scale -- a 10-item self-report of test-taking motivation) and scored significantly higher on the Proficiency Profile than students in the control group. Students in the institutional condition also reported higher motivation than those in the control group, and also scored higher on the Proficiency Profile. Of course the researchers (who work for ETS) do not accept the obvious implications of the results. They stick to the assumption that the assessment measures skills and knowledge -- so long as students are sufficiently motivated. What is needed, they conclude, is to motivate students to take the test seriously (sound familiar?). So ETS will now provide all test-takers with an electronic certificate that indicates score and may be shared. Oh boy, another rationale to take the Proficiency Profile -- it may be required by future employers.

What the results of the ETS study show is that the Proficiency Profile measures the motivation to perform the skills and demonstrate the knowledge captured by the assessment. The model underlying our assumption that tests measure skills and knowledge directly is faulty. Using the conventional model of human performance we imagine that competence is derived from (some mixture of) innate ability and appropriately stimulating prior experience. Further we imagine that levels of performance can then be measured by a test so long as the test-taker is sufficiently motivated.

Here’s a better model: personal competence is the motivation to perform the skill or demonstrate the knowledge. That’s what the ETS study revealed --  test scores correlated with motivation scores. Motivation is considerably more complex than just the perceived use of a test score, of course, but the result is generalizable. Why did Johnny score higher on the Praxis 1 than Jerry? Johnny was more motivated than Jerry, and likely in a much deeper and more complex way than his feelings about a single test.

Ability and experience are the mechanical components of human performance. They twist and pull us like the strings of a puppeteer. Motivation cuts through and surrounds all of that like a fourth dimension. It isn't the same as free floating and mysterious willpower, but it does reside in and ultimately define the person. For better or worse, it directly corresponds with the responsibility given to the person. We cannot motivate a person, but we can -- and should -- hold him or her accountable. A person must be credited for what is due and debited for what is owed. We needn't have tests for this -- the world has it’s own way of settling accounts -- but any test we administer to a person can only give us a measure of his or her motivation to take it.

Whose mastery, autonomy, and purpose are served by schools?

A popular TED talk features Dan Pink warning that business productivity suffers because managers remain captive to an outdated model of motivating employees with carrots and sticks. Rewards and punishments are effective, he says, but only for motivating routine mechanical tasks. Creative thinking requires more intrinsic sources of motivation. Workers who have control over their own tasks, who can see themselves improving, and who serve something bigger than themselves will be more motivated, and therefore more productive, than their counterparts who are micromanaged, are rewarded for meeting specific performance goals, and who focus on the single level at which they work.

The parallel problem exists in every classroom. Teachers are asked to motivate their students. Like the managers referred to by Pink, teachers are captive to a system of extrinsic motivators. They control nearly every aspect of a young person’s experience while at school. “This is your assigned seat, Johnny. If you sit and do your work you can join your classmates for recess. Line up to go to the restroom. You didn’t get finished so you have silent lunch. Great job, here is a smiley face. You get an A.”

It is too easy to criticize businesses and schools for the daily indignities imposed on old and young people like driving so many head of cattle – because that is how it is done. It is incumbent on us, however, to thoroughly understand something so ubiquitous. It worked in the past, Pink argues, because work was different in the past. Here’s the problem: it works now. If it didn’t, it would go away and be replaced by better motivational systems. In a Darwinian world any such system has to “pay for itself.” The only question, as the philosopher Dan Dennett frequently reminds his readers, is who (or what) benefits. Pink uses a word that captures the benefit of extrinsic motivation: if you want compliance, he says, then extrinsic motivators work very well. If you want engagement, you need intrinsic motivation. One way to look at schooling, then, is to see how we motivate students and to deduce from that what we value. We use extrinsic motivation, therefore we value compliance. That will be the subject of a future post. We will know that engagement is most important when we find that students are motivated intrinsically.

The fatal flaw in Pink’s appeal is his implied audience. He makes his pitch to managers as if it is up to them. If they would just realize what motivates people they could change something about what they do in order to get more out of their employees. The “more” they will get, however,  should not be confused with more progress toward the objectives of the manager. Managers and teachers cannot “use” intrinsic motivation as if it was an intervention to be imposed from the outside. Intrinsic motivation, by definition, emerges from individual employees or students. The manager cannot give autonomy in order for the employee to be more motivated to do the manager’s bidding. Autonomy means that the employee is free from the manager’s bidding. And what makes “purpose” motivating is when it goes beyond the narrow purposes of the manager. Even mastery is difficult for the manager to define. Improvement is motivating regardless of where one starts, and assessment of performance is likely to depend more on comparisons among peers than on standards defined from above.

Pink says he can show you that intrinsic motivation works better than extrinsic motivation. He uses the example of Google, who gives employees 20% of their time to use as they see fit. Half of Google’s great innovations, Pink says, have been hatched in that discretionary time. Autonomy, mastery, and purpose are quite motivating for Google employees, but not because Google managers applied them toward their own ends. First, Google employees are compensated with stock options, which means to some extent they ARE Google. Second, the innovations they come up with are pitted against all the others come up with by their peers in a highly competitive environment. Finally, the set of people employed at Google are there because they already took initiative and stood out from those around them. They are go-getters with their own, strongly held, interests. Pink should have addressed workers, noting that in the world of tech start-ups, their interests define the company’s success.

Schools continue to impose top-down compliance because they serve their own interests, not those of their students. In the emerging technology-driven world, the interests of young people will eventually drive the goals of successful schools. This will not happen because administrators foresee it and intervene, however. It will proceed in the way of evolution, on the backs of those systems, and students, that failed.