Periodically, I turn to Alfie Kohn to metaphorically kick me in the rear and make me question everything I do- because he may change what I’m doing, but also because he makes me get very certain about why I’m doing something.
Recently, he caused me to question rubrics. Rubrics are a central part of my work at New Tech Network. We use rubrics to define our goals and assess growth for everything from school success to student agency. And we’re working on a set of rubrics that help us assess essential skills in the disciplines and see where students are on a trajectory towards college readiness. We’ve been calibrating with our colleagues on these rubrics, using them to look at student work, and they’ve been a big part of our work with schools.
So it was good to read this:
“Elaborate rubrics used to judge students’ performance represent another form of standardized assessment that’s rarely recognized as such. The point is to break down something, such as a piece of writing, into its parts so that teachers, and sometimes the students themselves, can rate each of them, the premise being that it’s both possible and desirable for all readers to arrive at the same number for each criterion. Rubrics are borne of a demand to quantify and an impulse to simplify.”
from Why the Best Teachers Don’t Give Tests by Alfie Kohn
Good, as I noted above, because it made me question why we were using rubrics, and good because it made me really refine why I actually thought they were a good thing.
Kohn’s argument has a number of parts, but let me address a few of them.
- Point 1: “The point is to break down something, such as a piece of writing, into its parts so that teachers, and sometimes the students themselves, can rate each of them…”
NTN, and our partners at the Stanford Center for Assessment, Learning, and Equity, have been spending a lot of time breaking disciplinary thinking and writing into its parts, such as argument, evidence, and analysis and synthesis.
I’d actually argue that we’re not breaking disciplinary thinking down in this way just to rate students. We’re doing it because: 1) writers, especially novice ones, need clarity about writing expectations. And the various disciplines and genres of writing have specific constraints and characteristics, and I think it does students a disservice not to be transparent about them or assume students will simply absorb them. We shouldn’t shy away from defining quality- other industries and professions don’t.
And 2), we break disciplinary writing down because it helps us outline a learning trajectory from emerging to advanced mastery of a particular skill. Why is that learning trajectory helpful? It helps teachers see where to go next with a student- what will be the piece of scaffolding that will help them gain increased mastery of a particular skill? Is the student’s argument “unclear”? Ok, then what would it take to help them go to a “somewhat clear” argument? And, best of all, students can see their own growth, emphasizing a growth mindset.
- Point 2: “…the premise being that it’s both possible and desirable for all readers to arrive at the same number for each criterion…
I actually think it’s possible for readers to arrive at the same- or a similar (aka calibrated)- number for each indicator or dimension. In Beyond the Bubble Test, Brian Stecher notes that, “it is possible to train qualified rater to score well-constructed, standardized performance tasks with acceptable levels of consistency using thoughtful rating criteria” (chapter 2). Whether or not it’s desirable is another question. I’d argue that the conversation we’ve had around rubrics as we calibrate is really rich and worth it in itself. As we calibrate, we talk about the biases we need to leave at the door. We ask ourselves, and each other- “What evidence did you use to determine that score level?” forcing us to go back into student work rather than make assumptions. And, very naturally, the conversation turns to how we could design a better task that would really show us what students know, and what it would take to ensure that this student grew the next time around.
It’s true that arriving at a number or score level to encompass student achievement can be reductive. But I trust that teachers can use rubrics in thoughtful and purposeful ways. I know that they can think about rubrics and a piece of student work as one piece of evidence about a student, not the sum totality of this student’s achievement. I believe that, with support, they can benefit from the richness of the calibration conversation, and use the results of scoring to improve their future instruction, not just quantify and categorize students.
- Point 3: “Rubrics are born of a demand to quantify and an impulse to simplify.”
I think I’ve addressed the “quantify” aspect above- it’s not just about quantifying. The notion that rubrics simplify writing and thus produce students who are little writing clones of each other still keeps me up at night. However, writers do express themselves in very particular ways depending on the discipline (think about how someone in the discipline of English would write about a poem vs. how a mathematician would write a proof). As I noted above, we need to be really clear about these disciplinary differences, and rubrics are one way to do so. In addition, as we’ve looked at different examples of student work and seen how different students express themselves, I actually think we’ve expanded teachers’ notions of how a student might show mastery of a particular skill. We recently read a student paper which really pushed teachers’ notions of proper organization- it was an example where the organization was really driven by the argument, not a predetermined outline. Many teachers initially found it to be disorganized because it didn’t follow a formula, but began to see its internal logic after discussion.
Rubrics can just be standardized checklists of what we want students to do. We could use them in such a way that we narrow our focus and narrow our definition of student achievement. But I think that when rubrics are used thoughtfully, as part of conversations about student work where we try to leave our biases at the door and really focus on what students can do, that they can be rich tools for teaching and learning.
Alfie Kohn ends this particular section by saying,
“Some insist, against all evidence to the contrary, that there’s no problem as long as one uses a good rubric.”
I’ve already tried to outline a different way to use rubrics. I’m going to try to insist that good rubrics do matter… next time.