PBGrading Pitfalls

Two of the selling points of Proficiency-Based Education are the elimination of the "False Accuracy" of percentages and the averaging of things that have nothing to do with each other.

We will score a test on Quadratics as 90%, score a test on rational functions as a 40% and then average those two scores to a 65%. Throw in a couple missing homework assignments and it's a failure. Add a bunch of homeworks handed in (100% each, weighted average), another test on square roots (80%), "Participation points" for having a pencil every day and not being an asshole, and some "extra credit" for a well-done project on exponential functions that was mostly a rehash of something done in Algebra I, and now this is a C+ or a B grade. If it's 79.43%, then it's a C+.

• How in the name of Cthulhu can we be that accurate?
• Why does having a pencil raise your grade?
• Why does missing homework lower your grade?
• How does "extra credit" on one topic cover the fact that you don't know what you're doing on a second or third topic?

Can't solve an equation, can't find asymptotes or holes, can't factor quadratics if a != 1, can't determine the missing terms in an geometric sequence ... but can grub points here and there, and "Boy, he's trying really hard and he deserves to get a few extra points so his grade is above 80."

How can you assure the Pre-Calculus teacher that this kid is ready for it? How about the college professors who are constantly droning on about freshmen in remedial math classes?

Look at those standards. Sure, they're all about working with quadratics in some form or other, but skill in N.CN.2 does not equate to skill in A.SSE.3 or in F.IF.7. So how does the good grade we get in part of this "help raise" the poor grade in another?

Shouldn't we be asking for skill and understanding in each of these? Don't we want proficiency (to some standard) in all of these before we say "Algebra 2" on a transcript?

And so we arrive at Proficiency-Based Grading.

At its ideal, it's perfect.

• List all the proficiencies.
• Set up a scale: Proficient w/Distinction, Proficient, Nearly Proficient, Emerging Proficiency.
• Assess: Decide the rubric/scoring method, be consistent, ignore the names, begin.

Let's pretend we've decided that proficiencies #1 through #10 are required for a credit in algebra 2. Determine (using as much time as needed) whether each student can do the things you want understood for Algebra 2. If retakes are needed, do that. If they're good on 9 of those standards, then they haven't fulfilled your requirements. No credit until they understand all ten.

Repeat to the students, "There are ten things you need to know before you can say 'I understand Algebra 2' and can take that to the next course."

Ah, but this is education, and now we need to "fix" things.

First, having only ten grades in the gradebook is not going to cut it with secondary level administrators.  You need to include all of your formative and summative assessments.

Then, because we have to use PowerSchool, we need to list all of the standards for math, even if we are only focused on those 10 for this course. The other math teachers need their 10 things, and Powerschool can't be configured differently for each course ... blah, blah, blah. Probably it can, but the tech people and the curriculum coordinator can't figure it out, so fuck you.


Every column needs a grade, so the pilot teachers enter E for everything not covered in Algebra 2. (That's a lot)

Parents immediately complain that there are all these Es, "Why is this?" So we make a fifth category, "N/A, Haven't done this yet."

People who should know better insist that everything have a numerical value. So we label the levels 1, 2, 3, 4 (and 0 for the Haven't Done it Yet" category) ... and PowerSchool promptly averages the scores.

That's right, it takes the old problem of averaging things that have nothing to do with each other and magnifies it by averaging Ordinal Data of things that have nothing to do with each other.

"Advanced Understanding of Adding and Subtracting complex numbers" combined with "Nearly Proficient in Graphing Functions" somehow equates to 3, Proficient.

Not only that, but if you have something like N.CN.2 which you have determined to be only a 1, 2, or 3 scale ... well, your students are going to be shocked when they can't "get a 4" for the course.

XKCD: 937

Ordinal data is qualitative data; you can't average qualitative data. Doing so is a sin against mathematics. It would be akin to a newspaper writing this headline about a marathon. "The top ten people averaged fifth-and-a-halfth place" -- how stupid is that statement?

Then there's accuracy. How accurate is that 3 or 4, anyway? The teachers who piloted this program in the other building began to think that "this 3 is different from that 3; I want to show progress" and promptly began to use halfs.

Then came re-takes. If I give a ten-question assessment of N.CN.2, and a student is deemed nearly proficient on those ten questions, does that mean the student is proficient? Probably not. Let's test him again. He takes four more tests over the next few days, scores proficient all four times. His average is less than 3. To be exact, 14/5 = 2.8

So we compensate by telling PowerSchool to "take the most recent four scores" every time, thinking that we want to see improvement. The kid who scores 4 because he absolutely understands it and can use this knowledge to write a computer program to run a Lego MindStorm robot to draw the function on the hallway floor, still has to take the test three more times so PowerSchool can find an average of the "most recent four". And, just to be funny, he scores 4, 4, 4, and 0, and lets PowerSchool average that to "Proficient".

Remember that comment about needing all of the formative and summative assessments?  Formative work is simply assignments and quizzes that help your students learn. They try, and fail, then try again. You need to assess this work, but it doesn't count. All you want here is "Does the student understand N.CN.7?"

I said that formative be recorded but be worth  0% - enough to be noticed but not enough to matter. Of course, telling admin that something won't count means they assume that students won't do it, so it has to count. Those "in charge" at my school decided formative was 25% of the grade, summative 75%.


Thus, formative scores of 1, 2, 2, 1, 2, and 3 (because the student is still learning) and then summatives of 3, 3, 3, and 3 (all proficient, meaning this kid understands this topic) will result in a final mark of 2.7 (nearly proficient).

How does this make sense? It doesn't.


I'm certain that many of you are saying "Hey, the old way did most of this, too?"

Yes, Mr. TuQuoque.


My point is that we should have Proficiency-Based Grading without these pitfalls. If PowerSchool can't do it properly, then you should stay with the old grading methods until you get a proper gradebook.

Warning Signs that you're Doing it Wrong:

• You find yourself using 3.5 because the student is "More Proficient than just Proficient" but not quite "Advanced Proficient"
• You average what shouldn't be averaged.
• You let the learning process alter the proficiency measurement. 
• 90% of the marks in your gradebook are 0 because those standards aren't in this course.
• Workarounds of any kind in PowerSchool.

Charter Schools, Surprise!

In Spending Blind: The Failure of Policy Planning in California Charter School Funding, Gordon Lafer -- a University of Oregon prof who also works for Oakland's The Public Interest -- finds "hundreds of millions of dollars ... spent each year without any meaningful strategy... on schools built in neighborhoods that have no need for additional classroom space, and which offer no improvement over the quality of education already available in nearby public schools. In the worst cases, public facilities funding has gone to schools that were found to have discriminatory enrollment policies and others that have engaged in unethical or corrupt practices."

On Relevance

John Spencer has this on relevance.

I despise the notion that urban, low-SES students have to analyze hip hop before they can "get into" poetry. It's not that I'm opposed to hip hop poetry (we do a few Def Jams poems and analyze the occasional rap song), but I disagree with the notion that poetry can only speak truth to coffee shop geeks or grad students in the literature department.

Which is a great point. I would argue that it is either relevant or not; if you feel the need to "make it relevant," you will fail and the lesson will fall flat. Students do not need to be conned by relevance and will resist any imposed relevance.


Sometimes Math is just math. It isn't Real World. It has nothing to do with answering "When am I ever gonna have to use this?"

It's a topic we're exploring, and we can follow it a ways down this path.

Two roads may diverge in the yellow wood, but fortunately we can follow both. In most of high school mathematics, there's no need to choose only that which is "relevant".

In fact, that's probably the worst option available.

After a while you give up.

ISS

Consider the following scenarios:

A conference at St. Michael's College in Burlington, Vermont. The speaker is Professor Robert Talbert (@RobertTalbert) and he is presenting his work with flipped learning to an audience of college professors and a few HS teachers. He speaks well, he is prepared, and he has given out material well in advance that he expects that everyone will have read, and questions that he expects everyone will have answered. Not surprisingly, everyone has. We discuss FL and his new book. We see what he has done with FL: what works well and why, and what doesn't and why. People work throughout the time, notes are taken, the food is lovely and the workshop is a success.

Or a Bio-Ethics conference at UVM Medical Center. Similarly, everything is straightforward, talks are given, information is presented, people listen and take notes. the food is lovely. The workshop is a success

A running theme throughout is the assumption of competence, treating everyone as if their time was valuable and that they were there because they wanted to be, that they wanted to hear what the speaker had to say.

Then, there's a conference geared towards high-school teachers.

Invariably, there's a bowl of candy. Your choice is used to sort you into groups. Most of the time, you need to stand up and hold hands with the next person and play "telephone" to demonstrate that ground-breaking idea that teenagers don't always listen attentively to your instructions.

There's a stack of post-it notes to stick to chart paper. Here's some colored markers. This presenter uses grade-school vocabulary and that sing-song voice one uses with 10 year-olds. That coordinator has 60 people crowd into a 20x20 foot space and then "Move to that side if you agree that that differentiation is a good thing, move to this side if you are wrong."

Did I mention that you have a Master's degree?

You'll be handed a paper copy of an article that was emailed out a few days before because "Not everyone is as advanced technologically as you" and couldn't be expected to have read anything ahead of time. I know, right? We can't expect anyone to know how to read something sent by E-Mail, apparently.

"Read this article and find the sentence that means the most to you."
 The "scribe" will write it on the chart paper.

"Find the phrase that means the most to you."
The "scribe" will write it on the chart paper.

"Find the word that means the most to you."
 The "scribe" will write it on the chart paper.

90 minutes later, you are done with a 3-page double-spaced article on the wonders of Common Core State Standards. You'd think we'd be further along in this process eight years after the introduction of CCSS, but no, we're examining a propaganda piece instead of developing course material to align with standards.

Did I mention that you have been teaching this material for 30-odd years?

Did I mention that the state of Vermont has decreed that all public school will be using the Common Core and that there's no particular reason to read about how wonderful it is?

I am not exaggerating. The whole thing would be demeaning and ridiculous but you get the feeling that the presenters aren't capable of anything more strenuous intellectually than 6th grade social studies -- your course work is gibberish to them.

"We're going to use the Gallery Walk protocol today."

"You have thirty-two minutes to discuss this topic in your groups of four. Person A will speak for 3 minutes with no interruptions. Following that, the table with remain quiet for 2 minutes to deeply consider what was said. Then persons B, C, and D will take 1 minute each to respond."

"There are easels around the room. Please take your Post-It note and attach it the the chart paper next to the statement that most closely matches your opinion."

You can resist only so much. After a while, you begin to go along with it all just to keep some progress happening. You know that if you ask a question, the gears in their heads will seize and jam and you'll never get anything done.

You dutifully watch the videos in the group, even though everyone is watching it starting at a different instant and the cacophony is making people twitch.  The video could have been viewed on our own time, but I guess not. In Bizzarro World, it's better to have us all use the limited time we have together to not work together. Again, it's not informative; it's a college kid's project touting the glories of the work we're going to begin doing someday.

*HeadDesk*

Then, you'll write down what color the video was ... I chose "FUSCHIA" because it included lots of letters from the word I really wanted to use.


It becomes easier to let them ramble, to let them play their games of "Pass the ball of yarn back and forth and then it will represent the network of caring that we have here." You resign yourself to never getting anything actually finished during inservice. Proficiency-based grading isn't slated to be implemented for another three years because "some people can't even use Google Docs" and "I'm trying to teach the faculty how to fill in my template for learning" - you know we should be doing this more quickly but it's easier to just give up.

Let's call it: "Inservice Stockholm Syndrome."

Missing the Point

Twitter:
If most students learn better one-on-one, why then don't we always break class into groups or make learning partners?

Maybe because students "learn" from teachers and "practice" with other students?