Friday, April 21, 2017

PBGrading Pitfalls

Two of the selling points of Proficiency-Based Education are the elimination of the "False Accuracy" of percentages and the averaging of things that have nothing to do with each other.

We will score a test on Quadratics as 90%, score a test on rational functions as a 40% and then average those two scores to a 65%. Throw in a couple missing homework assignments and it's a failure. Add a bunch of homeworks handed in (100% each, weighted average), another test on square roots (80%), "Participation points" for having a pencil every day and not being an asshole, and some "extra credit" for a well-done project on exponential functions that was mostly a rehash of something done in Algebra I, and now this is a C+ or a B grade. If it's 79.43%, then it's a C+.

How in the name of Cthulhu can we be that accurate?


Why does having a pencil raise your grade?

Why does missing homework lower your grade?

How does "extra credit" on one topic cover the fact that you don't know what you're doing on a second or third topic?

Can't solve an equation, can't find asymptotes or holes, can't factor quadratics if a != 1, can't determine the missing terms in an geometric sequence ... but can grub points here and there, and "Boy, he's trying really hard and he deserves to get a few extra points so his grade is above 80."

How can you assure the Pre-Calculus teacher that this kid is ready for it? How about the college professors who are constantly droning on about freshmen in remedial math classes?

Look at those standards. Sure, they're all about working with quadratics in some form or other, but skill in N.CN.2 does not equate to skill in A.SSE.3 or in F.IF.7. So how does the good grade we get in part of this "help raise" the poor grade in another?

Shouldn't we be asking for skill and understanding in each of these? Don't we want proficiency (to some standard) in all of these before we say "Algebra 2" on a transcript?

And so we arrive at Proficiency-Based Grading.

At its ideal, it's perfect.
  • List all the proficiencies. Let's pretend we've decided that proficiencies #1 through #10 are required for a credit in algebra 2. 
  • Set up a scale: Proficient w/Distinction, Proficient, Nearly Proficient, Emerging Proficiency.
  • Assess: Decide the rubric/scoring method, be consistent, ignore the names, begin.
Determine (using as much time as needed) whether each student can do the things you want understood for Algebra 2. If retakes are needed, do that. If they're good on 9of those standards, then they haven't fulfilled your requirements. No credit until they understand all ten.

Repeat to the students, "There are ten things you need to know before you can say 'I understand Algebra 2' and can take that to the next course."

Ah, but this is education, and now we need to "fix" things.

First, having only ten grades in the gradebook is not going to cut it with secondary level administrators.  You need to include all of your formative and summative assessments.

Then, because we have to use PowerSchool, we need to list all of the standards for math, even if we are only focused on those 10 for this course. The other math teachers need their 10 things, and Powerschool can't be configured differently for each course ... blah, blah, blah. Probably it can, but the tech people and the curriculum coordinator can't figure it out, so fuck you.


Every column needs a grade, so the pilot teachers enter E for everything not covered in Algebra 2. (That's a lot)

Parents immediately complain that there are all these Es, "Why is this?" So we make a fifth category, "N/A, Haven't done this yet."

People who should know better insist that everything have a numerical value. So we label the levels 1, 2, 3, 4 (and 0 for the Haven't Done it Yet" category) ... and PowerSchool promptly averages the scores.

That's right, it takes the old problem of averaging things that have nothing to do with each other and magnifies it by averaging Ordinal Data of things that have nothing to do with each other.


"Advanced Understanding of Adding and Subtracting complex numbers" combined with "Nearly Proficient in Graphing Functions" somehow equates to 3, Proficient.

Not only that, but if you have something like N.CN.2 which you have determined to be only a 1, 2, or 3 scale ... well, your students are going to be shocked when they can't "get a 4" for the course.

XKCD: 937
Ordinal data is qualitative data; you can't average qualitative data. Doing so is a sin against mathematics. It would be akin to a newspaper writing this headline about a marathon. "The top ten people averaged fifth-and-a-halfth place" -- how stupid is that statement?


Then there's accuracy. How accurate is that 3 or 4, anyway? The teachers who piloted this program in the other building began to think that "this 3 is different from that 3; I want to show progress" and promptly began to use halfs.

Then came re-takes. If I give a ten-question assessment of N.CN.2, and a student is deemed nearly proficient on those ten questions, does that mean the student is proficient? Probably not. Let's test him again. He takes four more tests over the next few days, scores proficient all four times. His average is less than 3. To be exact, 14/5 = 2.8

So we compensate by telling PowerSchool to "take the most recent four scores" every time, thinking that we want to see improvement. The kid who scores 4 because he absolutely understands it and can use this knowledge to write a computer program to run a Lego MindStorm robot to draw the function on the hallway floor, still has to take the test three more times so PowerSchool can find an average of the "most recent four". And, just to be funny, he scores 4, 4, 4, and 0, and lets PowerSchool average that to "Proficient".

Remember that comment about needing all of the formative and summative assessments?  Formative work is simply assignments and quizzes that help your students learn. They try, and fail, then try again. You need to assess this work, but it doesn't count. All you want here is "Does the student understand N.CN.7?"

I said that formative be recorded but be worth  0% - enough to be noticed but not enough to matter. Of course, telling admin that something won't count means they assume that students won't do it, so it has to count. Idiots in charge at my school decided formative was 25% of the grade, summative 75%.

Thus, formative scores of 1, 2, 2, 1, 2, and 3 (because the student is still learning) and then summatives of 3, 3, 3, and 3 (all proficient, meaning this kid understands this topic) will result in a final mark of 2.7 (nearly proficient).

How does this make sense? It doesn't.


I'm certain that many of you are saying "Hey, the old way did most of this, too?"

Yes, Mr. TuQuoque.


My point is that we should have Proficiency-Based Grading without these pitfalls. If PowerSchool can't do it properly, then you should stay with the old grading methods until you get a proper gradebook.

Warning Signs that you're Doing it Wrong:

  • You find yourself using 3.5 because the student is "More Proficient than just Proficient" but not quite "Advanced Proficient"
  • You average what shouldn't be averaged.
  • You let the learning process alter the proficiency measurement. 
  • 90% of the marks in your gradebook are 0 because those standards aren't in this course.
  • Workarounds of any kind in PowerSchool.

Monday, April 17, 2017

Charter Schools, Surprise!

In Spending Blind: The Failure of Policy Planning in California Charter School Funding, Gordon Lafer -- a University of Oregon prof who also works for Oakland's The Public Interest -- finds "hundreds of millions of dollars ... spent each year without any meaningful strategy... on schools built in neighborhoods that have no need for additional classroom space, and which offer no improvement over the quality of education already available in nearby public schools. In the worst cases, public facilities funding has gone to schools that were found to have discriminatory enrollment policies and others that have engaged in unethical or corrupt practices."

Sunday, April 9, 2017

On Relevance

John Spencer has this on relevance.
I despise the notion that urban, low-SES students have to analyze hip hop before they can "get into" poetry. It's not that I'm opposed to hip hop poetry (we do a few Def Jams poems and analyze the occasional rap song), but I disagree with the notion that poetry can only speak truth to coffee shop geeks or grad students in the literature department.
Which is a great point. I would argue that it is either relevant or not; if you feel the need to "make it relevant," you will fail and the lesson will fall flat. Students do not need to be conned by relevance and will resist any imposed relevance.


Sometimes Math is just math. It isn't Real World. It has nothing to do with answering "When am I ever gonna have to use this?"

It's a topic we're exploring, and we can follow it a ways down this path.

Two roads may diverge in the yellow wood, but fortunately we can follow both. In most of high school mathematics, there's no need to choose only that which is "relevant".

In fact, that's probably the worst option available.

Tuesday, April 4, 2017

After a while you give up.

ISS

Consider the following scenarios:

A conference at St. Michael's College in Burlington, Vermont. The speaker is Professor Robert Talbert (@RobertTalbert) and he is presenting his work with flipped learning to an audience of college professors and a few HS teachers. He speaks well, he is prepared, and he has given out material well in advance that he expects that everyone will have read, and questions that he expects everyone will have answered. Not surprisingly, everyone has. We discuss FL and his new book. We see what he has done with FL: what works well and why, and what doesn't and why. People work throughout the time, notes are taken, the food is lovely and the workshop is a success.

Or a Bio-Ethics conference at UVM Medical Center. Similarly, everything is straightforward, talks are given, information is presented, people listen and take notes. the food is lovely. The workshop is a success

A running theme throughout is the assumption of competence, treating everyone as if their time was valuable and that they were there because they wanted to be, that they wanted to hear what the speaker had to say.

Then, there's a conference geared towards high-school teachers.

Invariably, there's a bowl of candy. Your choice is used to sort you into groups. Most of the time, you need to stand up and hold hands with the next person and play "telephone" to demonstrate that ground-breaking idea that teenagers don't always listen attentively to your instructions.

There's a stack of post-it notes to stick to chart paper. Here's some colored markers. This presenter uses grade-school vocabulary and that sing-song voice one uses with 10 year-olds. That coordinator has 60 people crowd into a 20x20 foot space and then "Move to that side if you agree that that differentiation is a good thing, move to this side if you are wrong."

Did I mention that you have a Master's degree?

You'll be handed a paper copy of an article that was emailed out a few days before because "Not everyone is as advanced technologically as you" and couldn't be expected to have read anything ahead of time. I know, right? We can't expect anyone to know how to read something sent by E-Mail, apparently.

"Read this article and find the sentence that means the most to you."
 The "scribe" will write it on the chart paper.

"Find the phrase that means the most to you."
The "scribe" will write it on the chart paper.

"Find the word that means the most to you."
 The "scribe" will write it on the chart paper.

90 minutes later, you are done with a 3-page double-spaced article on the wonders of Common Core State Standards. You'd think we'd be further along in this process eight years after the introduction of CCSS, but no, we're examining a propaganda piece instead of developing course material to align with standards.

Did I mention that you have been teaching this material for 30-odd years?

Did I mention that the state of Vermont has decreed that all public school will be using the Common Core and that there's no particular reason to read about how wonderful it is?

I am not exaggerating. The whole thing would be demeaning and ridiculous but you get the feeling that the presenters aren't capable of anything more strenuous intellectually than 6th grade social studies -- your course work is gibberish to them.

"We're going to use the Gallery Walk protocol today."

"You have thirty-two minutes to discuss this topic in your groups of four. Person A will speak for 3 minutes with no interruptions. Following that, the table with remain quiet for 2 minutes to deeply consider what was said. Then persons B, C, and D will take 1 minute each to respond."

"There are easels around the room. Please take your Post-It note and attach it the the chart paper next to the statement that most closely matches your opinion."

You can resist only so much. After a while, you begin to go along with it all just to keep some progress happening. You know that if you ask a question, the gears in their heads will seize and jam and you'll never get anything done.

You dutifully watch the videos in the group, even though everyone is watching it starting at a different instant and the cacophony is making people twitch.  The video could have been viewed on our own time, but I guess not. In Bizzarro World, it's better to have us all use the limited time we have together to not work together. Again, it's not informative; it's a college kid's project touting the glories of the work we're going to begin doing someday.

*HeadDesk*

Then, you'll write down what color the video was ... I chose "FUSCHIA" because it included lots of letters from the word I really wanted to use.


It becomes easier to let them ramble, to let them play their games of "Pass the ball of yarn back and forth and then it will represent the network of caring that we have here." You resign yourself to never getting anything actually finished during inservice. Proficiency-based grading isn't slated to be implemented for another three years because "some people can't even use Google Docs" and "I'm trying to teach the faculty how to fill in my template for learning" - you know we should be doing this more quickly but it's easier to just give up.

Let's call it: "Inservice Stockholm Syndrome."



Monday, April 3, 2017

Missing the Point

Twitter:
If most students learn better one-on-one, why then don't we always break class into groups or make learning partners?
Maybe because students "learn" from teachers and "practice" with other students?

Thursday, March 30, 2017

Innate Skills

Every time we talk about "digital natives" and "Kids' innate skills with technology", we reinforce the idea that you've either got it or you don't, that if you are over thirty then you can't be good with tech, that if you're under thirty you don't need any training because you're simply imbued with an understanding of all silicon-based circuitry.

Fatuous self-indulgent hokum.

Let me state it for the record: There is no such thing as "Innate Skill" with technology. Kids have had more practice at playing games and chatting via FB, text, or IM, but nothing else. They are, on average, more comfortable holding a device but not better at using it for anything academic or work-related (unless that use includes playing games, or chatting via FB, text, or IM).

We are running counter to the ideas of lifelong learning, laying a downfield block on any need for a student to persist when faced with a computing obstacle. In fact, we are teaching them and they're learning.

We're teaching them to give up instantly.

We have had decades of computer games with puzzles and problems and every single one has a cheat code or "God mode" that is readily found on the Internet … meaning that every student has learned to try a problem for approximately 15 seconds and then Google the shortcut or cheat code.

But we still have to start with the simple problems.

College professors who shout from their Ivory Towers that "If you can Google the answer, you need to ask a better question" are foolish. Those who advocate for direct plagiarism in all things under the premise that "Research skills are important in the modern world" are delusional and flat-out wrong.

The simple and the intermediate questions are already answered somewhere, but we can't give up and jump right to the higher-order connections because the kids have not answered the simple questions yet -- Google is not answering. They don't have the simple understanding they need in order to make the higher-order connections and that critical thinking EduWonks are always going on about.

The simple questions that need to be asked first (formative) are being ignored for rote guessing, but we still have to find a way to ask them anyway.

The intermediate questions that form the bridge between formative and summative and require the mental processing to form long-term memory through understanding are being ignored, but we still have to ask them anyway.

This came up most obviously in my Intro Coding class. I gave some students selected problems from ProjectEuler.net, wanting them to have a serious mathematical question to answer using spreadsheets.

Here is one of the early ones:
By considering the terms in the Fibonacci sequence whose values do not exceed four million, find the sum of the even-valued terms.

Instead of thinking through the issues and working towards an understanding of the tools at hand, they googled the question ("project euler question 2") and worked backwards from the MathBlog entries.

I learned quickly to change the targets and to not advertise the source of the problem. The Archimedes Cattle problem becomes much harder to solve when you don't tell them "Archimedes" or "Cattle" and change the wording from "1000 cattle" to "1500 horses".

If we ever expect them to do anything more complicated, we have no choice.

Sunday, January 29, 2017

Evaluations are difficult everywhere.

One of the common complaints I hear is that of "not being able to fire bad teachers." It comes only slightly more often than "Teachers make too much money" and "Why are there so many bad teachers?" What we have here is a difficulty of evaluating teachers by administration, but I figure it's more than that.

It's that evaluation is a bitch, and it's not just teachers. It's everyone and everywhere.

When you complain that teachers make too much money, you are subconsciously saying that it's too much money for someone that bad, meshing the top-of-the-scale salary with the bottom-of-the-scale ability and assuming that it's the same person (it rarely is).

The anti-collective bargaining group wants to be able to pay the superstars at a superstar rate and pay the slugs with a pink slip.

The problem is, of course, figuring out who is who.

Years ago, I had a terrible dentist. He drilled holes everywhere and messed up my teeth pretty badly. There was no way to find out if he was bad or not before I sat in the chair. My current dentist insists that the teeth that are deteriorating under the old dental work would have done so anyway but it's stunning how well the tooth right next to it is doing under his work. Thankfully I have dental insurance and I'm getting a lot of this work done.

You will never hear a dentist talk ill of another - it's always your fault for not brushing or flossing enough.

Mrs. C. had an even worse dentist experience and has had to undergo a great deal of expensive repair. Lawsuits finally brought the guy out of his office but it turned out that the dental review board didn't dare state publicly that they thought he wasn't up to snuff. They didn't want "to turn on one of their own."

I've finally found a decent mechanic, no thanks to any review board or any kind of Craigslist or Angie's-list or State Cert Panel. Certifications line the walls of every shop in town, but mostly they suck.

When we look around, we find that every job is filled with average workers. Some are good and some are terrible but most are just okay. Teachers are no different.

A local first responder just got his fifth DUI.
Contractors are legendary for their variability and lets not make any cracks about plumbers. ;-)
The writers for the paper are so-so and don't always compare favorably with the students who write for the high-school section.
Politicians? We won't go there.
Doctors? Lawyers?  Used Car Salesmen?

How do we judge thee? Let me count the ways.

Teachers judge teachers very differently than admins do. Parents use a different yardstick and the taxpayer with no kids and an attitude about taxes and education still another.

Algebra in The RealWorld

Dan Meyer poses Three Questions about a problem from his professional development:
A youth group with 26 members is going to the beach. There will also be 5 chaperones that will each drive a van or a car. Each van seats 7 persons, including the driver. Each car seats 5 persons, including the driver. How many vans and cars will be needed?
"One, is the problem realistic? Would a real person need to solve this problem?
"Two, is the solution realistic? Would a real person solve the problem using a system of two equations?
"Three, in what ways does this problem help our students become better problem solvers?"

I think he's right, in many ways. The problem he's discussing is contrived and convoluted at best. It does have that "algebra-book word problem from hell" feel about it, but word problems in math textbooks are a funny thing. They have to straddle the line between being realistic and being useful in a classroom for teaching. When constructing a word problem or using one, you need to keep this line in mind. If the word problem doesn't fit your topic, you should change it.

I take a slight issue with the questions, though, in that I don't want to always be asking for a real-world solution that a real-world person would find for a real-world problem.

First, define a "real person". Do I pick me or a mathphobe? If you purposefully want a problem that takes creativity to solve, then separate it from the track you're in. Call it the Puzzle of the Week.

This particular problem doesn't have enough information to come to a single solution anyway - how about adding in the cost per mile of van and car, as well as an additional payment for driver responsibility. Are we trying to minimize cost or make sure all chaperones drive? Do we have a reason to not use the vans unless necessary?