The Problem with Education Research

"Education Research." Even in these times of political ignorance of research, science, and fact-based decision-making, there's still a place in every American's brain for education research.

It's probably due to the ever-present mantra of "Won't somebody PLEASE think of the children?" coupled with an ferocious need to believe that one's own children would be superstars if only the damned teachers weren't so terrible. Parents tweet, post, and search for information about "best-practices", proCCSS or anti-CCSS, pro-disease or pro-vaccination, in a desperate search for confirmation that they have a brilliant child.

The problem, of course, is that the searchers don't connect with the research.

Linda Graham, in an article on TES, Teachers Need to Trust Research Again, complained that

Just over a year ago, I was disturbed to read the suggestion – tweeted by a teacher attending a ResearchED conference at the University of Cambridge – that education academics should be made to pay schools for access to research participants. I was shocked because education research was clearly not being perceived as a public good; something we should support in the way that we do other forms of research.

I'll say this: it takes a certain chutzpah to complain that education researchers should be any different from others and pay subjects for their time. If you can't do that, then the taxpayer funded research based on studying taxpayer's kids in taxpayer-funded schools should at least be made available to read after its completed, without a $49.95 access fee. It's not that I think this research is a public bad, it's that few understand it and I want to see that it says and means what those above me think it says and means.

This giant game of "telephone" is getting frustrating. I've named it the "Workshop Effect". Here's how it works:

1. Educational researcher (e.g., Kamii) presents results from her research (e.g., examining 3rd and 4th graders and the appropriateness of the common algorithm for subtraction) at large conference with consultants and workshop presenters in attendance. These folks take notes. Some completely understand what's being said, others less so. Not everyone is an elementary school teacher with a nerd-on for math.
2. Consultants and presenters then travel, collecting $3000 for a day's workshop in Central Vermont. The presenter has collected several sets of research results and displays them all. Superintendents and Principals from K-12 are here because that's $3000 and "let's make the most of it."  They pick up some details to bring back.
3. High School Principal hold faculty meetings or PD and mandate that "Research has shown that students should not be taught the common algorithm for subtraction." 
4. Curriculum coordinators and teachers spend months adapting curriculum to the new paradigm. Anyone who objects, or wants verification, is called "Anti-Team Player", a "Naysayer", a "Curmudgeon", or is criticized or written up for "not obeying District policy."

And that's how the rot begins.  Why should my 10th grade Geometry students be bound by research on third-graders, research that expressly states that it is done on 3rd graders? Nothing in the paper said that extrapolating 7 or 8 years held any meaning.

Underlying much of the critique of research in education is the charge that it doesn’t tell stakeholders “what works”. My first objection to the “what works” mantra is that this is based on a very insular view of what is important in education. My second objection is that it completely discounts the importance of researching what doesn’t work, particularly from the viewpoint of the largest stakeholder group: students. Nonetheless, the value of research in education is increasingly being judged in relation to the “what works” agenda: if something works, then there must be evidence to prove that it works. If there isn’t evidence (perhaps because the research is not about what works but what doesn’t), then that research has no value.

Maybe the criticism that Graham reads is like this, but mine is over not being able to see the original documents. I am not going to spend the money to download and read this research. I only got the Kamii paper because someone sent it to me (Grant Wiggins, Dave Coffey, Bowen Kerins? I can't remember). I understand that research often is intended to find a connection, a correlation, and that a cause is more elusive. I understand that sometimes we need to run the same study again and again to confirm (or not) previous findings.

The problem is in the interpretation and filtering that happens between the researcher and the teacher. What did the research actually say, and what can I actually take from it?

Teachers are now being encouraged to “challenge” education researchers for “evidence” to support their views. That’s OK – if the request is accompanied by an understanding of the research process and how knowledge is accumulated.

Sure. It's called peer-review.

It would be nice to be able to tease out findings instead of leaving it up to the ex-fifth grade teacher - turned curriculum coordinator.

Publish your work or face the criticism.

If you'll excuse me, I've got to get back to work.

Things We're Going to Need You To Stop Saying, part 5

False Dichotomy, aka. Twitter Broadside

Education seems to be full of these things, but perhaps they're in every business and I'm only paying attention to education. You see them often, pithy statements that fit into 140 characters by eliminating all the gray area and reducing everything to black or white extremes of "The Right Way" vs "What You're Doing". Often well-meaning but ultimately harmful:

If your exam questions can be googled, then you're asking the wrong questions.

Google is useful for information, less so for understanding. Googling the answer doesn't "show your work" and, given the nature of the Internet, isn't particularly trustworthy.

If kids in your class are more engaged by a fidget spinner than they are by your lesson, the spinner isn't the problem. Your lesson is.

Learning is hard. Kids fidget. Fads come, then go. Your lesson doesn't suck simply because two kids out of 25 are fiddling with this thing.

If your exam questions are multiple choice, then you aren't asking the right questions in the right way.

There's always a place for quick, multiple choice questions, even on summative assessments.

If your exam questions only use integers then they aren't Real World^(tm) Questions.
If your exam questions require a calculator, then you're asking the wrong questions.

Integral answers allow students to show their work, are useful to the learning process because the arithmetic is secondary to the learning. Integral answers can also encourage students to search for different solution methods. Decimal answers that require a calculator are great for Real-World data but Real-World data is often confusing and isn't usually appropriate during the learning process. Learn first, then use the learning. Calculators make guessing too easy and encourage kids to waste time with it.

If you are asking questions at all, then your students aren't agents of their own education. 

This is just silly. Teachers are there to teach. Sometimes the students "lead" the class down the carefully prepared road through the weeds ... but the teacher has laid the groundwork for that.


I am really tired of this nonsense. These blanket statements that reduce the complex world we teach in to just two colors (what you're doing and the right way) are unnecessarily reductive. It encourages simple-minded extremist fads that wither away after a couple of years of damage to children's education.

It's a false dichotomy and we're really going to need you to stop saying it.

After a while you give up.

ISS

Consider the following scenarios:

A conference at St. Michael's College in Burlington, Vermont. The speaker is Professor Robert Talbert (@RobertTalbert) and he is presenting his work with flipped learning to an audience of college professors and a few HS teachers. He speaks well, he is prepared, and he has given out material well in advance that he expects that everyone will have read, and questions that he expects everyone will have answered. Not surprisingly, everyone has. We discuss FL and his new book. We see what he has done with FL: what works well and why, and what doesn't and why. People work throughout the time, notes are taken, the food is lovely and the workshop is a success.

Or a Bio-Ethics conference at UVM Medical Center. Similarly, everything is straightforward, talks are given, information is presented, people listen and take notes. the food is lovely. The workshop is a success

A running theme throughout is the assumption of competence, treating everyone as if their time was valuable and that they were there because they wanted to be, that they wanted to hear what the speaker had to say.

Then, there's a conference geared towards high-school teachers.

Invariably, there's a bowl of candy. Your choice is used to sort you into groups. Most of the time, you need to stand up and hold hands with the next person and play "telephone" to demonstrate that ground-breaking idea that teenagers don't always listen attentively to your instructions.

There's a stack of post-it notes to stick to chart paper. Here's some colored markers. This presenter uses grade-school vocabulary and that sing-song voice one uses with 10 year-olds. That coordinator has 60 people crowd into a 20x20 foot space and then "Move to that side if you agree that that differentiation is a good thing, move to this side if you are wrong."

Did I mention that you have a Master's degree?

You'll be handed a paper copy of an article that was emailed out a few days before because "Not everyone is as advanced technologically as you" and couldn't be expected to have read anything ahead of time. I know, right? We can't expect anyone to know how to read something sent by E-Mail, apparently.

"Read this article and find the sentence that means the most to you."
 The "scribe" will write it on the chart paper.

"Find the phrase that means the most to you."
The "scribe" will write it on the chart paper.

"Find the word that means the most to you."
 The "scribe" will write it on the chart paper.

90 minutes later, you are done with a 3-page double-spaced article on the wonders of Common Core State Standards. You'd think we'd be further along in this process eight years after the introduction of CCSS, but no, we're examining a propaganda piece instead of developing course material to align with standards.

Did I mention that you have been teaching this material for 30-odd years?

Did I mention that the state of Vermont has decreed that all public school will be using the Common Core and that there's no particular reason to read about how wonderful it is?

I am not exaggerating. The whole thing would be demeaning and ridiculous but you get the feeling that the presenters aren't capable of anything more strenuous intellectually than 6th grade social studies -- your course work is gibberish to them.

"We're going to use the Gallery Walk protocol today."

"You have thirty-two minutes to discuss this topic in your groups of four. Person A will speak for 3 minutes with no interruptions. Following that, the table with remain quiet for 2 minutes to deeply consider what was said. Then persons B, C, and D will take 1 minute each to respond."

"There are easels around the room. Please take your Post-It note and attach it the the chart paper next to the statement that most closely matches your opinion."

You can resist only so much. After a while, you begin to go along with it all just to keep some progress happening. You know that if you ask a question, the gears in their heads will seize and jam and you'll never get anything done.

You dutifully watch the videos in the group, even though everyone is watching it starting at a different instant and the cacophony is making people twitch.  The video could have been viewed on our own time, but I guess not. In Bizzarro World, it's better to have us all use the limited time we have together to not work together. Again, it's not informative; it's a college kid's project touting the glories of the work we're going to begin doing someday.

*HeadDesk*

Then, you'll write down what color the video was ... I chose "FUSCHIA" because it included lots of letters from the word I really wanted to use.


It becomes easier to let them ramble, to let them play their games of "Pass the ball of yarn back and forth and then it will represent the network of caring that we have here." You resign yourself to never getting anything actually finished during inservice. Proficiency-based grading isn't slated to be implemented for another three years because "some people can't even use Google Docs" and "I'm trying to teach the faculty how to fill in my template for learning" - you know we should be doing this more quickly but it's easier to just give up.

Let's call it: "Inservice Stockholm Syndrome."

How to be a Math Coach

As a long response to Math on the Edge, I have a few questions for prospective and current math coaches. Hereafter, I will use the words "we" or "us" to refer to me and/or to the "teachers who you plan to coach".

Here are nine questions. If you can't answer "Yes" to at least one of these nine questions, then you have no business being a "coach".

Subject Matter Knowledge:

1. Do you know more about math in general than we do?
2. Do you know more than we do about a specific topic in our subject?
3. Have you done research in our field that would prove useful to us in the classroom - a new understanding, a new meaning?
4. Have you spent time creating content that other math teachers have found useful?

Relevant Experience

5. Do you have experience (that we do not have) teaching our subject at the grade levels we teach?
6. Do you have significant experience teaching our subject at any grade level?

 Teaching Techniques

7. Do you know teaching techniques that we do not - new ideas that we could learn from and improve with? Theoretical knowledge is useful.
8. Do you have a lot of experience teaching anything at any level - experience that might inform someone in a completely different situation? Practical knowledge is useful, too.
9. Have you done research into teaching that would prove useful to us in the classroom -- cognitive research into children's learning or a change in the philosophy of classroom management or the use of a new technology that makes a true difference -- to the extent that you know this work, know what we might do to improve?

"Yes" to any one of these? Great. If not, don't expect that any of the people you are pretending to "coach" will take you seriously.

If you said "No" to all of that, you are not a coach ... you are an intern and I am coaching you.  I may learn a lot from you, but only as a test subject or a guinea pig in my experiment.

Coaches teach their teams, guide their teams, or at least pressure them into doing things in a certain way together. These people make the team successful, improve the team. If they don't, they're fired. If this isn't what you think you're doing, don't call yourself a coach.

For 1 - 4.
I enjoy talking to, and working with, people who know more than I do about a topic. My education was in engineering, not in pure mathematics and not in secondary teaching. It is refreshing to have Mike Olinick in to give a cryptography lecture to an auditorium full of high schoolers or Jeff Suzuki to give one on an important aspect in math history.  If I could convince Don Steward to fly over and work with us, that would be very cool. Dan Meyer's a good speaker and a good coach and he has ideas that resonate. These people all have something in common: they've worked with math in ways I haven't. I can learn from them and others like them. Even if their contribution is a different way to work with math, like Fawn Nguyen's Number Talks, or visualpatterns.org - these are people from whom we can learn.

If you haven't worked with math in ways that I haven't, if you don't bring anything new to the table, if you aren't improving our/your instruction in SOME way, then you have to be outstanding in some other way.

For 5-6:
If you don't have any experience, what can you offer me? What are you basing your "coaching" on if you have never done any teaching yourself? The bar is higher for you because my field is so very different and my difficulties and problems are not solved by amateurs and dilettantes.

What do you base your "advice" on, if your sole experience is teaching a existentialism course at a small New England prep school? Your research? Your work as a content aggregator? Your charming personality? If your opinion contradicts my own experience, then you need to really convince me that you aren't just another blithering idiot on the Internet.

For 7-9:
If you can't say WHY your new idea/technique is a good one, then I'm not going to take your word for it. I might implement it in my classes but only after considering how the change will play out, what I might do to measure its effectiveness, or at the very least consider how the class is currently running so I have a clear and concise memory to compare the "after" version to.

If all you have is a "yes" in 7-9, then you're gonna need your research, your "proof" or at least a damned good explanation. I want to see that your research/evidence applies to my discipline and my grade levels. Don't tell me that I need to change my HS math classes based on research done on K-2 students. I look very skeptically on any statistics that include Singapore and Hong Kong, or Finland and Norway.

I understand that I am not completely up-to-date on everything to do with teaching. I'm using technology a lot, in some very interesting ways, and my students are doing things I'd never imagined in the 80s and 90s. I read others in the #MTBoS and I travel quite a bit to hear them speak. But I know there's a lot going on out there and I'm in the middle of a very poor section of a tiny state.

SO, if you're going to come into my classroom and tell me to change, for example, to Proficiency-Based Grading, then you had better understand it yourself. If you are going to tell us to re-imagine our teaching in a certain way, then you NEED to be competent in its variations and minutia.

If not, then you're not a coach, you're a pain in the ass.

DO NOT do as our curriculum coordinator did: raise up a book at the first *weekly* in-service (2.5 hours each) and announce that you had just read this idea in this book and you think we all should do it. If you haven't been trained in ThisWork, haven't done any work in or research in ThisWork and your only teaching experience is as a less than stellar elementary teacher whose "specialty" was social studies ... then you shouldn't be a math coach.

DO NOT speak to a faculty meeting and say, "I don't know anything about what you're doing but I've just been hired for this new job to help you with your teaching. Here is a list of topics that I can help you with by *Googling* them for you", which is what two fresh new admin did.

DO NOT insist that teachers adhere to the practices of your 4th grade classroom when you are attempting to "coach" us in these new ideas that you don't particularly understand. We're all adults and most of us have master's degrees and we average 15-20 years of experience in our fields, some of us more than 40. You're not being cute. You're not being clever. You're being a pain in the ass.

DO NOT walk in and say "I'm the Math Coach" and proceed to tell us nothing new, even going so far as to use material from *my own website* in your presentation (true story).

I could continue but I've got to get back to work.



TL:dr;
Bring something to the table or GTFO.

Grading and Homework

from Justin Tarte, one of the signals that there is a grading problem in your classroom is:

When talking with parents at parent teacher conferences (which honestly need a complete overhaul by the way) you find yourself telling multiple parents that their child would be doing much better grade-wise if they would just do the homework.

It struck me that there are two ways to parse this.

(1) That the child was not doing the homework and the homework is graded so the zeroes bring the grade down even though the student understood everything and could prove it ... OR
(2) that the homework is key to building automaticity and understanding, and that by not doing any of it, we shouldn't be surprised that the grade, which is a reflection of the understanding and ability to use the material in new ways (proficiency) was an indicator of that lack of proficiency.

Is it any wonder that I hate professional development that takes such a simplistic and uninformed statement and builds policy around it ... despite the ambiguity inherent in the statement?

"Stop grading homework!"
"Stop giving homework!"
"Homework is counterproductive!"
"Proficiency-based Grading explicitly rejects grading homework."

Leading to a blanket policy across the board:

Is this really a good idea?