Incorrect Data isn't Useful

The other day I went to the Health Center for a followup checkup. I had been in previously and had gotten some anti-biotics for an insect bite that got infected. Simple, right? As part of the visit, the nurses are instructed to take routine weight and blood pressure measurements.

I know my blood pressure, so I was surprised that her diastolic reading was 20 points lower than it normally is. I remarked on that. Her reply was "Lower score is good, right?" in the tone of voice that conveyed clearly that I shouldn't be questioning her.

I'm thinking, "Sure ... unless it's a bad measurement." I get that BP is inexact, but it's a bit silly to refuse to re-measure it when the patient points it out. 20 points can make all the difference to the doctor's diagnosis of my overall health.

I decided that I would request the printout from the front desk as I left, the one with all of the day's numbers and decisions from the visit. I read the scale ... same weight as two weeks ago. On the printout, though, it was different ... she had obviously transposed digits when entering the data. In two weeks, I had "gained 23 pounds" and then lost it again in the 30 minutes it took to drive home. My blood pressure changed by 20 points, and that's a lot.

Bad data makes for inappropriate diagnosis.

Bad data makes for bad education policy, too.

The state of Vermont is "suffering" through the release of the first round of SBAC scores despite our scores being better than most other states (we're usually top 5).  "Results are much lower" and already my principal is bitching about it, despite declaring at the time, "We don't care what the scores are, we just want to get the process right." (I'm paraphrasing but that was the intent.)

I'm all for improvement, but I hate basing change on the back of bad data.  Our diagnosis is flawed because our data is flawed, and the prescription runs counter to other policies that the State has imposed.

First, the SBAC has measurement errors just like my nurse had.  Many students took that test knowing that scores would not be held against them, that there was absolutely no chance that anyone would see the scores in fewer than six months or act upon them to set courses for this year or college applications. Additionally, the test itself is drastically different in format (and it's all done through the Chromebook) ... a test completed entirely on-line.

There are no multiple choice questions and kids can have scratch paper, but they're not used to doing math that way. There's a lot of "drag the factors to the answer box" and write three paragraphs explaining why you know that this is a straight line .. and few can stretch out an explanation that far.

Second, and just as  important, the SBAC "passing scores" were decided upon after the fact, to make the percent-passing numbers match what the state had decided they should be ...

That's right. Before the kids even took the test, they told us that there would be a state-wide passing rate of 33% on the HS math test. Then they set the cut-score to match.

Third, add in the fact that we are a small school and we pride ourselves on being able to provide a more personalized education that your average public school, including having personalized learning plans that had quite a few students taking Algebra 2 as seniors. I'm sure you can see where this is going: many of our kids were taking a test heavily based on mathematics they hadn't seen yet.

This runs directly counter to another major initiative in the State of Vermont, the Personalized Learning Plan. Sometimes called "Personal Pathway to Graduation", the initiative requires schools to design different course pathways to graduation for each student as appropriate. This includes allowing schools to schedule certain kids into a faster progression for math and others into a more moderately paced path that might not even include algebra 2. It means that "pre-algebra, algebra 1, geometry, algebra 2" might be the most appropriate for a student.

Taking that approach and then complaining that they don't know algebra 2 by March of their junior year is silly.

It's the rhetorical equivalent of reading a graph that says that Pre-calculus students do better on the NAEP and then concluding that we must make sure every student takes Pre-calculus by the time state tests are given in the junior year
 ....

which a previous principal actually said.

So when the bright bulb in the room points out that we teachers should prepare the kids for college and careers, and should have prepared the kids better for this test, and "If you hold the kids to a higher standard, they'll rise to meet that standard," I will calmly channel Dick Cheney and say that "you go to war with the students you have, not the students you wish you had."

Finally, the teachers are not allowed to know what's on the test.  I don't want to teach to the test, but I'd like to know what is included.I'll give the same assessments I would already have planned, but I might change some questions to a similar format, for example.

Also, I'm not willing to just take their word for it that the test is appropriate. We can't check for bad questions that might have tripped up our students and we can't check that the answers they gave were correct or not. We have to take Pearson's word that the scorers actually knew what they were doing.  After reading Todd Farley's book, Making the Grades: My Misadventures in the Standardized Testing Industry and others with similar tales of the realities of corporate test making and scoring, I'm not particularly willing to do that.

The NY Regents is an example of a relatively open and transparent test-making system, but it has many errors. The NY teachers can catch these problems and get them fixed. If we look at Mr. Honner's long-running series reviewing the NY State Regents exams in mathematics, why should we expect that the SBAC tests will somehow be perfect if there is no chance for oversight?

The SBAC is a closed system with no accountability that scores the tests in strange ways, fails to take into account the realities of the students, will not allow anyone to analyze or even examine any of the questions (unlike the SAT which I can see in its entirely within a few weeks), spits out pre-determined results that do not reflect student abilities, and makes everyone wait an unconscionably long time for those results ... much too long for the school to do anything with them.

I can't use the scores because they aren't detailed enough, timely enough or accurate enough.

I guess I'll just teach math and ignore all that bluster.

SAT scores are linked to Family income. So what?

Bumped to the top from 2009


Just a warning ... I'm going there.

Flypaper, the edExcellence Blog, has commentary on the SAT scores that refuse to go up.

"Gaps widening a bit by race, income, parental education. Indeed, the tidiest relationships and smoothest curves are those that continue—as they have for as long as anyone can remember—to show the steady upward progression of average SAT scores (pdf) as family incomes and parents’ education rise."

Chester then goes on to show some details and to say,

What does this say about 26 years of education reforming since A Nation at Risk? For starters, it says the reform efforts haven’t seriously penetrated our high schools. Then it says that current moves (e.g., the “Common Core” national standards project of the governors and chiefs) to align high-school exit expectations to college and workforce readiness are urgently needed, indeed long overdue.

That's an interesting take. If you can't get results in 26 years of trying reform after reform, let's try another reform! (Insert sound of Bells and Whistles). Then there is the sentiment that "Reforms are obviously urgently needed."

It's true. The only correlated data that ETS has with an R-value greater than 0.1 or so is between scores and family income. The NYTimes has this:

Everyone, of course, dismisses that income-scores correlation as silly, saying "You can't just give the family more money and raise the scores, ha, ha." That's right, but I don't think that money causes good scores, although there is a lot to be said for SAT prep courses, which are really a total review of math and basic grammar and that's good no matter what.

I think the problem is an incorrect correlation, a confounding factor. It's not that A (money) causes B (scores). It's that C causes D which affects B. Simultaneously D causes A.

Parents are the confounding factor.

As I see it, smart parents are likely to have smart children. More importantly, motivated, dedicated, educated and intelligent parents are likely to have M, D, E, and I children. They are also more likely to have money because they work harder for it and are more capable of holding the job and moving up the ladder to the big bucks and higher family income. Simultaneously, their M,D,I, and E children are more likely to have better scores.

Children whose parents were poor because of misfortune or some other external factor don't tend to fit this pattern. They're the Horatio Algers of the world, the kid who worked his way up from the mailroom to CEO. There's no reason to assume the black or Hispanic kid can't be doctor, lawyer, etc. In fact, the kids of those successful (and higher income) blacks and Hispanics are likewise high-scoring and successful. The single mother who instills dedication, motivation, and a healthy respect for education into her kids might not have money but her kids will.

Children whose parents were not motivated, etc., always fit the income-score correlation. Race has rarely been a factor in determining motivation and dedication (simply look at KIPP schools to see that), but it has been an indicator of income due to longstanding segregation and migration patterns. Money doesn't seem to drive scores, but it does correlate.

The constant desire for improvement and reform and reform and change and reform again is doomed to repeat its cycle of failure.

Are reforms necessary? Sure, if you can point to a definitive improvement that will result. I'd appreciate it if you'd define "improvement," first. Improve what and by how much? Student satisfaction, tech-toys, test scores, athletic titles, graduation rates, future wages? Fundamentally, is "success" as defined by "average scores increasing yearly" or "adequate yearly progress" possible? I don't believe so.

I think we should stop looking for the perfect reform because it doesn't exist. We should instead focus our attention on doing the best we can each year with those we have in front of us.

Just sayin'.

Correlations in education

Scott Macleod said on Twitter: From my doc student's dissertation: Student w/ 2 years of high school 1:1 said "I know way more #edtech than my college profs!" #true #sad

Does that say more about the profs, the education or the power of correlation?

Data-Driven is Driving out the Good Ones

from Jay Mattthews:

Cotton worked hard. He said his evaluations from classroom observers got better as the year went on. But his students failed to outdo their suspiciously high fourth-grade scores to save his job. Those test results counted 50 percent in Cotton’s final evaluation.

He might have been a lousy teacher, but most first-year teachers need a lot of practice to get this job right. It takes about three years to catch up to the standard. It makes no sense to toss him under the bus.

The data requires me to be 
stupid, that's why.

The test maker sees evidence of cheating in the previous year's answer sheets, for a "suspicious number of wrong to right erasures". A former teacher is fired for cheating, or as the administration puts it, "is no longer at the school" because of  "a lapse of integrity".  What to do?  Fire the new guy because his more honest scores are too low, because you can fire a probationary teacher without cause or reason, because you will be "doing something", because "we will turn this place around".

Decisions must be data-driven instead of intelligence-based, apparently. 

Sleep and Academic Achievement.

Why didn't they use a multiple bar chart?

Joanne Jacobs says "While federal guidelines recommend 9.25 hours of sleep each night for students, that could be too much for teens, concludes a new study, which correlated students’ hours of sleep to reading and math scores."

The study found the optimal sleep amount for 10-year-olds ranges between 9 and 9.5 hours, while for 18-year-olds it is slightly less than 7 hours. At ages 12 and 16, children need between 8.34 to 8.43 hours and 7.02 to 7.35 hours, respectively, the study found.

First, very few people are capable of determining how long they stay asleep because they have no way of knowing when they fell asleep; the study was based on numbers that were self-reported (by children!). "There is heaping of sleep hours in the data on 0s and 5s. The distributions are computed with a smoothing parameter of 0.85." If you can't measure sleep time empirically, your data becomes suspect. To be fair, the researchers acknowledge "The first issue concerns the non-experimental nature of our data. Because our data were not generated by a controlled experiment, we cannot rule out the possibility that we are measuring a correlation between sleep and test scores rather than a causal relation."

While this is interesting, it's not too useful.

Second, there was no control for studying time. Were those students studying if they slept less? Maybe the 18ers who are getting 6-7 hours of sleep are the ones studying and learning for hours while those who are sleeping for 9.5 hours are working less hard. Maybe no one is studying. Maybe they all are. Maybe the school should start at 8:30 instead of 7:30 (with a one-hour bus ride before that).

"To determine the optimal amount of sleep, they compared standardized test scores in mathematics and reading with the self-reported number of hours students were typically sleeping each night. "

Without knowing WHY the students are sleeping more or less, this is just another useless education study, listed in the advance online publication section. The results were released and everyone jumps on it to further some educational fad or just to make copy deadlines. "The study's results raise the issue of whether students receiving too much sleep may see a reduction in academic achievement. While more research is needed, the authors conclude that this is possible." Actually, the researchers don't get anywhere near that definitive a conclusion.

Let's look at their aggregate data; maybe that will clarify things.

The hours of sleep range from 4 to 12, with σ = 1.45 and x̄ =8 so we have a pretty narrow window here but it's still worth exploring. Income isn't ... wait ... mean income is 67k, range is 1.4 million, σ=$92,000 ! Wow, I sure hope they controlled for income, but a look through the report doesn't show any indication that they did.

I know where I'd start round two.

Fallacy of Relevance

Here is some material for an exploration ANYQs. You have to be careful with medical issues because kids misinterpret and will sometimes take this as a criticism of their lifestyle, parents, capitalism, world-view.  They will also assume that I'm out to bash big Pharma (why they care is beyond me).

I start with the first article.

From the tone, it is obvious to someone that OMG OMG OMG OMG OMG

Fake drugs are increasingly being sold on the Internet in a global counterfeit medicines market that has doubled in the last five years to more than $75 million. The medicines, many of which are life-threatening, have even turned up in the legitimate supply chain and found their way into pharmacies, according a review by Dr Graham Jackson and colleagues published in the March issue of the IJCP, the International Journal of Clinical Practice.

Is that a problem? For most kids, that scares the germs right out of them until you line that statement up against this one:

Accounting for 6.5 percent of the total market share, statin drugs are the most widely sold pharmaceutical drugs in history. To date, Forbes Magazine tells us that statins are earning drug companies $26 billion in annual sales. ... Pfizer spends over $3 billion each year to convince us that we need more and more drugs to be healthy.

And that's just for Lipitor, the cholesterol-lowering drug. So, if $26 Billion is only 6.5% of market, that means the market is worth about $400 billion.

Aside: Pfizer pushes their drugs so hard that it incurred a $2.7 Billion fine for health care fraud (partly the largest criminal fine ever in any case in the US and partly for violations of civil law, the False Claims Act.) Pfizer sales in that same year: $48 billion. Pfizer profits in that same year, after paying that fine: $5.7 Billion. Much of that despite the fact that many other countries force Pfizer to charge far less for their product, but we digress ...

And then you find that the other, "much more acceptable", fake drugs have a fairly strong market as well:

The market for homeopathic and herbal remedies increased 17% from 2005-09 to reach $5.9 billion. As these once considered “alternative” remedies continue to transition into the mainstream, Mintel expects growth to continue at a steady rate, averaging 3.5% growth annually through 2015. This report explores the market for homeopathic and herbal remedies, looking at sales across all channels, as well as consumer usage habits and attitudes towards such remedies. Topics covered in this report include:..

The report seems interesting, but I did not want to pay for that report to find out more details; the price was £2,464.53! "Add to Cart?"  .... um, No.

That's the United States. Now multiply by 5 (maybe?) to get the Global ... $400 Billion for the US * 5 = $2 trillion.

75 million in (possibly) fake vs 2 trillion. Which means that fake-fakes are 0.00375% of the market. I'd interpret that as something to be improved but pretty damned good.Acceptable fakes (homeopathics), on the other hand are 0.295% of the market, nearly 80 times as big a problem.

Kinda makes that $75 million fade into insignificance, doesn't it?

Samples must be numerically significant or else conclusions are worthless.

Parents Against Tired Truckers is losing it's religion over the results of a new study on an experiment up here in Vermont. Just as in education, small sample sizes and incomplete data are being misunderstood and misrepresented to further a viewpoint that may do more harm than good. PATT has it's heart in the right place, but it's brains are sorely lacking.

Vermont asked the Feds to study whether allowing 100,000 lb rigs on major highways would be more dangerous than having them travel the back roads.

The study results came out. PATT shouted

The Trucking Industry Is Wrong on the Maine and Vermont 100,000 lb. Truck Pilot Program – DEAD Wrong 

Wow. That must be some study. "Dead wrong" isn't mincing words.

" .. the Truck Safety Coalition (TSC) released startling information revealed in response to a Freedom of Information Act (FOIA) request sent to the Vermont Agency of Transportation (VTrans)."

Impressive. "Startling", you say? Took a FOIA request, huh? I must read further. "Catastrohic results" "People needlessly died." Damn.

The number is SO BIG.

Documents show that during the 100,000 lb. truck pilot project in 2010, Vermont’s commercial motor vehicle fatal crash rate tripled from .49 fatal crashes per 100 million miles traveled in 2009 to 1.44 fatal crashes (“Vermont Truck Interstate Pilot Study- Report to Congress (State of Vermont Version for Review) – Summary Report (Draft)” prepared for FHWA by Cambridge Systematics, Inc, hereinafter “Vermont Report”).

What does that mean Regis? The Death rate tripled.  Holy Batman, mackerel.  They're quoting Government documents and it sounds so official.

Well, actually, it doesn't mean much at all. You see, Vermont had one death involving trucks on its roads in 2009 and three in 2010. Yeah, the death rate "tripled" but you need to have a bigger sample size before you can claim that trucks are making things more dangerous.

You also need to look at the reality of those crashes. In the one crash, two trucks and a car were involved in an accident that was blamed on icy roads and bad conditions. One of the truck drivers and the car's driver were killed. In the other accident, the car (probably drunk) crossed the 50-foot median and hit the truck head-on. Again, hardly the fault of the truck driver.

As in education, there's always some fool trumpeting results based on small sample sizes and assuming the study will scale up. Remember when Bill Gates spent nearly a billion dollars to create the Small Schools Initiative? The smaller schools that did better than the large public schools were showcased until the next year when the same school would do worse, at which point the deformers would shout about some other school which HAD done well that year. Variation of the small groups, not the inevitable superiority of the charter school, small-school, voucher school, Catholic School, whatever.

To give you another example, consider Daisuke Matsusaka (RedSox). He had four starts. Two were terrible and then two were decent. Can we say that trend is positive? Yes, but I'm not giving him a contract based on that.

Still another comes from here:
Last week I tossed a coin a hundred times. 49 heads. Then I changed into a red t-shirt and tossed the same coin another hundred times. 51 heads. From this, I conclude that wearing a red shirt gives a 4.1% increase in conversion in throwing heads.

Pretty foolish.  Besides, everyone knows that wearing a red shirt is tantamount to a death sentence anyway, so I'm not sure what can be made from this "study" either.

Government is shrinking

Interesting graph out of Silicon Valley. The Federal workforce as a percentage of the private workforce has been dropping for decades.

The spikes seem to be census-related.

Selection Bias at KIPP again

It's getting rather repetitive. Lots of people have said it, Jim Horn, for one. It seems as though the journalists aren't listening, or at least are swallowing the bait hook, line and sinker when it comes to the press releases from that grandfather of charter schools, KIPP.

Education News put out this blurb committing that journalistic sin: copy without critical thinking.

As of Fall 2010, 33 percent of students who completed 8th grade at a KIPP middle school ten or more years ago have graduated from a four-year-college. This rate is above the national average (30.6%) and four times the rate for students from low-income families.

It's never been about the money and it certainly isn't about the KIPP program. Socio-economic status is an indicator of educational success not because money is the key factor but because selection is.

What kids are likely to apply to KIPP, suffer through the selection process, stay in KIPP throughout the 8th grade, have the ability to maintain focus under pressure, put up with the incredible stresses and challenges demanded of them, and attend the mandatory summer and Saturday classes? These same traits are a damn good indicator of college success, too.

KIPP isn't the key to their success - it's merely the spotlight that shines on them, taking all the credit for their success while ruthlessly weeding out any child who isn't up to the standard, in much the same way that the average height would increase if you shot all the short people.

Education through academic eugenics.

Testing, Scoring and Trusting the Data

A fascinating case study is the NY Regents, scoring, and the unintended (or purposeful) consequences of a line in the instructions to the scorers.

First, here's the graph of the number of kids getting each score (from WSJ).  The issue can be seen quite clearly. The graph overall is a typical left-skewed distribution, as you'd expect from this type of test. The trouble comes when you explore that jump in the middle at the passing mark.

Of course, the nattering class is all up in arms over this, claiming fraud and misconduct.  You can almost hear jeers of "Union Bastards trying to save their jobs by lying on the tests."  Unfortunately for those people, the reason comes down to one sentence:

"[State officials] note that the state actually requires teachers to regrade certain Regents tests where the student barely fails in order to check for grading errors."

When you are singling out tests for special consideration, and the stated focus is to look for "scoring errors" on "barely failing tests", the only way for the scores to change is up.  Since it's easiest to give one or two points to a 64 or 63, it's logical that those would be the most effected.


Looking at the graphs, you can see that some teachers (probably a school at a time) set their cut off at 50 while the majority set the cutoff at 55.

Why the negative slope in that region? When looking for ambiguous answers which could be scored higher, you have to "rescore" each problem until you get an appropriate number of points. It's easier to find one such than 10 such.

Similarly, since there was no reason to check more answers than just enough to get the kid to pass, the uptick was only to 65, though it seems that many teachers weren't keeping close track of the extra points and brought the kid up to a 66 or 67.

Conversely, if the teachers had been instructed to rescore all students within ten points of the cutoff instead of just those who were below it, then you would have seen some students adjusted downward, balancing out much of the upward movement.  If there were a similar uptick at the 65 mark in this hypothetical, then and only then can you claim that teachers are deliberately mis-scoring to jigger their VA measures. (and even then, I'd put the reason as teachers wanting to help students rather than being so coldly self-interested.)

The place where I found the link to this article had this comment: "Teachers don't want to flunk kids that just barely miss the passing score. Until all responsibility for creating and scoring state exams is given to an independent body with no interest in the results of the tests, the results reported should be viewed skeptically."

Ummm, no. Scoring tests is really complicated.  Pearson, the biggest company, uses part-time, barely out of college, minimum wage people to do the scoring. Getting the "right" score is more a matter of whether or not the scorer speaks English and actually knows the material.

Frankly, given the mess that the testing industry is in when it comes to scoring, I have a feeling that the teachers are doing a more conscientious job. If you want a nasty introduction to the follies of testing company scoring sessions, check out Todd Farley's "Making the Grades." It's well-written but damn depressing if counting on accurate test scores because you're stuck in the hell of value-added and merit pay.

It's a Rosy Picture, Rotten at the Core.

I shouldn't need to add much to this:

New York City’s top-ranked school is under investigation for cooking the books, reports the New York Times. Theater Arts Production Company School, a middle and high school located in a low-income Bronx neighborhood,  graduated 94 percent of seniors, more than 30 points above the citywide average. The school earned a near-perfect score in “student progress,” based partly on course credits earned by students.  The school’s no-failure policy requires teachers to pass all students who attend class, regardless of their performance; no more than 5 percent of students can get D’s.

In practice, some teachers said, even students who missed most of the school days earned credits. They also said students were promoted with over 100 absences a year; the principal, rather than a teacher, granted class credits needed for graduation; and credit was awarded for classes the school does not even offer.

When you tie teacher and admin pay to "performance", tie school survival to "attendance" and "graduation rates," it's not a stretch to imagine why people would lie. Should we blame Klein, Black, Bloomberg, Duncan, Obama, GWBII?  Some combination of all of them? Or just go firing school officials?

Ray Allen wins again.

Funny guys, those Boston Celtics. Winning on an awesome Ray Allen shot and immense Shaquille O'Neal shoulders.

And yes, that tweet makes total sense.

BTW,
Reggie Miller: 2560 three-pointers.
Ray Allen: 2533 three-pointers. 13-21 shooting threes for the last five games. When's the record gonna be?
Data below the jump.

PISA scores - another look.

What do you see? When you look solely at schools with fewer than 10% of students on FRL (i.e., poor), US schools would be better than those of the top of the chart, Korea. When you include the whole spectrum of US schools and their students, the US is much lower.

The US has a much bigger spread than any other country (the largest standard deviation of wealth in the developed world).

Our overall scores are unspectacular because we have a high percentage of children living in poverty, over 20%. This is the highest among all industrialized countries. In contrast, child poverty in high-scoring Finland is less than 4%. For Cleveland and for the US as a whole, the major problem is poverty. Before we worry about teacher quality, institute longer school days, and increase testing, we need to make sure that all children are protected from the effects of poverty: This means adequate health care and nutrition, and access to books. When we do this, American test scores will be at the top of the world.-- Stephen Krashen

On jumping to conclusions and historical data.

In discussions that refer to data about historical topics, the non-math folk often fall for the correlation - causation error, as well as misinterpreting their evidence.

It is generally accepted that height actually declined somewhat in late period and early modern times (see, for example, preserved sites like Williamsburg or Jamestown, where door frames and hallways are significantly smaller than nowadays), the reason having to do with more widespread malnutrition and chronic health problems emerging in the 14th century and continuing on until the 18th century. (from a mailing list focusing on medieval history.)

Average is a really tough thing to deal with here and there are a bunch of problems with this statement.

Let's deal with averages first. It's pretty easy to find the average height of modern people. They're alive and measurable. You call together twenty folks at the mall and ask them their height. Convenience sample leads to selection bias, so you retry with a cluster sampling. That's better, but did you happen upon the community of immigrants who are shorter? SO you expand your search and try a more systematic approach. In the end, you have a sense of the average height.

Now try that for the 14th century. Your sources are whichever bones you've been able to find and measure and a few written claims. Not much for numbers there.

Second issue is that pre-19th century doors, hallways and buildings weren't built for standard-sized people. They were built for the occupants and those occupants had different needs. Stating that door frames and hallways are smaller than currently because of a lesser average height is a conclusion that doesn't necessarily follow.

Your average tells you about the group but not much about individuals, so your building codes are set for the 99.8% who are shorter than +3 standard deviations from the mean so that no one is inconvenienced. But what is the average height? That's easier to tell now, but not so much for earlier demographics.

Room size does not correlate to occupant height either. My classroom ceiling, for instance, is nearly twelve feet high. Can we really conclude anything from that? On the other hand, all of the occupants for the last twenty years have been shorter than 6'5" and half of them were averaging 5'4" (the girls). All classrooms built in the last twenty years have 8' or 9' ceilings, becuse of changes in the building code, not because people kept hitting their heads.

The windows are huge, nearly 3' by 8', but that speaks to the need for free light and air rather than a race of giants.  Need and purpose are better indicators of door size in pre-code days.


The typical pre-1850 house has low ceilings and doors I want to crouch for (but that are still taller than I am), for a couple of reasons.

A room with a low ceiling is easier to heat, less difficult to build, and uses less wood in the building process. A 7' ceiling is not a problem for anyone who was going to use it - the builders were making houses for themselves and they didn't feel the need to give themselves 2' of headroom.

Sure, it's shorter than today, but only relatively. We notice because the standards have changed and it FEELS lower, not because we need to duck.

The same is true for the door. It's shorter. It was custom-made and of varying heights. Often it wasn't "square" but rather built to fit the space rather than the other way around. It might have a corner cut off to fit under the eaves or clear a stair. In any case, the door was built for the prospective occupants, not the rare fraction who needed a 6'8" door. Andre the Giant could bloody well duck.

Likewise hallways. What's the need for a wide hallway? It's just wasted space. Your big furniture was on the main floor or was carried up in pieces and assembled in place. There were no ADA-type requirements. Again, we feel constricted, but the person of that time would consider our modern requirements to be ostentatious displays of wealth. The typical house of the 15th century was maybe half the square footage of 20th century houses, and that is being dwarfed by 21st century ones. This correlation is with wealth, not height.

Other factors chime in: the cost or taxes assessed on road (or canal in this case) frontage might dictate your building style. For an extreme example of this, check out houses in Amsterdam: narrow, almost useless staircases and a hoist mechanism outside for lifting pianos and furniture to the second or third floor.


The takeaway for math teachers is to stress that we need more than correlations to justify cause. I might state the original paragraph and have the kids try to punch holes in it. Critical thinking and all that. I'd be looking for points like those above and:

• If the claim is that people were taller before the Templars' suppression at the hands of Philip the Fair, became shorter during the 14-17th centuries and then started getting taller again in the 18th, then your doors should show the same trends for the same time periods, in the same types of houses and at the same socio-economic levels. If the doors to churches showed the same decrease and increases, that would be a good sign.
• How could the builder know the average height of 15th century mankind? Communications were difficult and slow, the size of the foot varied by region among other factors. Besides, even if he did know, why would he care? He's building for the man in front of him, not some mythical beast. Is there evidence that builders knew the average?
• Is the rural house different from that in the city?  See the Amsterdam note above.
• What are other reasons for the differences do you see? 

• War.  Make your attackers duck as they enter. Some Indian adobes have four foot doors for this reason. This picture is from Doune castle.
• Security. A smaller door is easier to bolt closed.
• Religion. Make your visitors bow in humility as they enter.
• Practical. They wanted to reach the ceiling beams for storage hooks.
• Sexism.  Only women used this door so who cared about it being more than 6'? 
• This was where the children slept.  There was plenty of room under the eaves.
• Mathematical considerations:
• Can the average height be measured?  Is that number meaningful here? Should the anthropologist use median or mode instead?  What was the builder using?
• What kind of sampling would work best for ascertaining that measure?
• Historical considerations:
• How do we know that people did get shorter? Did they or is this a myth?
• If they were, was it due to nutritional deficiencies? Were chronic health problems all that widespread in just those four centuries or merely the Black Death taking all the headlines? Was the 9th century healthier?  How about NYCity in the 1800s tenements?  London's Soho at the time of cholera epidemics?
• Were the doors shorter during the time period in question? What was he looking at when he measured?
• What would the students do as step two to confirm or deny these theories? Why haven't they started?

I love math.

What happened to "Education"?

Joanne Jacobs has a piece on evaluation programs that are looking to videotape teachers as they give lessons and then "Go Look at Tape." The issue seems to be that

More than 99 percent of teachers are rated satisfactory by their principals, reports a study on “the widget effect” by the New Teacher Project.

and this is somehow "bad," hence the need for overhaul. While I would argue that you need to improve the system so that middle-managers (Admin) can appropriately rate, measure and evaluate their assembly-line workers (teachers), I'm not sure that going to videotape is much of an improvement. If the Principal can't see the "errors" while sitting in the class, how is he supposed to make anything out of the 17th such review, even if he did have the time. Well, they have an answer for that ... Gates figures we'll pay outside evaluators who will somehow be able to see what the highly paid admin can't or won't.

The Gates Foundation is developing a new model, with the help of social scientists and teachers, reports the New York Times. Outside evaluators analyze videotapes to determine whether teachers are teaching well.

We've got money for outside evaluators? Pretty cool. I'll sign up. What gives you the idea that I'll be any more or biased/petty/superficial than the current system? The fact that I'll be able to rewind the tape? More likely I'll fast forward through it. The fact that I don't know the teacher so I'll be more honest? More likely, I'll make a snap judgment and move on.

You know we'll pay for training out of our RTTT money or something, hire bunches of consultants to teach teachers to evaluate other teachers who are trying to teach students. Pretty amazing, ain't it?

Hundreds of teachers will be trained to review 64,000 hours of classroom video. They will look “for possible correlations between certain teaching practices and high student achievement, measured by value-added scores.”

Let's say it's 800 teachers - that's 80 hours of tape to watch, per person. Asking a lot? I can't even spend that much time watching TV or movies in a month while sitting mindless and half-comatose. I'd need to be unemployed to look at 80 hours of tape multiple times and evaluate someone.

But here's the crux of my complaint ... those correlations. I've had students who completely and utterly bombed in class but did well on testing. I've also had plenty who did well in my class, well in the next couple of classes, had a great high school career, and then a great college career, and are now cranking out 6 figure salaries ... but didn't do well on testing. How do I know they did well? They told me.

Since when did value-added scores on NEAP mean an education? Since when could anyone figure out what I do that "works" by watching 10, 20 hours of videotape? Really? Do they know me? Since when did someone watching a few hours of video actually have a clue as to what went on and who and how the students were effected? Ten hours of video? That's two days. Does anyone here think that I couldn't fake it for two days and mess around the rest of the year if I was of a mind to?

I didn't think so.

The Further You Extrapolate, the Sillier You Look

I'm always bemused by graphs that try to extrapolate too far.  Case in point at right (found by Darren).  The numbers previous to today are obviously known well but the future is a little cloudier.  Nevertheless, this graph claims to see a trend going rather upward, even though the trend for the next few years is downward.

Like the "population bomb" scare stories of the recent past, this one is just silly.

Why silly? Because it assumes that "given no change in policy, the numbers will do this."

Given no change? This is America.  We can't stop changing. The one thing that will never happen is steady state.  Somebody will introduce a bill to change this or that and your whole projection becomes worthless ... as if it meant much to begin with. We adapt in this country.  When the deficits were spiraling out of control under Reagan/Bush I, the country adapted, we nearly balanced the budget and we were starting to work on the debt.  Clinton's policies were less of a driver than the tech boom but, c'est la guerre. I'm frankly surprised that the National Review published it because it shows the sharpest drop under Obama's second term.


I had to tweak Darren, since he's a staunch conservative:

"Come on. Extrapolating a possibly exponential curve out to 2082? You really shouldn't, as a math teacher, have let that go without some comment. 

But I'll play along ... let's read the graph. The percentage surges upward in 1980-1984 when Reagan was doing those enormous deficit budgets -- to drive the Russians to bankruptcy, yes, but it still increases.

Then the economy recovered and those percentages dropped under Clinton (not that he was totally responsible for that, but it does make a good way to tweak conservatives!).  Likewise, the percentages rises under GWB, hits a peak in 2010 under Obama, and then shows a tremendous downward slope under the remaining years of Obama's term (and into his next?). Then, in 2016 (when the Republican presumably gets elected), the percentage starts to climb again.

I'm not sure that's exactly what you had in mind."

Darren responded later ...

"It's when the health care costs really start to kick in. Of course extrapolations that far out are silly, which is one of the reasons why I love the global warmers so much. Still, I don't see anyone arguing that our interest payments are going to go *down* any time soon."

No. But why is that graph any less ridiculous? The spending by Government is totally under the control of the Legislature and can be adjusted yearly as the political winds blow. If they wanted to, the Afgan war could cease in days, the bailouts could stop, the contracts could be ended and penalties paid.

Global warming isn't quite up to a vote by the House Ways and Means Committee. Natural processes don't stop on a whim.

I don't particularly care about the gloom and doom part of the debate anyway, so I'm probably not a good person to argue this with. I don't care about global climate as such, but I do care about MY air. If NYCity goes under water, it'd be turned into a modern Venice pretty quickly and Mankind would adapt. Florida would build dikes like Holland and Mankind would adapt. Ideal croplands would be found further north and, well, you get the idea.

One-percent of a Standard Deviation

... is quite a bit smaller than a one-percent increase. In fact, 1% of a standard deviation is pretty damned small. It seems somebody needs a Statistics course:

Public schools located near private schools increased reading and math scores more than public schools that had little competition.

Huge, I tells ya.

For every 1.1 miles closer to the nearest private school, public school math and reading performance increases by 1.5 percent of a standard deviation in the first year following the announcement of the scholarship program. Likewise, having 12 additional private schools nearby boosts public school test scores by almost 3 percent of a standard deviation. The presence of two additional types of private schools nearby raises test scores by about 2 percent of a standard deviation. Finally, an increase of one standard deviation in the concentration of private schools nearby is associated with an increase of about 1 percent of a standard deviation in test scores.

Test scores rose more for elementary and middle schools than for high schools, perhaps because the scholarship made K-8 private schools affordable but didn’t cover as much of the tuition at private high schools.

Hummmm ...

Did the scores of the private schools drop at the same time as the public school rose?  If the public school scores rose, was it because the parents of weaker kids took the money and ran? Was it because Florida is investing heavily in on-line learning and told certain kids that their behavior was unacceptable IN school so they had to switch to the private school or take courses online? We'll never know but this is an equally valid interpretation of the facts as presented.

I find the "1.5 percent of SD per mile" statistic interesting but pretty meaningless. That's not a standard deviation, it's one-one hundredth of a standard deviation. That's the equivalent of SAT scores rising 1 point. Read the collegeboard's take on significance.

Just because you can see something in your educational microscope doesn't mean there's anything worth looking at.
The quote also says that this happens only in the first year following the announcement. So the average SAT scores went up about 1 pt.
Once.

Here are Florida's average SAT scores for the last couple years. Notice the yearly fluctuations larger than that touted by the article. Note, the standard deviation for SAT scores is typically 100 - 110 points. So 1.5% of a standard deviation would be 1.5 points.

The timeline is also interesting. The idea that the mere announcement of a private school makes a difference in the first year (but only in the first year) indicates that it's got nothing to do with the education provided since it takes some time for a kid to get an education. Statistically insignificant.

I'd be looking for information on who paid for this study and who has the most to gain by falsely trumpeting miniscule gains and falsely attributing them to the glorious private schools.

Polls may be hitting a wall.

Slashdot has a thought:

"The 'cellphone effect.' In 2003, just 3.2% of households were cell-only, while in the 2010 election one-quarter of American adults have ditched their landlines and rely exclusively on their mobile phones, and a lot of pollsters don't call mobile phones. Cellphone-only voters tend to be younger, more urban, and less white — all Democratic demographics — and a study by Pew Research suggests that the failure to include them might bias the polls by about 4 points against Democrats, even after demographic weighting is applied."

This will make the "science" of polling even more suspect.  It's a factor I hadn't really considered until now, but everyone that I know who has dropped their landline for a cell-only life is definitely in the Democratic profile. 

On the other hand, those people most likely to skip voting entirely are also in the exact same demographic.

Vouchers again

Voucher results mixed
Overall, public-school students did better on state tests
By Jennifer Smith Richards / Columbus (OH) Dispatch

On the whole, Ohio students who used tax-funded vouchers to attend private schools last school year did no better on state tests than public-school students.

One commenter wrote:
"That's not the issue, is it? What we want to know is whether private school students did better than they would have in a public school. That's the issue. The only way you can come close to answering that question is with random samples and control groups--neither of which were used here."

Very true. In fact, random samples and control groups would tend to eliminate the selection bias inherent in the voucher system and make the comparisons even worse for the voucher schools. Add to that a size bias - the small voucher school that randomly does well is used as a club over the heads of the public school.  At the same time, the small voucher school that randomly does poorly is ignored .... until next year's random increase means that you can trumpet "Its huge improvement is the result of vouchers."

When the one with all the advantages doesn't win,
there is usually a reason.

Fundamentally, public schools should be the recipients of public school money. They are run by the town for the benefit of its citizens, are controlled by a School Board elected by the citizens, use a budget that is voted on by its citizens, and by and large are staffed by its citizens. Accountability is to the town.  At least in Vermont.

Private schools are not accountable to the town, do not have to make their finances public, do not have to answer to the taxpayers, and are run for the benefit of the school. Private and charter schools do not have to follow federal mandates for the education of all students, can dismiss or expel students for disciplinary or educational reasons without refunding the town, and can send kids back to the public school if their academic performance isn't up to par. Private schools can skim the best students off the top, "recruit" from other districts, include I20 students rather than IEPs or 504s, offer "scholarships," and do many other things to "win."

What's the real issue? If the charter and private schools cannot out-do the public schools despite their very real head start, we shouldn't be blaming the public schools.

Common Sense on Drug Stats

Every once in a while we get hit with the workshop that exists only to justify its own existence, while keeping the presenter employed. The drug-scare workshop is the prime candidate.

1. Funny About Money went to one. They said 50% of CC students are coming to class stoned or high on multiple drugs.
2. Our Drug Awareness Dude from the community center showed up with two suitcases of things to look for and police-approved fake samples of paraphernalia. 
3. The ex-cop came in to show us those brain scans of druggies vs non-druggies. "Look at the "holes" in this one and at the nice smooth brain over here."
4. The ex-druggie claimed to have taken this entire list of drugs and abused his kid, and here she is to confirm that he was an asshole. The poor girl stands there looking embarrassed as hell as he lists all the signs of alcoholism and drug use in your students and describes how he used to mistreat his family while under the influence.

All this time, I'm thinking, "Calm down. Scare tactics only work for a few minutes until people start thinking again." It burns me up. Here we are paying this clown a couple thousand to preach bullshit and take up a lot of time. Isn't there a better use of our resources?

The ex-druggie is spouting utter crap, trying to make himself look better in his own eyes. His daughter is still being abused and I have no sympathy for him. He's telling us the signs of alcoholism - slanty eyes in kids - and the admin are eating it up. No one is surprised when the email comes by the next day ... "Please stop asking kids about their parents' alcoholism." He's telling the kids the dangers of drugs and how they'll ruin your life. Most of the kids noticed he's getting paid a couple grand to speak to a gynasium full of kids for an hour. A few also noticed that he "wants to come back a lot because this is the worst school in the whole state and I'd really like to come back often to help you guys." Gee, thanks.

The ex-cop has gotten hold of some pictures that scare the bejeezus out of him, except that he doesn't understand them. "Look at the holes in this kid's brain." then, the best line, "See the awful colors? Those are caused by drugs."

Ahhh, it's a computer generated scan of activity levels, Dude. It's a graph. The colors were chosen for contrast. There are no physical holes, just activity levels below an arbitrary cutoff.

Then, there were the line graphs. No vertical or horizontal scale, no mention of what was actually being measured. "This is a graph of memory for a kid on drugs." Me: "Time in days, months? Number of items, type of items, first 800 digits of pi? What memories, what kind of mental tasks?" Him: "I don't know. I just got these graphs from their website." Me: "...."

But wait, there's more. There's the community anti-drug crusader.

Apparently, according to him, our students are taking any or all of over a hundred different types of drugs. He had displays in the two suitcases so therefore he must be telling the truth. We had problems: meth, heroin, speed, downers, pot, booze, cocaine, peyote, mushrooms, wine coolers and hard lemonade, injectables, glue and other inhalants -- even (this is no shit) "Some girls are soaking their hair scrunchies in gasoline so they can sit in class and sniff the fumes all period." Really? Don't you think someone might notice that? Doesn't this seem a bit far fetched?

So I asked him, "According to what you have seen and found, according to what the Police Chief standing next to you has seen and found, according to what the kids themselves tell you about what they've heard other kids are doing, what is really happening in this town?" "Alcohol, mostly. Marijuana."

What about the other things you're showing us? "Well, there was that one kid three years ago." Here's a little math for you: 100 - 2 = 98% bullshit.

Funny with Money was told by the college people responsible for policing and preventing it, that about fifty percent of students in a community college classroom, at any given time, are likely to be abusing some kind of drug, legally purchased or not, often more than one something.

"Substances" range from meth to over-the-counter cold pills and nostrums. So, the homeopathic "cold medicine" that contains less of anything useful or harmful than does the average placebo is also lumped into the statistic. And probably Hall's and Ricola.

Anyone else wondering why these geniuses are telling us all this? Maybe to justify their own salaries through irony? If this is such a problem, why haven't they done anything about it?

And seriously? Most community college kids are there on borrowed money and many are working fulltime, desperate for an education. They aren't going to class wasted, they're going to class tired. If they were wasted, they wouldn't bother going to class.

Why do I complain? Because overloading the classroom teacher with trivial bullshit that has no bearing on anything in their classroom is a lose-lose situation.


If you have us looking for wraiths that do not exist, then we will find those wraiths anyway and waste time, effort, soul, and most importantly, trust, barking up the wrong trees. Everyone loses.


If you tell us that slanty eyes in kids are a sure sign of parental alcoholism, then that is what the faculty will see. Don't be surprised if parents thus accused do not take it well.

If you tell us that 95% of our students are on drugs right now (via the ex-druggie, with the principal standing right next to him, nodding wisely), then we will misinterpret tired or worried or frustrated as "drugged." If you tell us that we should report everything, then an awful lot of nonsense will get thrown into the system -- parents will be called, kids will be called in, DYS will have to be notified -- because a kid had a fight with his girlfriend and didn't sleep well for the last two weeks.

That's how you lose when you are dealing with students.

You lose. YOU. LOSE.

Kids won't open up when they really need help. They will fall into the patterns you accuse them of. The parents will learn to NEVER trust the faculty or the school. The kids will withdraw from all of the possible support. Your school will take on a whole new feel and no one will like it.

If you try and tell students that the brain doesn't stop growing until the age of 25, so "Students shouldn't drink until then," you will come off as loony and hypocritical. Kids watch TV on Sunday for the football. We shouldn't expect the other messages to magically disappear: Alcohol is cool and very desirable and the perfect home has hundreds of cans in the fridge, strength and power are all-important (steroids and other PEDS), and women are supposed to dress like that.

If your presentation is mostly hyperbole and loaded with graphs you don't understand, your rhetoric will pass right on by. Kids are good at ignoring adult exaggeration and they hate hypocrisy.

Kids laugh at the lameness of the "Eggs on Drugs" commercials. They don't listen to teachers who bullshit them.

Is there a problem in my school? Yep.
Should the school discuss it? Yep.
Should we do something? Absolutely.

If you have information for us in the school, we will gladly listen to it and do our best to act on it.  Just follow one simple rule:

"Tell me the truth without exaggerating and stop wasting my time."