Samples must be numerically significant or else conclusions are worthless.

Parents Against Tired Truckers is losing it's religion over the results of a new study on an experiment up here in Vermont. Just as in education, small sample sizes and incomplete data are being misunderstood and misrepresented to further a viewpoint that may do more harm than good. PATT has it's heart in the right place, but it's brains are sorely lacking.

Vermont asked the Feds to study whether allowing 100,000 lb rigs on major highways would be more dangerous than having them travel the back roads.

The study results came out. PATT shouted

The Trucking Industry Is Wrong on the Maine and Vermont 100,000 lb. Truck Pilot Program – DEAD Wrong 

Wow. That must be some study. "Dead wrong" isn't mincing words.

" .. the Truck Safety Coalition (TSC) released startling information revealed in response to a Freedom of Information Act (FOIA) request sent to the Vermont Agency of Transportation (VTrans)."

Impressive. "Startling", you say? Took a FOIA request, huh? I must read further. "Catastrohic results" "People needlessly died." Damn.

The number is SO BIG.

Documents show that during the 100,000 lb. truck pilot project in 2010, Vermont’s commercial motor vehicle fatal crash rate tripled from .49 fatal crashes per 100 million miles traveled in 2009 to 1.44 fatal crashes (“Vermont Truck Interstate Pilot Study- Report to Congress (State of Vermont Version for Review) – Summary Report (Draft)” prepared for FHWA by Cambridge Systematics, Inc, hereinafter “Vermont Report”).

What does that mean Regis? The Death rate tripled.  Holy Batman, mackerel.  They're quoting Government documents and it sounds so official.

Well, actually, it doesn't mean much at all. You see, Vermont had one death involving trucks on its roads in 2009 and three in 2010. Yeah, the death rate "tripled" but you need to have a bigger sample size before you can claim that trucks are making things more dangerous.

You also need to look at the reality of those crashes. In the one crash, two trucks and a car were involved in an accident that was blamed on icy roads and bad conditions. One of the truck drivers and the car's driver were killed. In the other accident, the car (probably drunk) crossed the 50-foot median and hit the truck head-on. Again, hardly the fault of the truck driver.

As in education, there's always some fool trumpeting results based on small sample sizes and assuming the study will scale up. Remember when Bill Gates spent nearly a billion dollars to create the Small Schools Initiative? The smaller schools that did better than the large public schools were showcased until the next year when the same school would do worse, at which point the deformers would shout about some other school which HAD done well that year. Variation of the small groups, not the inevitable superiority of the charter school, small-school, voucher school, Catholic School, whatever.

To give you another example, consider Daisuke Matsusaka (RedSox). He had four starts. Two were terrible and then two were decent. Can we say that trend is positive? Yes, but I'm not giving him a contract based on that.

Still another comes from here:
Last week I tossed a coin a hundred times. 49 heads. Then I changed into a red t-shirt and tossed the same coin another hundred times. 51 heads. From this, I conclude that wearing a red shirt gives a 4.1% increase in conversion in throwing heads.

Pretty foolish.  Besides, everyone knows that wearing a red shirt is tantamount to a death sentence anyway, so I'm not sure what can be made from this "study" either.