For the most part, I think Tom Horne has done a pretty good job as state superintendent of public instruction.

He inherited an impending train wreck with AIMS as a high school graduation requirement, with a likely failure rate that would have been politically unacceptable. Horne finessed the issue, leaving the state with a graduation test of dubious value, but with the state's overall accountability-through-testing regimen intact.

Horne has been heroic in the litigation battle over English learner funding, defending representative government from an overreaching judiciary when every other superintendent and governor has caved.

These days, however, Horne is gilding the lily about student achievement in Arizona and being blind to possible deficiencies in Arizona's dual-purpose assessment exam.

This is, in part, understandable. Both the left and the right chronically exaggerate how bad public education is in Arizona - the left to make the case for more funding, the right to make the case for more alternatives to traditional public education.

In reality, when held constant for demographic differences, student achievement in Arizona is pretty much right in the middle compared to other states. Moreover, there aren't a lot of differences among the states in student achievement, again adjusted for demographic differences.

Horne, however, insists that Arizona students are actually performing above average. He bases this in part on Arizona students doing better than average on the main college entrance exams, the SAT and the ACT.

Critics correctly say that the SAT and ACT are flawed measures of cross-state performance, since participation is self-selective and differs across states. However, the fact that Arizona students perform above average on both major college entrance exams does suggest that Arizona schools are doing a decent job, compared to other states, of educating college-bound kids.

However, Horne's main basis for the above-average assertion is the state's TerraNova test results, and therein lies the problem.

In testing, there are two things that should become known. First, whether students are learning what the state wants them to learn. Second, how Arizona students compare to students in other states.

Arizona used to administer two different tests to acquire this information. To reduce the time spent testing, Horne combined the two into one test. Although the number of questions asked was drastically reduced, he claimed that the national comparisons, based upon a subset of TerraNova questions, would still be valid.

A recent Goldwater Institute study questioned this claim. Horne felt personally attacked by the study and the way in which the institute marketed it, with some justification. And he has reacted defensively.

However, there are serious questions as to whether the dual-purpose test yields reliable data for national comparisons. There has never been a sample of students taking both the stripped-down national questions and the full TerraNova battery to see whether the results are, indeed, the same. So, the state is flying a bit blind when it comes to what is called norm-referenced testing.

In a previous column, I suggested that a legislative committee be set up to look into this, and that Sen. John Huppenthal lead it. Huppenthal likes data-driven issues such as this and is pretty good at sorting things out.

Huppenthal didn't wait for the assignment and jumped right into the data. His preliminary conclusion is that there is an inflationary factor in the TerraNova test, as there tends to be with all of the major norm-referenced tests. So, caution should be used in making the claim that TerraNova proves that Arizona students are performing above average.

According to Huppenthal, the stripped-down TerraNova probably provides reliable information at a state, district and school level, but perhaps not so much on an individual student level, particularly at the extremes of student performance.

If Huppenthal is right, perhaps the solution is a parental option for a student to take the full TerraNova battery, or some other national norm-referenced test.

In the meantime, Arizona does have some difficult education challenges because of our demography. The achievement gap that all states are wrestling with is simply more important to overcome here.

Gilding the lily, while perhaps an understandable reaction to the "woe is us" excesses about public education in the state, doesn't set up the public policy environment for the tough work ahead.

In a column published in last Sunday's Arizona Republic, Horne wrote: "There's more to achieve, but let's not ignore the legitimate good news."

Unfortunately, the legitimate good news is mostly that the bad news is overstated.



Reach Robb at robert.robb@arizonarepublic.com or (602) 444-8472. Read his blog at robblog.azcentral.com.