Schools of Thought

Happily, the New York Times appears to have noticed the elephant in the room:

After Tennessee tested its eighth-grade students in math this year, state officials at a jubilant news conference called the results a "cause for celebration." Eighty-seven percent of students performed at or above the proficiency level.

But when the federal government made public the findings of its own tests last month, the results were startlingly different: only 21 percent of Tennessee's eighth graders were considered proficient in math.

Such discrepancies have intensified the national debate over testing and accountability, with some educators saying that numerous states have created easy exams to avoid the sanctions that President Bush's centerpiece education law, No Child Left Behind, imposes on consistently low-scoring schools.

A comparison of state test results against the latest National Assessment of Educational Progress, a federal test mandated by the No Child Left Behind law, shows that wide discrepancies between the state and federal findings were commonplace.

What's most interesting about this article, however, is not the finding which most of us who follow this subject already knew, but rather the myriad attempts by states to explain away the gulfs.

The National Assessment uses three performance levels to classify student results: advanced, which denotes superior performance; proficient, which indicates that students have "demonstrated competency" and basic, which indicates students have attained only "partial mastery of prerequisite knowledge and skills." Many students also score below basic, which the National Assessment's governing board does not classify as an achievement level.

On Oct. 19, the day the federal results were released, Ms. Spellings urged reporters to compare the percentage of students performing at the proficiency level on state tests with the percentage of students performing at the basic level on the federal test.

Many state officials said they also preferred that comparison, which greatly softens the discrepancies. In Tennessee, for instance, the 66-point gap between the federal and state results in eighth-grade math shrinks to just 26 points if the state results are compared with the federal measure of basic skills.

"NAEP's basic is comparable to our proficient," said Kim Karesh, a spokeswoman for the Tennessee Department of Education. "Now whether Tennessee's test is stringent enough is something that we're reviewing constantly. Nobody here would say we have a perfect test."

So, in other words, the Secretary of Education and the state of Tennessee are admitting that proficiency on most state tests means at best partial mastery of the skills, yet...

Ms. Spellings has declined to criticize states whose tests appear to overstate the percentage of their students who are proficient..."We're not going to sit up in Washington and look at all those moving parts."

No, because that would make accountability actually matter. The problem with standards and accountability is that they can't function when the standards are bogus and no one holds states accountable for it. For instance, when states start making things up:

Officials in many other states whose scores differed sharply from those of the National Assessment cried foul over the very idea of comparing the results.

"The comparison to NAEP is not fair," said Mitch Edwards, a spokesman for the Department of Education in Alabama, where 83 percent of fourth-grade students scored at or above proficient on the state's reading test while only 22 percent demonstrated proficiency on the federal reading test. "Making comparisons to the NAEP becomes very difficult without giving the impression that some states are not measuring up to others or to the nation."

In Georgia, 83 percent of eighth graders scored at or above proficient on state reading tests, compared with just 24 percent on the federal test. "Kids know the federal test doesn't really count," said Dana Tofig, a spokesman for the State Department of Education. "So it's not a fair comparison; it's not apples to apples."

Of course, NAEP is a sample-size assessment that is statistically designed to be a valid comparison. Moreover, while the question of motivation is a legitimate concern for high school results, the National Center of Educational Statistics, which administers the NAEP, has found little reason to believe it plays a significant role in middle school, much less in elementary school. It comes down to states -- understandably given the politics, though still unacceptable -- making excuses to avoid admitting their failure. Yet they must confront the problem in order to conquer it, and that might mean external pressure. Accountability cannot function without honesty, and the states are not being honest.

The Washington Post's Marc Fisher has a great column in Tuesday's paper about the D.C. public schools and how to make use of their prepetually mismanaged money.

One of the sharpest divides in perceptions about the challenges and solutions facing education surrounds the role of schools. Put plainly, it's the related questions of how much schools can be expected to do and how much schools should be expected to do. On the one extreme are those who believe schools should be social institutions as much as educative ones, thinking of the school as a place that should actively counter issues such as health care and poor parenting. The polar opposite camp wants the social work extricated from the schools, demanding that students leave their problems at the door as they enter the building where it is their sole job to learn. Few people actually fall squarely into either philosophy, but we can generally correlate "liberal" with the former and "conservative" with the latter.

What's really intruiging is that neither extreme comes to the conclusion that the schools can and should be expected to teach every student to a point of proficiency that maximizes his or her opportunity. Take, for example, hardline anti-NCLBer and fellow blogger Jim Horn, who argues that:

The sad, but irrefutable, fact remains that the achievement gap that we hear so much about is a product of the income and opportunity gap, which is a product of a tradition of racism that goes back over 300 years in this country. It is a problem that will not be resolved by pretending it is the fault of the schools.

This logic has its counterpoint in the position that it's up to the students to decide to learn, and that external factors are simply excuses.

Now, first of all, the evidence undercuts both trains of thought. Poverty and socioeconomic factors clearly do play a role in education, if for no other reason than schools in higher-poverty districts recieve less money, less experienced/excellent teachers, etc. Yet at the same time, there are high-poverty schools that face these challenges and overcome them to the tune of nearly complete proficiency. Witness Elmont H.S. in New York, M. Hall Stanton, City Springs, Dayton's Bluff, J.E.B. Stuart, etc., etc. It's fine to offer that these exceptional schools are unusual, but that's only because they are currently exceptions; there is no reason to think that such success isn't replicatable.

But what's more interesting to me is the logical conclusion of either the "all-environment" or "no-environment" crowds (or, to be fair, those falling near those extremes): Simply, that some students are not going to learn. You're never going to be able to alleiviate poverty, though it's a noble goal; you're never going to be able to have every student just buck-up, though it's a good goal. So, what happens to those kids on the margins? And, what happens to the kids during the generational task of repairing societal fault lines? Sorry, wrong place at the wrong time, have a nice not-very-productive life?

I don't buy it. As Aristotle's truism has it, everything in moderation. We can't ignore the effects of poverty, not on the schools or on the kids. A bad home life is not conducive to wonderful learning, and low-SES kids have a bounty of challenges that their wealthy peers don't ever have to worry about. There is massive social work to be done, and every effort will ease the challenges a bit more. Yet at the same time, we can't use poverty as a wringing-our-hands excuse for failing to educate these students. Not when we know for a fact that poor children can succeed at just as high a level as anyone else.

Let me sum it up this way: Kids born into a low-income family currently have a 9% chance of recieving a bachelor's degree by age 25. That either means the bulk of the other 91% are stupid and lazy, or inexorably crippled for life by the conditions of their birth. Or, the third option -- they're a segment of the population with unique obstacles that our educational system has yet to properly address.

The fork between assigning the school no responsibility and assigning the school every responsibility is a choice between Scylla and Charybdis. The only safe course is to steer carefully through the middle.

Believe it or not, I am indeed alive. Turns out that the combination of schoolwork, the newspaper, job hunting and the last month of undergraduate college isn't very conducive to blogging. But, I'm not going anywhere, just perhaps receding in frequency a tad.

The biggest edunews in the past few days is undoubtedly the Federal Department of Education's announcement that they would be letting 10 states do test runs on growth models as a form of accountability. As reported in the Boston Globe:

In an experiment that's been months in the making, up to 10 states will be allowed to measure not just how students are performing, but how that performance is changing over time.

Currently, schools are judged based only on how today's students compare to last year's students in math and reading -- such as fourth-graders in 2005 versus fourth-graders in 2004.

Many state leaders don't like the current system of comparing two different years of kids because it doesn't recognize changes in the population or growth by individual students.

[...]

The states that win approval for the new flexibility, however, must do more than show growth. They still will have to get all children up to par in reading and math by 2014, as the law requires, and show consistent gains along the way.

The Education Department, eager to show it is not weakening the law, will require states to take many steps before they can qualify for the "growth" option.

States must have data systems to track individual students, close achievement gaps between whites and minorities, and prove they have at least one year of baseline testing.

Having worked on growth models a great deal at the EdTrust over the summer, this news isn't particularly shocking; it was always more of a when than an if. So, what to make of it? The Feds backing down on accountability? Victory for the states? Or just another incarnation of a high-stakes mistake, as the virulently anti-NCLB crowd might have it?

Really, none of the above. Growth models are a natural evolution from the "status" models epitomized by AYP (though I should point out that AYP has a psuedo-growth model built in via the safe harbor provisions). They are, if applied properly and with appropriate safeguards, in nearly every way superior to status models. Don't get me wrong, they aren't fundamentally different -- most of the negative externalities of high-stakes accountability still exist. But on the quest for the crucial item called accountability, growth models are essentially the next-generation tools.

The key is going to be ensuring that they are applied properly and with appropriate safeguards. At their most basic level, growth models let you see the isolated progress of a student or a class from year to year. This is different than looking at one year's 4th graders vs. the next year's 4th graders in assessing progress of teaching 4th graders -- it's asking the question, how are one year's 4th graders doing in 5th grade, in 6th grade, etc.?

It's not a simple matter of looking at test scores from one year to the next, however; first and foremost, you have to be able to track students as they move through the grades, something only 32 states can do at the moment, and with varying levels of effectiveness. Statistical modeling is also ideally involved to try and tease out the progress -- or lack thereof -- that can actually be attributed to the classroom (though some may argue that the source isn't as important as the result, but I think both are necessary pieces of information).

So I'm pleased that the Feds seem cognizant of these facts and I hope they actually push hard for the logistical prerequisites. A bad growth model is far worse than a medicore status model.

The other, and ultimately more important, concern is what type of growth models will be allowed. There are three main options: Expected-growth, average-growth and growth-to-standard. Expected-growth means that a school/class/student's growth target is equal to what equivalent groups have done in the past. For instance, the Tennessee Value-Added Assessment System (TVAAS), the creme de la creme of existing growth models, currently has it so Tennessee 4th graders have to make the same progress as 4th graders did in a baseline year. Average-growth means that the unit has to make progress equal to the average of its peer group. Growth-to-standard sets a uniform target and a time by which, through whatever amount of growth in each year, everyone must meet the standard.

The growth-to-standard is the only respectable option, even though it's the most difficult to implement and least utilized. Both expected-growth and average-growth take kids who start low and leave them low relative to their higher-performing peers, and neither requires that anyone reaches proficiency. If it's easier, think about a graph of achievement with three students starting at varying levels and a flat proficiency line way above, then think about what expected-growth and average-growth does to the three points. Growth-to-standard, on the other hand, takes all the advantages of growth models -- accuracy, a truer picture, more honed analysis that can be used diagnostically , etc. -- and combines it with the accountability which may be the best part in a largely flawed AYP formula.

So, I'm cautiously encouraged by this move. The Feds need to be vigilant to ensure that growth models are employed appropriately -- with the proper infrastructure and growth-to-standard in place -- but if that happens, they will likely prove far superior to the current system. Now, if we could just salvage those pesky exams...

Trayshawn Wright, 14 ... voiced what many adults remember about the [algebra] course: "I get it. But I get confused at times."

She said the effort would be worth it, though: "Algebra's very important. It's going to help you in the long run. You're going to need it for many things."

For example: "I want a diploma."

I've heard rumblings about the need for national standards/assessments more often lately, and in today's NYT Diane Ravitch goes to town:

WHILE in office, Presidents George H. W. Bush and Bill Clinton both called for national academic standards and national tests in the public schools. In both cases, the proposals were rejected by a Congress dominated by the opposing party. The current President Bush, with a friendly Congress in hand, did not pursue that goal because it is contrary to the Republican Party philosophy of localism. Instead he adopted a strategy of "50 states, 50 standards, 50 tests" - and the evidence is growing that this approach has not improved student achievement. Americans must recognize that we need national standards, national tests and a national curriculum.

The release last month of test results by the National Assessment of Educational Progress, which is part of the Department of Education, vividly demonstrated why varying state standards and tests are inadequate. Almost all states report that, based on their own tests, incredibly large proportions of their students meet high standards. Yet the scores on the federal test (which was given to a representative sample of fourth and eighth graders) were far lower. Basically, the states have embraced low standards and grade inflation.

[...]

Why the discrepancies? The states function in a political environment. Educational leaders and elected officials want to assure the public that the schools are doing their jobs and making progress. The federal testing program, administered for the past 15 years by an independent, bipartisan governing board, has never been cowed by the demands of parents, school officials and taxpayers for good news.

In the No Child Left Behind law of 2001, Congress left it to each state to develop its own standards and tests, but added that the tests given by National Assessment of Educational Progress should serve as an external gauge of national and state-level achievement. The federal tests are considered the gold standard for good reason: they are the product of a long-term federal investment in research and development. Unlike the state tests, the federal program tries to align its performance standards with international education standards. Many states model their testing on the national program, but still cling to lower standards for fear of alienating the public and embarrassing public officials responsible for education.

I think Ravitch and the national-solution crowd have a tremendous number of good points. I've written numerous times about how states manipulate, fudge, spin and otherwise mask the truth of education in their districts. I don't believe their intentions are sinister so much as understandable -- as Ravitch notes, it's politics, stupid. Especially in states with elected school boards and elected state superintendants, the downside to public accountability is that there's a great big carrot enticing officials to engage in sleight of hand. To be honest, for a long time I fervently believed that federal imposition was going to be the only way to get all of our ducks in a row.

I just don't see any concievable way that it actually happens.

States are notoriously cagey when it comes to their domains, and perhaps moreso on education than any other issue. We've seen the states moan and groan about every requirement of NCLB, and we've even seen them recoil at the suggestion that the federal government enforce a standardized graduation rate across the land. The second the government passes a law requiring states to participate in national assessments and to face consequences as a result of those assessments, there's going to be a lawsuit. And unless the Supreme Court comes up with a very creative reading of the commerce clause, my rudimentary knowledge of constitutional law tells me there is little chance the feds will win.

The other option is the voluntary agreement approach, embodied by the National Governor Association's pact between 45 states to come up with standardized graduation rate metrics. But graduation rates and standards/assessments are orders of magnitude apart. It would take an enormous amount of public pressure to force states (especially those authentically on the bottom half of the spectrum) to give up that which they feel is nothing short of a birthright -- how to educate their kids.

I don't see that happening anytime soon, but it's certainly the most viable way to reach the necessary threshold of truth. Every new demonstration of the states' chicanery -- every person who realizes that 40% less kids are passing the NAEP than are passing their state assessments -- brings us closer to that critical mass. It might take a funding nudge from the feds, and it's certainly going to take a degree of the state's shooting themselves in the foot to remove a tumor. We've seen it happen before, though -- Washington State superintendant Terry Burgeson (an elected offical) recently came out and said that her state's graduation rate was actually far, far lower than they had been reporting. Despite admitting a failure, she won accolades for honesty and guts. So perhaps it's not hopeless.

The bottom line is that 50 ad hoc systems with different quality assessments, cut scores, n-sizes, content standards and achievement standards is begging for trouble, and trouble is exactly what we've got. "Nationalization" simply isn't a practical option; voluntary agreements may be the only way to go.

A must-see from the New York Times. I'm skeptical of some of the numbers, but even if you halve each of the stats (or cut them by two-thirds!), it tells you everything you need to know about how important education is.