<body><script type="text/javascript"> function setAttributeOnload(object, attribute, val) { if(window.addEventListener) { window.addEventListener('load', function(){ object[attribute] = val; }, false); } else { window.attachEvent('onload', function(){ object[attribute] = val; }); } } </script> <div id="navbar-iframe-container"></div> <script type="text/javascript" src="https://apis.google.com/js/platform.js"></script> <script type="text/javascript"> gapi.load("gapi.iframes:gapi.iframes.style.bubble", function() { if (gapi.iframes && gapi.iframes.getContext) { gapi.iframes.getContext().openChild({ url: 'https://www.blogger.com/navbar.g?targetBlogID\x3d15526040\x26blogName\x3dSchools+of+Thought\x26publishMode\x3dPUBLISH_MODE_BLOGSPOT\x26navbarType\x3dBLUE\x26layoutType\x3dCLASSIC\x26searchRoot\x3dhttps://haspel.blogspot.com/search\x26blogLocale\x3den_US\x26v\x3d2\x26homepageUrl\x3dhttp://haspel.blogspot.com/\x26vt\x3d-2837553055558778188', where: document.getElementById("navbar-iframe-container"), id: "navbar-iframe" }); } }); </script>

Schools of Thought

Translating education research into usable knowledge

Update: ASU/Great Lakes study

Tuesday, September 27, 2005
Roddy Stinson, a columnist for the San-Antonio Express News, has a slightly different take on why the study about high-stakes testing recieved so little press: "Because it bordered on heresy." Not sure I'm buying it.

Posting this week

Monday, September 26, 2005
With midterm season starting and U.Va.'s fall break this weekend, posts may be slack for a little while; expect at least one by Thursday evening and one by Tuesday evening, after which time things should be back on track.

Additions to the blogroll

Sunday, September 25, 2005
I've added a few new links to some blogs that are worthy of note: Kindling Flames, by a group of George Washington University ed policy grad students, and three teacher blogs (since where better to go than the source?) in Mr. Babylon, Ms. Frizzle and the UK's Tales from the Chalk-Face. Happy reading!

Communication breakdown

Saturday, September 24, 2005
When the research group of a respected university issues a report that leads with the paragraph:

Under the federal No Child Left Behind Act of 2001 (NCLB), standardized test scores are the indicator used to hold schools and school districts accountable for student achievement. Each state is responsible for constructing an accountability system, attaching consequences—or stakes—for student performance. The theory of action implied by this accountability program is that the pressure of high-stakes testing will increase student achievement. But this study finds that pressure created by high-stakes testing has had almost no important influence on student academic performance.

It might be expected that someone would listen. But "High-Stakes Testing and Student Achievement: Problems for the No Child Left Behind Act," produced by the Education Policy Research Unit of Arizona State University and funded by the Great Lakes Center for Education Policy and Research, has recieved almost no traction in the press so far. Various searches on GoogleNews reveal only a handful of media outlets which have picked up on the report, and none of the major ones like the New York Times, Washington Post or L.A. Times.

The report's conclusions are provocative, and attacking NCLB is not exactly a topic that lacks sexiness. So why the thud? I think it's directly tied to the obtuse and dense style of the document. The report is 118 pages long, and once you get past the executive summary and introduction (which it's hard to simply take on faith), the evidence becomes an impassable swamp of matrices, indices, equations and advanced statistics. If a would-be wonk like myself is having trouble slogging through the pages, its hard to think that education reporters, much less the public, are going to be clamoring to write/read about it.

Education policy, like many other disciplines, constantly runs into a logjam between the desired input of science and the desired output of accurate, comprehensible publicity. The mediators -- those who are supposed to bridge what has turned into a yawning chasm -- are primarily the media (and secondarily advocacy groups). But because the media is necessarily in the middle, the researchers must meet them there. In practice, this means writing in clear, concise language that uses as little jargon as possible and including as many easy-to-understand graphs as will fit. Education reporters are on the whole bright and curious, but they are also very busy; there are only certain degrees of complexity which will hold anyone's attention outside of the academe. Let the academics hash out the tertiary equations in the appendices.

If the science of policy continues to be presented like something you would find in an academic journal, much of it will continue to be ignored. I know via my experience as a journalist and my experience working on the other side of the print with the Education Trust that the initial connection between policy and policy-conveyor is the circuit through which most power runs; sever that line, and the public never even knows it's missing something. EdTrust is a pro at this, which is one reason why they are so effective at getting out their crucial message regarding the achievement gap.

The ASU/Great Lakes study is important; it asserts, for example, that there's no correlation between the pressure of high-stakes testing and scores on the NAEP. It further claims "High-stakes testing pressure is negatively associated with the likelihood that eighth and tenth graders will move into 12th grade. Study results suggest that increases in testing pressure are related to larger numbers of students being held back or dropping out of school." If the report is accurate, it casts huge amounts of doubt on critical pieces of the nation's current education policy. It therefore deserves to be debated, to have statisticians and educational experts and teachers and students and the general public and everyone else chime in. But I'm not sure that's going to happen, because you want to start clawing your eyes out by about page 13.

Science must be translated into public knowledge in order to make a difference. The sooner educational researchers figure that out, the better off we'll all be.

Update: Let's score this

Thursday, September 22, 2005
I just discovered that Florida's DoEd has included an answer key with the percent of students that answered each question correctly. Let's look at our questions below:

#8 (moderate difficulty) -- 82% correct
#9 (m) -- 64% correct (so, over a third of 10th graders can't put together "rain" and "can't swim")
#10 (m) -- 80% correct
#11 (m) -- 77% correct
#12 (low difficulty) -- 91% correct
#13 (m) -- 82% correct
#14 (l) -- 78% correct
#15 (l) -- 73% correct

Before you say "75 or 80% isn't bad," let me just say that when a quarter or more of a state's 15-year-olds can't answer exceptionally basic reading questions, "bad" is the only word for it.

In terms of scoring, a student needs to get at least about 31 of the 56 multiple-choice questions, or 55 percent, of the reading questions correct in order to pass. Again, 32% of 10th graders (and 13% of blacks, 22% of latinos) did so on the first go-around in 2005. On math, a student only needs to get about 16 of the 60 of the multiple-choice and constructed-response questions, or 27%, correct in order to pass. Again, 63% of 10th graders (and 40% of blacks, 56% of latinos) did so on the first go-around in '05.

There's a reason passing the FCAT with a minimum of 300 equates to a 780 composite SAT, and I think there's a reason why Florida fought releasing these details about the FCAT.

Anyway you try to spin it, this is bad.

This is kind of depressing

As promised, Florida released an extended version of the 10th grade FCAT yesterday -- the graduation exam that students have six chances to pass between 10th and 12th grade. A minimum passing score on math and reading equate to about a 780 on the SAT. Yet on a test which isn't at all rigorous by that measure or by comparing it to NAEP scores, only 32% of 10th graders pass the reading test on the first go! Well, now we can see exactly what they aren't passing.

And we should be outraged.

For sake of space, I'm only going to examine one reading comprehension passage in this post, perhaps look at math in the next.

Tarantulas on the Lifebuoy
Thomas Lux

For some semitropical reason
when the rains fall
relentlessly they fall

into swimming pools,
these otherwise bright and scary arachnids.
They can swim a little, but not for long
and they can’t climb the ladder out.

They usually drown—but if you want their favor,
if you believe there is justice,
a reward for not loving the death of ugly and even dangerous (the eel, hog snake,
rats) creatures, if

you believe these things, then
you would leave a lifebuoy
or two in your swimming pool at night.

And in the morning
you would haul ashore
the huddled, hairy survivors

and escort them
back to the bush, and know,
be assured that at least these saved,
as individuals, would not turn up
again someday
in your hat, drawer,
or the tangled underworld

of your socks, and that even—
when your belief in justice
merges with your belief in dreams—
they may tell the others

in a sign language
four times as subtle
and complicated as man’s

that you are good,
that you love them,
that you would save them again.

Remember, these 10th graders should be taking the SAT in a year.

8. What does the lifebuoy represent?
F. fear of poisonous creatures
G. anger with irritating creatures
H. pleasure in beautiful creatures
I. sympathy for helpless creatures

9. What element of the poem’s setting creates the problem for the spiders?
A. flooding downpours
B. overflowing pools
C. sandy shores
D. slippery ladders

10. When the speaker in the poem says, “you would haul ashore the huddled, hairy survivors,” to what is he comparing the tarantulas?
F. seals
G. sea monsters
H. shipwreck victims
I. surfers

11. What does the speaker in the poem believe the tarantulas deserve?
A. death
B. fairness
C. gratitude
D. humiliation

12. What would the speaker in the poem most likely do if he found a tarantula in his boot?
F. leave it alone
G. attempt to tame it
H. kill it immediately
I. carry it back to the bush

13. The speaker in the poem addresses himself to “you” in order to
A. accuse the reader
B. engage the reader
C. confuse the reader
D. entertain the reader

14. The speaker in the poem says that spiders might
F. climb ladders.
G. speak English.
H. use signs for words.
I. have tangled dreams.

15. What would be another way of saying “not loving the death of ugly and even dangerous...creatures”?
A. accepting all forms of life
B. rejecting the idea of justice
C. understanding the nature of death
D. eliminating all threatening animals

I almost feel like I should just let this speak for itself. These questions aren't complicated analytical items that require a great deal of nuanced cognitive thought. These questions are the most basic, basic form of reading comprehension -- literally, can the student read? Take #15; that question could be answered without ever reading the poem. Once you read "not loving the death" you're already eliminating B and D, and then "ugly and even dangerous" hones you in on A. In what way is this challenging? In what way are these questions assessing ones advanced ability to critically read, draw inferences, make connections, interpret, apply, etc.? (and the whole reading test is along these lines). I didn't know the reading standard we were holding our high school graduates to was literally just being able to read.

68% of students failed the test the first time they took it, and 15,000 students still failed it after the sixth try!

There is no way around this truth: No student should get to the spring of 10th grade unable to answer these questions. Yet a stupefying number do, NCLB or no NCLB. It's time to consider that something is still going horribly wrong.

Carnival!

Wednesday, September 21, 2005
33rd edition of the Carnival of Education is up over at The Education Wonks. Some interesting stuff in there, definitely worth a look-see.

Do you know the latest bad news about education?

Tuesday, September 20, 2005
Jay Mathews has a cheery column in today's Washington Post in which he asserts that despite the seeming deluge of bad news about American schools, things aren't as bad as they may appear. I think Mathews is always thought-provoking, but here his self-described pollyana and the source he draws it from are being almost naive. I'm an optimist too, but one must understand the grave crisis facing American education to have a proper appreciation for the breadth of measures necessary to fix it. Mathews writes:

The Center on Education Policy report ["Do you know the latest good news about education"] cites several overlooked facts that appear to me to be uncontested:

* More Americans are completing high school or college. The percentage of Americans 25 or older who completed high school increased from 74 percent in 1985 to 84 percent in 2002. The portion of people in that age group who completed college rose from 19 to 27 percent in the same period. Much attention has been paid recently to the fact that other developed countries have caught up with the United States in college completion, but we should not begrudge them their own good news. At least we have improved.

* More children are getting more hours of early education. Full-day kindergarten, for instance, is serving more than 60 percent of children of kindergarten age, compared to less than a third in 1983.

* Broadly speaking, the achievement gap is narrowing. "On long-term NAEP [National Assessment of Educational Progress] trend assessments in math and reading, test score gaps between white and minority students have narrowed to the smallest margins in three decades," said the center's summary of the report.

* Average SAT scores are going up, even as more students take the test. The math average of 518 for members of the class of 2004 is 14 points higher than 1994 and 21 points higher than 1984. That class's verbal score of 508 is 9 points higher than 1994 and 4 points higher than 1984, after a decline in scores that inspired much of the public school restructuring of the 1980s and 1990s.

Let's take these one by one.

-More Americans are completing high school or college. This measure looks at the percentage of Americans age 25 or older. While it's great that the number is rising, it is not particularly precise, especially with regards to high school. Here's the important number: Best we can tell, somewhere around 30% of students nationwide who enter 9th grade don't graduate four years later. For blacks and latinos, that number is a jaw-dropping 50%. Completing high school in four or five years is a bare minimum requirement for having the social capital to fulfill ones potential, to say nothing of becoming financially stable. That one in two minority students don't graduate -- hundreds and hundreds of thousands of kids -- is not worth sugarcoating. It's worth solving.

-More children are getting more hours of early education. This is indeed an encouraging trend, and we need to keep up the momentum and extend it to Pre-K as well.

-Broadly speaking, the achievement gap is closing. This may be technically true, but it's not closing nearly fast enough to think we're truly on the right track. Digging into the NAEP report, we see that among 9-year-olds in reading, the black-white gap remains 26 points, down just 3 since 1996 and 7 since 1994. In math, the black-white gap is between 23 and 29 points for all tested age groups, with ten-year gap decrease of no more than 2 points.

On the 2003 main NAEP, black students in 32 states were at least 20 points, or 2 years worth of learning, behind their non-minority peers in 4th grade math, and latino students in 25 states were at least 20 points behind their non-minority peers in 4th grade reading. When you get into the state assessment numbers, the picture is much the same; huge 20, 30, 40 point gaps, and 3-year improvements usually in the low single digits. The achievement gap may be crawling closed, but it remains a huge, massive, pressing problem that current efforts are not sufficient to conquer.

-Average SAT scores are going up, even as more students are taking the test. This is nice, and i'm not going to deny that the math score improvement is solid (though one has to wonder how much technology/advanced calculators has to do with that). But, two points: First, a 4-point increase in reading scores since 1984. A 4-point increase in twenty-one years. Reading is a pretty static thing; you can read, or you can't. No potential confounding factors. A 4 point increase in twenty-one years is frighteningly poor, with or without more kids taking the test. The College Board itself even notes that no positive trend in verbal scores can yet be established (verbal scores went down as recently as between 2001 and 2002). Second, and this goes back to graduation rates, ~2.6 million kids take the SAT or ACT annually, but there are more than 3.5 million kids per grade in high school. With only 0.2 million more kids taking the ACT in 2004 than 1994 and 0.4 million more kids now taking the SAT, the trajectory is underwhelming.

I think optimism is good. I'm optimistic that things can change, that there's a future in which every child gets an unparalleled education that provides him or her with the industry to achieve his or her highest potential. But we're not there right now and we're not on course, and any attempt to gloss over the harsh realities does a disservice both to ourselves and to the millions of children crying out for a quality education.

Ha!

Monday, September 19, 2005
A good laugh to cure that case of the Mondays coming out of Florida today:

Parents, students and gluttons for standardized tests will be able to take a sample version of Florida's high school graduation test when it is released online Wednesday.

The Florida Department of Education will post the 10th-grade version of the Florida Comprehensive Assessment Test on its Web site, www.fldoe.org. The version is an actual test that was administered in Spring 2004.

Previously, only a small number of questions were publicly released.

Education Commissioner John Winn said the sample test is intended to help students and families become comfortable with the look, feel and experience of the FCAT.

...Winn also said he expects critics of the FCAT to complain about questions that may seem too difficult. (emphasis mine)

Too difficult? That's rich.

Say, did you know they took the word gullible out of the dictionary, too?

Logical leap?

Saturday, September 17, 2005
Interesting:

Granholm calls for statewide mandatory curriculum
Associated Press

LANSING -- Gov. Jennifer Granholm called Friday for a mandatory statewide curriculum for Michigan's high school students.

It's another step toward possibly changing the state's mandatory requirements for high school graduates. The only current state requirement is a civics class, with the rest of the standards set by local school boards.

"With Michigan's economic future on the line, we can't afford to have our 500 local school districts marching in different directions," Granholm said in her weekly radio address. "Instead, we need a high standards, mandatory curriculum to get all our students on the road to higher education and a good paying job."


Without passing judgment on the merits of the proposal, I think it's a fairly remarkable statement because it opens the door for someone to come along with this one:

"With America's economic future on the line, we can't afford to have our 50 states and 15,000 local school districts marching in different directions. Instead, we need a high standards, mandatory curriculum to get all our students on the road to higher education and a good paying job."

I'm far, far, far from ready to say we need a national curriculum, but, tradition and law aside, let me ask this philisophical question: Why should Alabama get to teach its kids math differently than Iowa, especially if it does it worse?

Close, yet so far

Friday, September 16, 2005
California has come up with a remarkable idea: Let parents know how their children are doing on state assessments. The Sacramento Bee reports:

School districts across the state are sending out the one-page STAR Student Report to the homes of the state's estimated 6 million public school students. Educators see the reports, all of which should be mailed by the end of this month, as a road map to improving each student's performance.

[...]

The multicolored report, printed on both sides of a single sheet of paper, includes a bar graph indicating a child's score on each subject-related test taken - English language arts, math, science and history-social science - and where that score falls on the proficiency scale: far below basic, below basic, basic, proficiency and advanced.

The state's goal is for all students to be proficient or advanced in all subjects.

On the back page, each subject is broken down, with the percent of questions answered correctly by a child compared to the percent correct of students statewide. The percentages listed by each subject area give parents and teachers a chance to tailor their efforts to help the child.


In reality, I shouldn't mock; sending out 6 million reports is a fairly monumental logistical task. One of my favorite things about NCLB is that it's finally forcing states to develop the data systems and infrastructure they should have had years ago -- after all, 19 states (CA included) currently don't have longitudinal student databases, or, in lay terms, 19 states currently can't keep track of their students from one grade to the next. Happily, 15 of those 19 are working on them.

Here's what worries me: Parents have to take it on faith that their kids' test scores are legitimate. Teachers are at least engaged in the literal activity of education, so they can put a kid's inflated passing score on Texas' TAKS or Arizona's AIMS in context.

California's actually pretty decent when it comes to the legitimacy and rigor of their California Standards Test; they rate a "B" on Peterson and Hess' ranking, and only 40-50% of 4th graders generally pass the reading test (Aside: how sad is it that the lower the pass rate, the more likely it is to be legitimate?). But the fact remains that parents have no way to know the rigor of the questions, the cut-scores, the content standards, etc. The numbers are just that -- numbers. If a 4th grader is "proficient" in reading, what does that mean? The percentage of questions answered correctly is equally crude; getting 20% of extremely hard questions correct is very good while getting 80% of extremely easy questions correct isn't. And while the numbers can be compared with other numbers, that relative measure is hardly a declaration of where the student is in terms of reaching absolute proficiency, which must always be the goal.

The Bee reports that the score reports go so far as to assign the students a reading level based on their performance on the CST and offer a list of level-appropriate books. I suppose this may be the crux of it: Parents should be invested in their children's education to the point that they don't need a pretty piece of paper to tell them how well their son or daughter can read. That's idealistic, I know; parents don't always have the time nor ability to engage as much as they would like -- they can likely tell broadly if their kid can read or not, but how many immigrants and non-native English speakers know the difference between a 5th grade reading level and 6th grade reading level? So if score reports must be the supplement (and for the moment, there's nothing better; don't get me wrong, this is a positive development), then it's time we consider that parents might be willing to work their minds to truly understand where their son or daughter stands and how best to help them.

According to the Bee, "Officials at the state Department of Education say they worked with graphic designers, educators and parents to design something devoid of 'educationalese' that would be easy to read for all parents." That's fine. But it can't be something devoid of necessary substance. With a measure as imprecise, ambiguous and malleable as test scores, that substance goes beyond simple numbers and percentages.

The sound of silence

Apologies for the lack of any posts lately; this week has been crazy in terms of work. Except a post by the end of the day.

British math study

Sunday, September 11, 2005
In an additional effort to clarify why I believe that Authentic Education is the course that will ultimately lead to all children gaining the skills to achieve their highest potential, let's look at a research study. I've provided statements of ideology and evidence supporting its genesis, and I've demonstrated some concrete examples, so this "objective" level seems the natural next step. Sadly, there are so very few schools that actually embody what I consider Authentic Education that it's difficult to point to a bevy of scientific studies.

Moreover, schools aren't laboratories, and while controlled, carefully designed scientific experimentation is of course crucial and should always be the goal (and science should always be the basis for policy), the fact that one class of students can be markedly different from another class in the same school in the same grade leaves things thornier than in most any other field. Thus, this research approaches Authentic Education somewhat indirectly, but is still extremely edifying. I don't pretend to say that it proves my case conclusively nor that the methodology is perfect; I simply offer it as another piece of evidence. Keep in mind while you're reading that Authentic Education is not synonymous with constructivism and does not demand a one-size-fits-all style of teaching.

As an aside, after this post I'm going to try to move on in terms of post topics. Authentic Education will be a heavily recurring theme on the blog, but there are plenty of other things going on right now in the world of education policy that are worth talking about. So, without further ado, the study I'm going to quote at length was explained by its author in EdWeek back in 1999:

Two schools in England were the focus for this research. In one, the teachers taught mathematics using whole-class teaching and textbooks, and the students were tested frequently. The students were taught in tracked groups, standards of discipline were high, and the students worked hard. The second school was chosen because its approach to mathematics teaching was completely different. Students there worked on open-ended projects in heterogeneous groups, teachers used a variety of methods, and discipline was extremely relaxed. Over a three-year period, I monitored groups of students at both schools, from the age of 13 to age 16. I watched more than 100 lessons at each school, interviewed the students, gave out questionnaires, conducted various assessments of the students' mathematical knowledge, and analyzed their responses to Britain's national school-leaving examination in mathematics.

At the beginning of the research period, the students at the two schools had experienced the same mathematical approaches and, at that time, they demonstrated the same levels of mathematical attainment on a range of tests. There also were no differences in sex, ethnicity, or social class between the two groups. At the end of the three-year period, the students had developed in very different ways. One of the results of these differences was that students at the second school--what I will call the project school, as opposed to the textbook school--attained significantly higher grades on the national exam. This was not because these students knew more mathematics, but because they had developed a different form of knowledge.

At the textbook school, the students were motivated and worked hard, they learned all the mathematical procedures and rules they were given, and they performed well on short, closed tests. But various forms of evidence showed that these students had developed an inert, procedural knowledge that they were rarely able to use in anything other than textbook and test situations. In applied assessments, many were unable to perceive the relevance of the mathematics they had learned and so could not make use of it. Even when they could see the links between their textbook work and more-applied tasks, they were unable to adapt the procedures they had learned to fit the situations in which they were working.

The students themselves were aware of this problem, as the following description by one student of her experience of the national exam shows: "Some bits I did recognize, but I didn't understand how to do them, I didn't know how to apply the methods properly."

In real-world situations, these students were disabled in two ways. Not only were they unable to use the math they had learned because they could not adapt it to fit unfamiliar situations, but they also could not see the relevance of this acquired math knowledge from school for situations outside the classroom. "When I'm out of here," said another student, "the math from school is nothing to do with it, to tell you the truth. Most of the things we've learned in school we would never use anywhere."

Students from this school reported that they could see mathematics all around them, in the workplace and in everyday life, but they could not see any connection between their school math and the math they encountered in real situations. Their traditional, class-taught mathematics instruction had focused on formalized rules and procedures, and this approach had not given them access to depth of mathematical understanding. As a result, they believed that school mathematical procedures were a specialized type of school code--useful only in classrooms. The students thought that success in math involved learning, rehearsing, and memorizing standard rules and procedures. They did not regard mathematics to be a thinking subject. As one girl put it, "In math you have to remember; in other subjects you can think about it."

The math teaching at this textbook school was not unusual. Teachers there were committed and hard-working, and they taught the students different mathematical procedures in a clear and straightforward way. Their students were relatively capable on narrow mathematical tests, but this capability did not transfer to open, applied, or real-world situations. The form of knowledge they had developed was remarkably ineffective. At the project school, the situation was very different. And the students' significantly higher grades on the national exit exam were only a small indication of their mathematical competence and confidence.

The project school's students and teachers were relaxed about work. Students were not introduced to any standard rules or procedures (until a few weeks before the examinations), and they did not work through textbooks of any kind. Despite the fact that these students were not particularly work-oriented, however, they attained higher grades than the hard-working students at the textbook school on a range of different problems and applied assessments. At both schools, students had similar grades on short written tests taken immediately after finishing work. But students at the textbook school soon forgot what they had learned. The project students did not. The important difference between the environments of the two schools that caused this difference in retention was not related to standards of teaching but to different approaches, in particular the requirement that the students at the project-based school work on a variety of mathematical tasks and think for themselves.

When I asked students at the two schools whether mathematics was more about thinking or memorizing, 64 percent of the textbook students chose memorizing, compared with only 35 percent of the project-based students. The students at the project school were less concerned about memorizing rules and procedures, because they knew they could think about different situations and adapt what they had learned to fit new and demanding problems. On the national examination, three times as many students from the heterogeneous groups in the project school as those in the tracked groups in the textbook school attained the highest possible grade. The project approach was also more equitable, with girls and boys attaining the different grades in equal proportions.

It would be easy to dismiss the results of this study because it was focused on only two schools, but the textbook school was not unusual in the way its teachers taught mathematics. And the in-depth nature of the study meant that it was possible to consider and isolate the reasons why students responded to this approach in the way that they did. The differences in the performance of the students at the two schools did not spring from "bad'' teaching at the textbook school, but from the limitations of drawing upon only one teaching method. To me, it does not make any sense to set any one particular teaching method against another and argue about which one is best. Different teaching methods do different things. We may as well argue that a hammer is better than a drill. Part of the success of the project school came from the range of different methods its teachers employed and the different activities students worked on.


So, there's another block on the tower.

Two examples of Authentic Education

Saturday, September 10, 2005
Debates over superior pedagogy in the conceptual realm are all well and good, but it's edifying to occassionally bring it to a very concrete level. I want to talk about two schools today, an elementary school and a middle/high school, both of which demonstrate amazing results on the back of programs in the neighborhood of my ideal Authentic Education. First we're going to visit Dayton's Bluff Elementary of St. Paul, MN and then the incredible H-B Woodlawn program of Arlington, VA.

Dayton's Bluff was profiled by the pro-NCLB Achievement Alliance. The piece was written by a colleague of mine associated with the Education Trust, a woman for whom I have a great deal of respect. Dayton's Bluff is a high-minority, high-poverty school, that, as the profile notes, five years ago "was known as the worst school in St. Paul and one of the worst in Minnesota." After recieving a new principal and initiating a new program of reforms, Dayton's Bluff has gone from under 20% of third-graders passing proficient on both the reading and math MCA to over 80%.

How was it done? There were plenty of factors, such as the new principal and an increase in community resources, but pedagogically the shift was to the America's Choice program developed by the National Center on Education and the Economy.

The way America’s Choice works is that schools adopt the NCEE’s “New Standards” – which are aligned with but more rigorous than Minnesota standards -- and assign at least two coaches, a design coach and a literacy coach, to work with teachers on organizing their classrooms. America’s Choice classrooms are built on a workshop model, in which teachers present ten minute, very focused, mini-lessons and then lead students in independent and group work and a classroom-wide sharing of results, all geared to the standards.


Note how this relates back to the previous discussion of constructivism; it isn't thematic learning, it isn't fact-less learning, and it isn't anti-intellectual anti-rigor. This is an example of a sustainable reform that pedagogically resembles Authentic Education on at least some levels. It also shows, of course, that correct pedagogy is a necessary but not sufficient cause for successful learning; a great style of teaching still needs good teachers, a good principal and good resources. But similarly, good teachers, a good principal and good resources aren't going to impart the deep critical thinking skills doing drill-and-kill, for example. The pedagogy is the most fundamental layer, and everything builds from that.

Dayton's Bluff, happily, had something of a "perfect storm" of these factors -- but it's not random, and it's somewhat replicatable. There is no one-size-fits-all model of educational reform, which in some ways is the beauty of Authentic Education; it is a broad set of principles that can be tailored to individual schools, given the appropriate tools for implementation.

One school which is markedly different from Dayton's Bluff but still embodies these precepts is H-B Woodlawn. Woodlawn is a relatively small, public "alternative school," and one has to apply to get in; however, by law the application process is decided by random lottery weighted to ensure a locally proportional makeup (6% black, 19% hispanic, 15% FRL). The hallmark of Woodlawn is student involvement; throughout the grades (one can start in the program at grade 6) students recieve incrementally more authority over their own education. This comes in the amount of "unsupervised" time students recieve to do with as they wish -- study, extracurricular activities, take a nap, etc. Up to the student. Additionally, Woodlawn students participate in the governance of their school via regular "Town Hall" meetings where substantive school policy issues are decided, and by sitting on faculty hiring committees.

Most importantly, students take an active role in the classroom. An article in the Virginia Education Association's Journal (not online) several months ago profiled a 7th and 9th grade science class at Woodlawn. Coyotes had been sighted in the woods around the school after no sightings for years, and the students wanted to explore the hypotheses that coyotes had returned to Arlington. Guided by the teachers, the students went about designing an experiment, learning the local topography and ecosystem, writing grant proposals for the money to purchase GPS equipment and laser-guided cameras, and then performing the lab. This style of hands-on student-led learning is by all reports seen throughout Woodlawn.

Students with unsupervised time, students on the faculty hiring committees, students choosing their own projects -- recipe for disaster, right? Woodlawn is consistently ranked as the top high school in Arlington and one of the top high schools in the nation. The average SAT score hovers around 1200, 150 points higher than the national average. Over 90% of the students go on to college, and not just any colleges -- among others, the class of 2004 sent students to Harvard, Stanford, William & Mary, Penn State, Johns Hopkins and my very own University of Virginia.

Authentic Education works not only in theory, but in practice; The H-B Woodlawn program and Dayton's Bluff are just two tangible examples.

On constructivism

Friday, September 09, 2005
Constructivism is one of the most misunderstood terms in the contemporary debate over education policy, and that's showing through in the discussion on the comments in the post below this one. If you asked ten wonks to define constructivism, you'd get eleven different answers. In order to have a frutiful debate, let's lay out a working definition and then explore its nuances.

Constructivism is a pyschological principle developed by Jean Piaget (1896-1980). While critics throughout the years have challenged Piaget's specific assertions, his broad explanation of cognitive development has stood the test of time. Instead of trying to define constructivism my college-student self, I'll let Snowman & Biehler's 11th edition "Psychology Applied to Teaching" do it for me:

Piaget believed that people are driven to organize their [mental] schemes in order to achieve the best possible adaptation to their environment. He called this process equilibration. But what motivates people's drive toward equilibration? It is a state of disequilibrium, or a perceived discrepancy between an existing scheme and something new. In other words, when people encounter something that is inconsistent with or contradicts what they already know or believe, this experience produces a disequilibrium that they are driven to eliminate.

[...]

Meaningful learning, then, occurs when people create new ideas, or knowledge (rules and hypotheses that explain things), from existing information (for example, facts, concepts and procedures). [emphasis mine] To solve a problem, we have to search our memory for information that can be used to fashion a solution. Using information can mean experimenting, questioning, reflecting, discovering, inventing, and discussing. This process of creating knowledge to solve a problem and eliminate a disequilibration is referred to by Piagetian psychologists and educators as constructivism.


Or, if Piaget may be allowed to speak for himself:

[G]lobal transformations of the objects of perception, and of the very intelligence which makes them, gradually denote the existence of a sort of law of evolution which can be phrased as follows: assimilation [interpreting an experience so that it fits an existing scheme] and accommodation [changing an existing scheme to incorporate the experience] proceed from a state of chaotic undifferentiation to a state of differentiation with correlative coordination.

So now we can start actually talking about constructivism; what it is, and what it isn't. What constructivism isn't is a form of pedagogy that takes facts, prior knowledge or teachers out of the equation. Any program that does that (and some do exist) are mangling the idea. It's right there -- in order to construct learning, you must have preexisting knowledge. That means you don't assume children are going to actively participate in learning to read without teaching them the alphabet. Constructivism is not simply group projects all the time. Constructivism, properly understood, is not at loggerheads with lectures; it simply puts the lectures in their correct context.

Constructivism when applied appropriately causes the learner to ingrain the new knowledge into their schemes. This is a permanent, flexible knowledge that stands in stark contrast to the surface knowledge gleaned through direct instruction and rote memorization. When I speak of Authentic Education, it's not synonymous with constructivism, but it is grounded in Piaget -- most deeply the idea that critical thinking skills are best gleaned by heading down this general bend in the road.

Discourse is a lot healthier when everyone's on the same page. I hope this helps clarify the terms of the debate, at least for the purposes of this little corner of cyberspace.

Authentic education

Wednesday, September 07, 2005
A great story is coming out of St. Paul today where thousands of middle schoolers are beginning school with a new, excellent type of pedagogy in place. As reported in the St. Paul Pioneer Press:

...[S]tarting last year, an attempt to revamp the middle school/junior high instructional model with an approach called "disciplinary literacy" was rolled out at four schools. This year, it will be instituted at all eight, plus the K-8 Monroe Community School.

The idea of disciplinary literacy is to train students to approach science, math, English and history "as a practitioner, rather than studying it," said Mike McCollor, principal at Washington, the first school to get the new program this year.

In science, for example, that might mean students collect leaves and start classifying them themselves rather than reading about taxonomy in a text and getting a lecture about it, said Davis, a coach brought in to help science teacher Sarah Weaver run her class according to the new model.

"(It's) doing science but learning content," Davis said.

As seventh-grader Brian King summarized it after hearing the science teachers introduce the concept: "They don't tell you, they let you figure it out."

English teacher Suzanne Myhre said she intends to have her students form strong opinions on the works they'll be reading as a way of deepening their understanding.

Students in Tara Brash's math class were asked Tuesday to figure out how to form eight triangles out of six toothpicks.

They were given some time to work on their own, but were soon grouped with others to share their ideas.

"In math, there is not always one single correct answer," Brash told the students.

Group work is a hallmark of the new approach, as is the posting of student work and ideas.

In Courtney Major's history class later in the day, student responses to the questions "What is history?" and "Why study history?" were recorded on big pieces of newsprint taped to the wall at the front of the classroom.


It's more than a bit sad that it's an "innovative" idea to have students engage in hands-on understanding rather than sit as mostly passive receptacles for information poured out of the hands of teachers. But this is an excellent example of what I'm talking about when I mean teaching thinking. I contacted the article's author to find out which schools had a year of the program under their belt. These four schools provide a stirring testament to the power of what can only be termed authentic education.

These schools are all racially diverse and, with the exception of Highland Park, have over 80% of students on Free and Reduced Lunch. The data show percent proficient on the 7th grade Minnesota Comprehensive Assessments.

Humboldt (90% FRL)
Reading, Gr. 7
2004 -- 21
2005 -- 32

Math, Gr. 7
2004 -- 18
2005 -- 34

Cleveland (91% FRL)
Reading
2004 -- 28
2005 -- 34

Math
2004 -- 20
2005 -- 45

Highland Park (65% FRL)
Reading
2004 -- 47
2005 -- 60

Math
2004 -- 45
2005 -- 59

Battle Creek (81% FRL)
Reading
2004 -- 27
2005 -- 43

Math
2004 -- 26
2005 -- 37

If you've been reading this blog, you know I'm skeptical of test results generally, but an aggregate jump of 15-20% in one year is truly astounding. At that rate, basic proficiency truly will be achieved in short order. We can further see the effects of the student-oriented program by comparing these schools with other St. Paul middle schools that didn't have disciplinary literacy last year. While these schools also made solid gains, they were not nearly as huge even though the non-program schools had smaller challenges to overcome.

Murray (62% FRL -- not DL program)
Reading
2004 -- 54
2005 -- 61

Math
2004 -- 55
2005 -- 64

Ramsey (68% FRL -- not DL program)
Reading
2004 -- 49
2005 -- 57

Math
2004 -- 51
2005 -- 62

Disciplinary literacy. Authentic education. Now we're getting somewhere.

It's the pedagogy, stupid

Monday, September 05, 2005
In order to stop beating what I hope is a dead horse, let's just stipulate for the moment that our schools aren't where they need to be and no current policy is impelling them forward adequately. The question then becomes: What's going wrong, and how do we fix it?

Many possible answers immediately come to mind: Funding, teacher quality, administrative structure, school choice, etc. All of these areas are certainly critical -- funding and teacher quality especially -- but they are all overlaid on a central tenet.

It's all about the pedagogy.

What our schools are set up to teach is information, not understanding. One only needs to look at the tests which assess "proficiency" to see endless reams of evidence for this truism. Teachers are now being given "pacing guides" telling them not only what to teach but when to teach it and when to move on to the next topic. To suggest that schools are teaching the lowest common denominator of skills isn't new or profound; what most people miss is that failing to teach thinking skills results in a failure of teaching any skills whatsoever.

There was an important report authored by Lauren Resnick back in 1987 that still rings with urgency on this subject. Resnick, tremendously well-respected in the field, a professor at the University of Pittsburgh and director of the New Standards project, wrote "Education and Learning to Think" for the National Research Council's Commission on Behavioral and Social Sciences and Education. Resnick writes:

The most important single message of modern research on the nature of thinking is that the kinds of activities traditionally associated with thinking are not limited to advanced levels of development. Instead, these activities are an intimate part of even elementary levls of reading, mathematics, and other branches of learning -- when learning is proceeding well. In fact, the term "higher order" skills is probably itself fundamentally misleading, for it suggests that another set of skills, presumably called "lower order," needs to come first. This assumption -- that there is a sequence from lower level activities that do not require much independent thinking or judgment to higher level ones that do -- colors much educational theory and practice. Implicitly at least, it justifies long years of drill on the "basics" before thinking and problem solving are demanded. Cognitive research on the nature of basic skills such as reading and mathematics provides a fundamental challenge to this assumption.

Indeed, research suggests that failure to cultivate aspects of thinking such as those listed in our working definition of high order skills [nonalgorithmic, complex, yielding multiple solutions, involving nuanced judgment, uncertainty, imposing meaning, etc.] may be a source of major learning difficulties even in elementary school.

[...]

Cognitive theory... suggests that processes traditionally reserved for advanced students -- that is, for a minority who have developed skill and taste for interpretive mental work -- might be taught to all readers, including young children and, perhaps especially, those who learn with difficulty. Cognitive research suggests that these processes are what we mean by reading comprehension. Not to teach them is to ignore the most important aspects of reading.

How often have advocates of the Standards Movement responded to criticism about the degree of rigor with "let's get these kids the most basic skills; thinking comes later"? It's a reasonable thought, and, as I've said before, it's great that the Standards Movement is putting all kids and quality education in the same sentence for the first time. But it's an incorrect thought. You want to know why NCLB isn't doing the trick, why I say we need a fundamental evolution? Here's the start of my hypothesis: By ignoring the teaching of thinking skills, growth is being stunted across the board.

Dual crises

Sunday, September 04, 2005
The unspeakable tragedy of Hurricane Katrina seems hardly related to education, yet when I gaze at the images beaming back from New Orleans, all I can think about is education, education, education.

The refugees are mostly poor. And, for the first time, the media appears to have woken up and realized that poverty can be crippling. I can't help but be reminded of Barbara Ehrenreich's monograph Nickel and Dimed, which demonstrates the incredible challenges facing the impoverished.

Having never been poor myself, it's hard for me to speak with any authority. But to me, the most striking thing about poverty is the litany of subtle hurdles. Not having a car, not having the social knowledge to work through the bureaucracy, not being able to eat heathily and as a result incurring medical costs you can't pay which in turn cause you to miss work so you have less money with which to buy any food at all, much less healthy food... the little things that we in the middle class don't even think about.

There's a reason only 7% of kids born into low-income families get a Bachelor's degree by age 26, and it's not because the other 93% are stupid and/or lazy. Breaking news: Education level and quality is directly related to income.

I'm often asked how I can have such an unyielding faith in public education. My answer is that education is the great equalizer. School is the one chance we have as a society to level the playing field. The one institution which every student must attend for 9 or if we're lucky 12 or if we're very lucky 16 years. An entire system devoted to the singular idea that no matter where you enter, you should leave with every skill necessary to achieve your highest potential. I don't know much that's better to have unyielding faith in.

What distresses me about the current state of education is that the institution seems broken to its very core. The sheer amount of progress that remains to be made and the torturous, slogging growth that is being made -- in other words, what I've been posting about for the past two weeks -- leads to an inescapable conclusion: We have to look deeper for both the causes and the solutions. No Child Left Behind doesn't eradicate 100,000 poor black New Orleaners, nor does the repeal of NCLB.

Our schools aren't teaching our children how to think. There it is, plain and simple as I can put it. Our schools aren't teaching our children how to think. And ultimately, thinking is the capital of opportunity. I believe that we are slowly laying the groundwork but misinterpreting it as tiling the roof. Too slowly. What we need is a punctuated evolution, an ascension to a style of education that takes every child and says not only are we not going leave you behind, we're going to give you the tools to get ahead.

Untold hundreds of people are dead in New Orleans because they didn't have the money to get out.

Education is everything.

Education's California Adventure

Thursday, September 01, 2005
Something strange is afoot in California -- or is it? The L.A. Times reports:
Nearly one-third of California public schools won praise today for meeting state achievement goals on test scores even while they were branded as failures for missing a federal gauge of success...

Campus leaders were left to decipher the differences between California's Academic Performance Index, which rewards incremental test score gains, and the federal No Child Left Behind law, which requires schools to clear a rigid achievement bar that rises regularly...

State education officials said they were pleased that 81% of the schools had met their state improvement targets, up from 64% last year.

But the officials were not happy about the results under the federal system: nearly 2,300 schools that met their state targets still fell short of the No Child Left Behind goal. More than 5,100 California schools met their federal goals.

That's because the federal bar rose for the first time this year, leaving many campuses unable to reach it.

To pass the federal bar this year, elementary schools and middle schools had to raise at least 24% of their students to the proficient level in English-language arts, up from nearly 14% last year.
What we have here is a case of a growth model going up against a status model. This is a topic near and dear to my heart, as I spent many weeks working on it during my EdTrust internship. Simply put, a growth model looks at a student or group of students' progress over time. A status model just checks to see if student achievement is above or below a certain line.

(Note: A value-added growth model, if you hear that term, refers to a growth model that tries to isolate student progress to a specific time and agent, usually with the help of statistics. The API, for reasons that will soon become clear, is not a pure value-added growth model)

California's API, which is based off of but separate from the state tests which are used for federal accountability, is a growth model of sorts. The API essentially synthesizes a school's performance on various tests, weighting each test differently (so that the math and reading California Standards Test are the most important, for example), and comes up with a score. The scores range from 200 to 1000, with 800 the statewide target. Each year, schools have an API growth target which is 5% the difference between their current score and 800. So, a school with a current API of 600 has a growth target of .05 x (800-600), or 10 points. Thus, schools further away from the standard have to make more annual growth.

Now, here's what you need to know about the API: It's not impressive. There are no real consequences to a school for failing to make its API growth target (go go gadget accountability!), and no substantive incentives for succeeding. Moreover, since California doesn't have unique student identification numbers yet, the API isn't even a true growth model -- it looks at one class of students compared to the next year's class of students (e.g. 2004 fourth graders vs. 2005 fourth graders) rather than tracking groups. Worse yet, each relevant subgroup -- racial, etc. -- only has to make 80% of the school's overall growth target to meet API regulations. In other words, a school's Latino students can only make 8 points of growth in our above example so long as the white students compensate for them. Soft bigotry of low expectations, anyone?

Growth models have a lot to be said for them if designed effectively and applied correctly. The API has some good aspects, but is in the aggregate weak. To be sure, status models like AYP also have plenty of flaws, such as ignoring the progress of previously low-achieving schools and holding schools accountable for students whose educations may have been previously slowed by other educative institutions. But let me say this -- Asking that one-quarter of your students be proficient in English language-arts is not unreasonable! Even in such a multiracial and multilingual state as California, 25% isn't exactly reaching for the sky. It's barely reaching for the ground.

So, on this count I'm squarely in the corner of the big bad federal government. That more schools are meeting their API growth targets is good news (though i'd like to see how many schools at the low end made their targets vs. schools already in the 700s), but adulation should be tempered by an understanding of just what the API is. California's defensiveness is like trying to claim victory after plugging one hole in a swiss cheese boat. The untold tens of thousands of California's kids who still can't read are the real story.