Test me
I've long made the contention that most multiple-choice standardized tests don't assess high-level critical thinking skills such as evaluation, judgment, inference and connection of disparate information. At the same time, I've also held that widespread assessment is crucial and necessary for an accountability system which ensures all students are being well served. A commenter below asked what my ideal standardized test would look like. By way of beginning to answer that, I think it's worthwhile to examine better and worse types of exams.
I loathe multiple choice items, and there's a reason -- the vast majority of them test nothing but basic information recall or one-step understanding. For example, these question from the 2004 8th grade Virginia science SOL:
15. Which of these planets in the solar system was the most recently discovered?
A Mars
B Venus
C Jupiter
D Pluto
20. Which biome contains large populations of grazing herbivores, few species of birds, and deep, rich soil?
F A taiga
G A tundra
H A deciduous forest
J A grassland
We can cross the country to see this at work in Oklahoma's 8th grade history end-of-course exam:
19. What battle convinced British leaders to begin the peace talks that ended the American Revolution?
A Charleston
B Camden
C Yorktown
D Savannah
29. Under the Articles of Confederation, Congress had no power to
A declare war
B impose taxes
C maintain an army
D regulate foreign trade
So what's the alternative? Luckily, we don't have to reinvent the wheel. Various former tests (most of which have fallen out of use because standardized MC tests are logistically simpler) provide a blueprint pointing towards the type of assessment we should want. Take the Maryland School Performance Assessment Program (MSPAP) which was in place throughout the 1990s. These tended to be weeklong interdisciplinary assessments which drew on a combination of short-answer, extended-response, a little MC and some group work to really provide a more holistic picture. For instance, take the start to this 1997 8th grade science assessment:
You belong to a team of scientists working at the Goddard Space Flight Center in Greenbelt, Maryland. For years, you have been collecting information about our solar system sent to you by a Voyager space probe. Recently, the probe has traveled out of our solar system. Imagine that Voyager is sending back data about an entirely new solar system. Here are some basic facts
about this new solar system:
• There are four planets revolving around a central star.
• For easy reference, the planets have been assigned symbols instead of names (triangle, x, circle, square).
• The planets have nearly circular orbits that do not overlap.
The table below and the diagram on page 30 of your Student Response Book show the information we currently have about the new planets in this new solar system. Study the charts to look for patterns in the data and to decide what additional information you would like to have to describe this solar system and its planets as completely as possible.
[table with data (tilt of axis, surface temp, number of moons, composition) and a diagram of the planets orbits]
Part A: Describe a pattern in this data which is similar to data from our solar system.
Part B: On the lines below, write AT LEAST three questions (more if you can) about additional scientific data on the new solar system and its planets that could be gathered with further study.
2. Working with the members of your group, use the data in Table II and the “Orbit Diagram” to complete the “Orbit Data Logs” on page 33. Draw in the symbols of the planets in their correct positions and order for January, February, March, and April. You may use the “Orbit Data Logs” worksheet to prepare a first draft of these drawings and to make certain that all members of your group agree on the answer. Then copy your group’s answers onto each of your “Orbit Data Logs” on page 33.
Later, questions include asking the students to think about how predictions they made in one part of the exam related to predictions they made earlier and to "Write a description of how your ideas or the ideas of others were influenced by working in a group."
I'm not going to claim that the MSPAP is a perfect exam by any strech of the imagination, but just consider the contrast between it and the SOL! The MSPAP requires active engagement of far more rigorous thinking skills and would be nearly impossible to pass without both scientific knowledge and knowledge of scientific thinking. It can also align nicely to a curriculum which is designed to teach those skills.
The downside of tests like the MSPAP is primarily logistic. They take longer to administer, they cost more, they take longer to score and the scoring is less "objective." Yet if logistics are the controlling factor keeping us from implementing a far superior system which will improve pedagogy and student learning, then logistics are a pill worth swallowing.
There are better tests and worse tests. The overwhelming bulk of state assessments fall into the latter category. What exams like the MSPAP show us, however, is that there is not an innate disconnect between teaching critical thinking and assessment/accountability. It's just a disconnect that currently exists and desparately needs to be addressed.
I loathe multiple choice items, and there's a reason -- the vast majority of them test nothing but basic information recall or one-step understanding. For example, these question from the 2004 8th grade Virginia science SOL:
15. Which of these planets in the solar system was the most recently discovered?
A Mars
B Venus
C Jupiter
D Pluto
20. Which biome contains large populations of grazing herbivores, few species of birds, and deep, rich soil?
F A taiga
G A tundra
H A deciduous forest
J A grassland
We can cross the country to see this at work in Oklahoma's 8th grade history end-of-course exam:
19. What battle convinced British leaders to begin the peace talks that ended the American Revolution?
A Charleston
B Camden
C Yorktown
D Savannah
29. Under the Articles of Confederation, Congress had no power to
A declare war
B impose taxes
C maintain an army
D regulate foreign trade
So what's the alternative? Luckily, we don't have to reinvent the wheel. Various former tests (most of which have fallen out of use because standardized MC tests are logistically simpler) provide a blueprint pointing towards the type of assessment we should want. Take the Maryland School Performance Assessment Program (MSPAP) which was in place throughout the 1990s. These tended to be weeklong interdisciplinary assessments which drew on a combination of short-answer, extended-response, a little MC and some group work to really provide a more holistic picture. For instance, take the start to this 1997 8th grade science assessment:
You belong to a team of scientists working at the Goddard Space Flight Center in Greenbelt, Maryland. For years, you have been collecting information about our solar system sent to you by a Voyager space probe. Recently, the probe has traveled out of our solar system. Imagine that Voyager is sending back data about an entirely new solar system. Here are some basic facts
about this new solar system:
• There are four planets revolving around a central star.
• For easy reference, the planets have been assigned symbols instead of names (triangle, x, circle, square).
• The planets have nearly circular orbits that do not overlap.
The table below and the diagram on page 30 of your Student Response Book show the information we currently have about the new planets in this new solar system. Study the charts to look for patterns in the data and to decide what additional information you would like to have to describe this solar system and its planets as completely as possible.
[table with data (tilt of axis, surface temp, number of moons, composition) and a diagram of the planets orbits]
Part A: Describe a pattern in this data which is similar to data from our solar system.
Part B: On the lines below, write AT LEAST three questions (more if you can) about additional scientific data on the new solar system and its planets that could be gathered with further study.
2. Working with the members of your group, use the data in Table II and the “Orbit Diagram” to complete the “Orbit Data Logs” on page 33. Draw in the symbols of the planets in their correct positions and order for January, February, March, and April. You may use the “Orbit Data Logs” worksheet to prepare a first draft of these drawings and to make certain that all members of your group agree on the answer. Then copy your group’s answers onto each of your “Orbit Data Logs” on page 33.
Later, questions include asking the students to think about how predictions they made in one part of the exam related to predictions they made earlier and to "Write a description of how your ideas or the ideas of others were influenced by working in a group."
I'm not going to claim that the MSPAP is a perfect exam by any strech of the imagination, but just consider the contrast between it and the SOL! The MSPAP requires active engagement of far more rigorous thinking skills and would be nearly impossible to pass without both scientific knowledge and knowledge of scientific thinking. It can also align nicely to a curriculum which is designed to teach those skills.
The downside of tests like the MSPAP is primarily logistic. They take longer to administer, they cost more, they take longer to score and the scoring is less "objective." Yet if logistics are the controlling factor keeping us from implementing a far superior system which will improve pedagogy and student learning, then logistics are a pill worth swallowing.
There are better tests and worse tests. The overwhelming bulk of state assessments fall into the latter category. What exams like the MSPAP show us, however, is that there is not an innate disconnect between teaching critical thinking and assessment/accountability. It's just a disconnect that currently exists and desparately needs to be addressed.
This entry was posted
on Monday, January 02, 2006 at 12:56 PM.
|
