Today’s topic in our Education in America series is standardized tests. As Mona and I have touched on briefly, they’re not the silver bullet which will expose and then fix all the problems in education. In fact, they might be making things worse. Let’s dig in, shall we?
Let’s start by comparing standardized tests to something we’re all familiar with and potentially misinformed about: dreaded BMI. The term and mathematical formula used to calculate BMI were created by Belgian Adolphe Quetelet in the mid 1800s, as a tool for measuring the general health of entire populations, not as a diagnostic tool for individuals. So, for example, if scientists wanted to examine the obesity rate in Paris vs. the obesity rate in Kansas City, MO, they would compare the average BMI of the respective populations, not individual BMI ratings. It’s not an accurate measure of individuals, because it creates a one-size-fits all picture of how bodies are constructed. Yet, nearly every ladymag, ladyblog, doctor’s office and a handful of late night infomercials will tell you it’s a great way for you to assign yourself a label, from underweight to obese.
Standardized tests are great for measuring strengths and weaknesses across large populations or demographic groups of children, but much like BMI, they don’t effectively measure the strengths and weaknesses of individual students or individual classrooms/teachers. This is why many teachers’ unions are fighting against merit pay – most unions are supportive of a merit-based pay system, as long as the tools used to measure a teacher’s effectiveness are both fair and accurate. Most state-wide standardized tests, the gold standard by which schools, administrators and teachers are weighed and measured, can’t possibly accurately or effectively measure any of the three. Here’s why, with bullet points, because I know these pieces get a li’l dry.
- Standardized tests are snapshots in time. You’ll hear teachers say this a lot, but it means more than you think it means. Standardized tests are given over a series of days; usually all public schools in a state will administer the tests within the same two-week window. Ideally, the standardized tests are aligned with both the state’s academic standards and the school’s curriculum, which gives students the highest chance of getting passing scores. A sixth grade student in Indiana, for example, will take the ISTEP test in April and May, and it will cover both fifth grade skills and sixth grade skills. Because it’s standardized, the tests are limited to multiple choice questions, except for essay writing, which is graded by people like this person reporting from the trenches of the scoring centers. The multiple choice format, as any education professional will tell you, is an ineffective measure of how much a student grasps about a concept, as it doesn’t venture north of the bottom sliver of Bloom’s. (I love my Bloom’s. Wait until I introduce you to Gardner’s theory of multiple intelligences.) On one tiny set of days which could be months or seasons after little Jimmy and Mary Sue thought they’d never have to multiply another fraction or identify the object of another preposition.
- Standardized tests can’t be teaching tools. The best and most effective student evaluation tool is the one you can interpret RFN to identify individual student strengths and weaknesses. In Indiana, the scores from the April tests aren’t returned until the following school year, when those hypothetical sixth graders are in seventh grade. While testing companies have gotten better about giving schools disaggregated data relating to specific skills or state academic standards as the years have gone on, scores aren’t individualized and itemized enough to be used as tools to guide future instruction.
- Standardized testing isn’t always fair or accurate. The rigidity of standardized testing ironically makes it biased against students who don’t speak English as a first language (in Indiana, immigrant children are only allowed to take the ISTEP test in their native language for one year after they enroll in an Indiana public school) and students who learn differently. Both groups have abysmal passing rates across all district types (urban, suburban and rural), and as a result many of these students are clustered in one or two schools per district. Since NCLB sanctions require schools that repeatedly don’t meet AYP to eventually restructure the staff from the administration to the secretary, these cluster schools are often in constant chaos, with high staff turnover, painful budget cuts and limited community involvement. If a city, district or staff can’t commit to saving a failing school, neither will the community surrounding it.
- Testing bias. Standardized tests are created by people who often have very few experiences in common with the individuals who are taking the tests. Being crucial in the design of a Massively Multi-Player Standardized Test requires at least a Master’s degree or higher, which we learned a few weeks ago only includes 8% of our American population, although more recent data I found (from 2007 instead of 2003) shows 10% of Americans with some type of advanced (beyond a Bachelor’s) degree. That already disproportionate number also contains only 3.9% of the Hispanic (of all races) population and 5.8% of Blacks. While there are statistical measures which can lessen bias, the fact that a very limited group of people is creating the yardstick by which all of our children are measured is worthy of questioning.
- Meeting NCLB goals is impossible with norm-referenced tests. NCLB requires that all students be 100% proficient in grade level skills by 2014. Using norm-referenced tests (in which scores naturally form a bell curve) makes it impossible for 100% of students to pass. If 100% of students got perfect scores, it would be an indicator of a very bad norm-referenced test.
To sum up, the collective interpretation of both BMI and standardized tests teaches us one important lesson: in our data-driven society, we need to do a better job in teaching our citizens about statistics.