NAEP and Comparing Schools

Now that NAEP and state scores are being released, the inevitable comparisons and ranting about the decline of America has begin. While I am in no way saying we are living in Lake Wobegon, focusing on the scores of American high school seniors in NAEP assessments is problematic at best. These tests, like the international TMSS, are completely voluntary, as students are asked to miss class time to take a test that will have no effect on their grades. Often, as my students have noted, the most motivated American students are unwilling to miss class time to participate, and they won't put the extra effort in to a test that means nothing. When students are asked to voluntarily take a zero-stakes test, there must be a consideration for the dubious quality of the results. In many districts nationwide, state test scores have gone down as ACT/SAT/PSAT/AP/IB exam scores have increased. Slate Magazine has an effective commentary on this:

http://www.slate.com/id/2124163/

The comparison to Asian schools is another problem, as it completely discounts cultural differences that lead to a widely different educational environment. The issue of school discipline alone - expectations of students and authority of school personnel to deal effectively with problems - makes a direct comparison nearly impossible and, again, disingenuous. Having lived and taught English in Taiwan for five years, I can assert that the environments simply do not translate. I can picture classrooms of 80-100 students who are sitting still in their chairs and vigorously writing down everything the teacher says, nearly word for word, as the teacher stands with a microphone and reads out of a book. Any disciplinary problem is dealt with immediately and harshly, and disruptive students do not have a “property right” to stay in the classroom. In fact, any non-academically motivated students are eliminated from the school by sixteen at the latest. Thus, their scores do not skew the NAEP and international test results, as they can in the U.S.

Additionally, people who haven't lived in Asia have no understanding of just how vast the cultural differences are and how deeply that can affect school culture and test scores. On the day that junior high school students in South Korea take high school entrance exams, the country shuts down air traffic for a half hour so the testing students can have absolute quiet during the oral part of the English exam. They hold national celebrations in Korea on the day their students take the international tests to promote national pride. Students in Taiwan who don’t test into a college-bound junior high school effectively eliminate their option of college at the age of twelve. Schools in Japan lock their gates at the start of school, and several years ago a high school student who was a few seconds late was crushed to death by the gate that is controlled by a timer. Students in Taiwan leave school at four and go to English and Math/Science cram schools from sometimes six to ten o’clock at night three or four days of week so they get the opportunity to go to an academic high school.

Clearly, there is an ocean (no pun intended) between academic expectations in the different countries. Putting emphasis on an international test that American kids don’t even know what it is and are told makes no difference to their grades or status is not telling the whole story. There is also no causation between high NAEP and international test scores and actual marketplace performance. Great test scores don't necessarily equal great doctors, engineers, teachers, etc. Thus, this data needs to be viewed with a interested, but skeptical eye.