High-stakes test madness

By Walt Gardner

The obsession with the College Scholastic Ability Test for 2018 in South Korea would be risible if it were not so tragic. To understand why, it’s necessary to take a closer look at how this standardized test is designed.

Despite its impressive sounding name, the CSAT’s ultimate goal is to rank students. Therefore, if its designers loaded up the test with items measuring the most important material effectively taught by teachers, scores would run the risk of clumping together.

To avoid that possibility, designers deliberately include items that have been shown in the past to produce score spread. These items disproportionately reflect the socioeconomic background of students. In other words, they largely measure what students bring to the classroom rather than what they learn in the classroom.

Compounding the confusion is widespread misunderstanding of the difference between an aptitude test and an achievement test. The former is designed to predict how well a test-taker is likely to perform in a future setting. In contrast, the latter is designed to measure the knowledge and skills that a test-taker possesses in a given subject. While scores on both types of tests may be related, they do not necessarily correlate.

The SAT is a case in point. The changes in the name itself are evidence of misunderstanding. In 1926, when the test was first conceived by the psychologist Carl Brigham as an instrument for promoting greater meritocracy, it was called the Scholastic Aptitude Test in the belief that it assessed innate ability. But by 1994, the College Board had second thoughts. It renamed its premier brand the Scholastic Assessment Test out of concern that the original designation was associated too often with eugenics. In 1997, the College Board altered the name to simply the SAT, which ironically stands for nothing.

Criticism of the test still falls into two broad categories: its coachability and its predictive value. The College Board has insisted that the test is not coachable. But Stanley H. Kaplan proved this was not the case by helping students in his Brooklyn, N.Y. neighborhood dramatically boost their scores. His secret was constant practice followed by immediate feedback. Parents in South Korea spend heavily to hire tutors who can help their children prepare for the CSAT. Whether their expenditures pay off is controversial. But demand continues unabated.

Equally troubling is the predictive value of the CSAT. Once again, the U.S. serves as a case in point. In 1984, Bates College decided to engage in a pioneering experiment. It made submission of SAT scores optional for students seeking admission. In the fall of 2004, the college announced that its 20-year study had found virtually no differences in the four-year academic performance and in the on-time graduation rates of 7,000 submitters and non-submitters of SAT results.

If universities in South Korea adopted a similar optional strategy, they would take a giant step toward reducing the intense anxiety surrounding the CSAT. Since Bates released its findings, a growing number of competitive colleges and universities have decided to forgo standardized tests entirely. To date, they have reported little differences in four-year student academic performance and on-time graduation rates. Although tradition dies hard in higher education in South Korea, a change would likely parallel the U.S. experience.

Walt Gardner writes the Reality Check blog for Education Week in the U.S.