Error found in school rankings
Comparison of similar campuses is pulled off Web for correction
By Dan Nguyen, Sacramento Bee, April 8, 2006
The discovery of a miscalculation has caused the state Department of Education to withdraw a ranking used to compare academic performance of schools of similar characteristics.
The state said Friday it had omitted a variable - the number of students whose ethnicity was unspecified - from its calculation of "similar schools" rankings.
The erroneous rankings, which were released in late March, were pulled from the state's Web site and are expected to be re-released in three weeks, said Pat McCabe, the state's director of policy and evaluation. These rankings have been released every year since 1999, when the state created the Academic Performance Index - the API - based on a school's test scores.
The similar-schools ranking compares the API of one California school against 100 other schools that most closely share its characteristics and challenges.
The comparability of schools is measured using 14 factors, such as parent education level, percentage of English learners, average class size and ethnicity.
The re-evaluation of the similar-schools rankings does not affect API scores.
Since its debut in 2000, the similar-schools ranking has sparked confusion and controversy. The state had to recalculate the first year results after it learned that schools had submitted incomplete data.
A parent in Southern California subsequently sued the state for not releasing details of its rankings formula.
On Friday, opponents of the rankings system welcomed the state's admission of an error.
Lynn Winters, assistant superintendent of the Long Beach Unified district, said her district for years had dismissed the rankings as irrelevant.
"Mathematically, it's uninterpretable," Winters said.
"When you have a similar schools ranking, it needs to make sense to people," she said.
But Winters said she took notice this year after principals had complained about inexplicable and dramatic drops in their school rankings.
Winters believes there are other errors in the state's formula that will necessitate an overhauling of the rankings system.
McCabe said the state's testing department will meet Monday with the Long Beach district to hear its complaints. But he said there are complaints about the rankings every year and they often stem from misunderstandings of the system.
"There's a lot of disgruntlement around the similar schools rank," he said. "A lot of it comes from people thinking that schools (on their lists of 100) ought to look like them."
McCabe said schools are grouped together because of the amount of challenges they face, even if the types of challenges are different.
This year's ranking gaffe comes in the same year that the state revised its formula and added six new factors, including the number of gifted and special education students.
San Juan Unified district officials believe the inclusion of gifted student numbers might have resulted in several of its schools taking a dive in their rankings this year.
Del Paso Manor Elementary saw its ranking plummet from a top rank of 10 to a rank of 2. This means its test scores were in the bottom 20 percent compared to the 100 schools in its group. Only about a dozen schools statewide fell as sharply.
Principal Phyllis Westrup said she was perplexed, because the school actually increased its Academic Performance Index from 862 - which is well above the state's goal - to 872. And she said the school's low-income students had made large increases in test scores.
"It's kind of hard to understand how your school could make such wonderful progress and yet drop in the similar schools rankings," she said.
Donna O'Neil, San Juan's director of accountability, said she had not seen the details of this year's calculation method, but has in the past encouraged principals to contact schools on their lists and compare notes.
She suspected, however, that Del Paso Manor's number of gifted students - nearly 40 percent of the school's population - put it in a more elite bracket of schools.
McCabe said he expected complaints about the new formula. In the past, he said, schools with small gifted programs complained about being compared against other schools with larger gifted programs.
"You get complaints and concerns from people on both sides. You'll always have winners or losers," McCabe said.
Long Beach's Winters said the fact that a school like Del Paso Manor could be ranked so dismally despite actually improving academically was proof of how unintuitive the ranking system is.
Winters said she supported the idea of comparing similar schools, but wanted the state to find a different system for doing so.
"We don't mind the (concept). But the system they chose is particularly complex and uninterpretable." she said.
