Cairo Association of Teachers - Newsletter

CAT Tracks for March 10, 2006 TESTING, TESTING, 1, 2...UH...4?

As we enter our own personal "testing season"...

If one were to be "CATty" - especially those of us who are held accountable to the ISAT test - one would wonder out loud how anyone could be surprised at the inability of the College Board (administrators of the SAT) to "get the math right". I mean, these folks can't even spell ISAT! (Okay...what do you expect at 2 a.m.?!)

And to continue my whimsical mode...I'm always looking for an opportunity to get a research grant...and FINALLY retire on major bucks!

There has been a lot of talk about the cultural bias in the standardized testing world. THAT ground has been plowed repeatedly and I probably couldn't get much there. BUT...what about "emotional bias"? There may be some fertile ground THERE for new reseach.

The reason for my speculation is the two reasons given by the testing company for their scoring errors:

• Because of "excessive moisture" that caused the answer sheets to expand before they were scanned at the company's large test-processing site in Austin, Texas. (Test-takers need to remember the old adages about "Never, ever, let them see you sweat!" or "Don't sweat the small stuff!"...although the negative impact of a bad SAT score may be anything but "small".)

• That its scanners did not pick up some lightly marked answers. (No room for "wishy-washy" test takers here! Be bold! Have confidence in yourself!)

Company's Errors on SAT Scores Raise New Qualms About Testing

By KAREN W. ARENSON
and DIANA B. HENRIQUES

The scoring errors disclosed this week on thousands of the College Board's SAT tests were made by a company that is one of the largest players in the exploding standardized testing business, handling millions of tests each year.

The mistakes, which the company, Pearson Educational Measurement, acknowledged yesterday, raised fresh questions about the reliability of the kinds of high-stakes tests that increasingly dominate education at all levels. Neither Pearson, which handles state testing across the country, nor the College Board detected the scoring problems until two students came forward with complaints.

"The story here is not that they made a mistake in the scanning and scoring but that they seem to have no fail-safe to alert them directly and immediately of a mistake," said Marilee Jones, dean of admissions at the Massachusetts Institute of Technology. "To depend on test-takers who challenge the scores to learn about system failure is not good."

These were not the first major scoring problems that Pearson has experienced. The company agreed in 2002 to settle a large lawsuit over errors in scoring 8,000 tests in Minnesota that prevented several hundred high school seniors from graduating. It also has made significant scoring errors in Washington and Virginia.

After those problems, company officials had assured clients that they had vastly improved their quality control. But the new problems on the October SAT turned out to be the most significant scoring errors that the College Board had experienced.

Pearson said yesterday that the SAT errors, which affected 4,000 students out of 495,000 who took the October test, arose partly because of excessive moisture that caused the answer sheets to expand before they were scanned at the company's large test-processing site in Austin, Tex.

Another factor, the company said, was that its scanners did not pick up some lightly marked answers.

The company said in a statement that it was taking steps to make sure that "this unfortunate situation will not happen again."

Chiara Coletti, the College Board's vice president for public affairs, said yesterday that the College Board has continuing confidence in Pearson.

"Pearson says they now understand the technical issues fully, and we know they can control for those issues now," she said. "We are confident of that because our operations people have been talking to their operations people steadily."

The College Board has said that most of the students affected had higher scores than were reported to colleges. The scores were off by as many as 400 points out of a possible 2,400 on the three-part exam covering mathematics, reading and writing, although most errors were smaller.

Pearson said yesterday that it had examined the scoring of all the subsequent SAT's, which were administered in November, December and January, and found no further problems.

But some critics were not reassured. Shawn Raider, the lawyer who represented the Minnesota families who successfully sued Pearson, questioned whether the company had made good on its promise to improve its procedures.

"They certainly said in the course of our lawsuit that they not only were going to, but already had, implemented new quality control measures," he said.

The Pearson testing unit, a subsidiary of Pearson PLC, the giant publishing company that also owns The Financial Times, was awarded the contract for scanning the SAT answer sheets in 2003, taking over some functions previously performed for the College Board by the Educational Testing Service. They began the work last year.

It was one of many contracts that have helped make Pearson a giant in a field that has grown enormously since President Bush signed the No Child Left Behind Law in 2002, spurring demand for state testing.

For 20 years, Pearson has worked on the Texas testing program that was the template for Mr. Bush's national testing initiative.

Nationally, the company scored more than 300 million pages of answers last year and about 40 million individual tests.

Even as the company explained what went wrong yesterday, new complaints emerged from students and educators who questioned how they could continue to have confidence in the nation's testing apparatus.

Joe Giglio, director of admission at St. Peter's College in Jersey City, said, "It seems that there is a need for some sort of outside auditing of their processes to insure the integrity of the testing from this point forward."

Philip Benoit, a spokesman for Franklin & Marshall College in Lancaster, Pa., said yesterday that at least one applicant whose SAT score was revised upward by more than 100 points, now qualified for the school's merit-based Marshall Scholarship of $12,500.

Beatrice Bradley, a senior at the Williams School in Connecticut, who discovered that her reported score on the writing section of the SAT exam should have been 700 instead of 690, said one of her friends had also had an Advanced Placement score increased last year after raising questions about it.

"You have to wonder how many things go unchecked," she said.

The SAT errors, which the College Board started to investigate only after two students questioned the scores they received in late December, were not unprecedented.

As testing expanded sharply in the last decade, many more errors have occurred and almost all of them have been detected by students, parents or school officials challenging the accuracy of scores. Pearson said yesterday that it did not learn of the SAT problems until early February.

Some testing industry executives acknowledged yesterday that the SAT errors will add to the pressures the industry is already facing.

"There's no question that the testing industry is challenged," said Stuart R. Kahl, president and chief executive of Measured Progress, a nonprofit testing publisher in Dover, N.H., that provides testing services to 24 states. "But with the growth in business, most companies are implementing systems to make this job doable, so I don't get a sense that there is likely to be an exponential growth in errors."

But Mr. Kahl said standardized tests at all educational levels were constantly being revised. "The SAT's have been undergoing a lot of changes," he said yesterday. "And when you're putting out new forms of tests every year, the challenges are tremendous."

Some testing critics like FairTest, a nonprofit organization that opposes most uses of standardized testing, also raised questions about the College Board's selection of Pearson to handle scoring given its history of problems. "Looks like we have a scoring recidivist to deal with," said Robert A. Schaeffer, public education director for FairTest.

For now, college officials, who were caught by surprise by the mistakes at the height of the admission season, said they were working to take the revised scores into account so that students were not disadvantaged by the errors, almost all of which lowered student scores. Although some of the mistakes cost students more than 300 points, the College Board said that 83 percent of the score errors were from 10 to 40 points.

A follow-up story - actually, make that a prequel - from CNN...

Admissions offices scramble after SAT scoring error

(AP) -- College admissions offices scrambled Wednesday to reconsider applications after learning they had received incorrect SAT scores for about 4,000 students who took the exam last October.

Some in the field criticized the College Board, owner of the exam, for failing to disclose the problem until so late in the admissions season.

College officials said they expected few admissions decisions would be changed, but they were taking a second look at applicants whose scores were reported incorrectly. Generally, SAT scores are only one factor schools consider, but they can be critical in admissions to particular programs or eligibility for merit-based scholarships.

The College Board told colleges about the error Tuesday and said affected students would get word by Thursday.

At the very least, the error was a major headache in admissions offices, who thought they were through the busiest part of the year. At the University of Massachusetts-Amherst, admissions director Kevin Kelly had finished printing and started mailing more than 12,000 decision letters when he opened his own mail to find a 13-page roster of students whose scores had been reported incorrectly.

"On March 7, to get this packet of information is a little startling," Kelly said. "Most of my immediate reaction is not printable."

The affected students appear to be clustered in the Northeast. At the University of Vermont, 107 applicants' scores were affected, though most by just a few points, dean of admissions Don Honeman said. By Wednesday afternoon, Honeman said his staff had already reconsidered them all. One student who had been denied was admitted, and three others were bumped to a higher scholarship level.

Colleges in other regions seemed less affected. The University of North Carolina-Chapel Hill had 71 affected applications, the University of California, Berkeley had 32 and the University of Georgia four. Earlham College in Indiana had five, but all had already been admitted.

The College Board said differences were less than 100 points, out of a total possible score of 2,400, for the vast majority of affected students. Error fuels critics of test

Several college officials credited the College Board for its thorough response -- three entire sittings of the exam were rescored -- but others said they should have been notified sooner, and some were sharply critical of how the news was conveyed.

"As the party who screwed up you have a responsibility to fess up to the problem and provide a clear explanation," Dennis Trotter, dean of admission at Franklin & Marshall College in Pennsylvania, wrote in a letter to a College Board official.

Critics of the SAT called the error the latest in a long line of shortcomings by the nonprofit College Board.

"The larger issue here is that the nation has put its trust in an unaccountable testing industry," said Robert Schaeffer of the group Fairtest, adding the error might never have come to light had a student not asked for a rescoring. "It's yet another one of those cases in which the testing industry's screw-up could significantly harm people's education and their lives."

College Board spokeswoman Jennifer Topiel said the organizations notified admissions officials "as soon as we possibly could." Affected students will get refunds, and a handful of incorrect scores that actually should have been lower will not be changed.

But several people, including Trotter and Brad MacGowan, guidance counselor at Newton North High School in suburban Boston, questioned the decision to let inaccurately high scores stand, wondering if those students would now unfairly displace others.

"Are these scores important, or aren't they?" MacGowan said.