Chemists say SOLs don't measure up (Daily Press)
Chemists say SOLs don't measure up
A professional organization has found errors on state high school chemistry tests, prompting recommendations for changes.
Larry Sacks has no quarrel with testing high school students' knowledge of chemistry. Nor does he object to state standards outlining what students should know about the subject.
But the retired Christopher Newport University chemistry professor takes issue with Virginia's end-of-year chemistry tests, which he says contain too many errors. And he insists a chemist should help choose questions and develop standards. That, he said, would help reduce the number of errors.
Sacks, who lives in Newport News, said he has counted as many as 10 errors - and as few as two - on the state's 50-question, multiple choice Standards of Learning chemistry test. He and other chemistry professionals have tracked errors since the state began using the tests in 1998.
"In general," he said, "they've gotten better over the last few years."
The four Virginia chapters of the American Chemical Society addressed accuracy concerns last month in a list of recommendations to the state Board of Education.
"Our greatest concern is over the accuracy of the items," said Sacks, chairman of the group's Hampton Roads chapter.
The group's report and recommendations state: "controls over accuracy of content have failed in numerous instances," resulting in major errors. The report further states that the errors "cast doubt on the validity of the test."
The state has tested high school chemistry students since 1998. Science test results are among four subjects used to determine whether a school receives accreditation from the state. Additionally, the federal No Child Left Behind Act requires schools to phase in science tests to measure student achievement.
State education officials say they have an effective process in place to revise tests and catch errors.
"Highly qualified" high school teachers develop and review test questions, said Charles Pyle, a spokesman for the Virginia Department of Education.
Pyle said the state revised the chemistry standards in 2003 and will do so again in a few years. Between revisions, Pyle said, the state reviews tests when there is "a change of knowledge," such as when astronomers downgraded Pluto from its status as a planet.
But Sacks said the process doesn't address errors quickly enough.
"If there's an error, it should be corrected today," he said. "It should not wait."
Stephen Gagnon, who maintains a Jefferson Lab Web site with math and science practice questions from SOL tests since 2000, said he has encountered errors in test questions the state has released.
Gagnon, a science and education technician, said he "repaired" those questions before posting them on the Web site.
Pyle said the state's science standards have received high marks from organizations, such as the Thomas B. Fordham Institute. The Washington, D.C.-based nonprofit research organization gave the state an A in a December 2005 report on states' science standards. The report focused on the state's physics and life sciences tests, but it called the high school sciences curricula sophisticated, "even in chemistry, where so many states fall down."
"We are very proud of our standards and test instruments," Pyle said.
But Sacks said the standards do not appear to have been developed with the help of a professional chemist or a college-level chemistry professor.
"There was no one in the system that was someone with expertise to understand the problem," Sacks said.
Pyle noted the tests are aimed at high school, not college, students.
Sacks also voiced worries about the types of questions on the tests. He said he's found only one question on a chemistry test that required students to analyze complex information. Most of the test questions focus on "straightforward recall and problem solving."
Sacks, who presented the recommendations to the state Board of Education last month, said board members thanked him for the recommendations but did not comment about them. He has presented information about the chemistry guidelines and tests to the state board before.
"I've submitted five pages of corrections to the state" over the years, Sacks said.
Pyle said the state uses such submissions when it revisits and revises standards.