Opinion: Who's Grading The Kids' Standardized Tests?

By John Dodge

CHICAGO (CBS) -- Every year, students across Indiana arrive in class to conduct what amounts to a performance evaluation of not only themselves, but their teachers and schools.

Ostensibly, they are taking the Indiana Statewide Testing for Educational Progress-Plus (ISTEP+) to determine if they meet the state's educational standards for their grade.

However, the stakes are much higher. In Indiana, these test results form the basis for a letter grade for their school and their district. Schools' reputations hang in the balance, along with funding.

Incredibly, the people grading these exams, which require more than simple multiple choice answers, are not testing experts, and may not even have a background in education.

Rather, they are $12 an hour temporary workers hired by testing corporations like CTB McGraw Hill, which operates Indiana's test, and Pearson, which oversees Illinois' Partnership for Assessment of Readiness for College and Careers (PARCC).

The state of Indiana has paid tens of millions of dollars to CTB McGraw Hill to administer the ISTEP, which is given to elementary and middle school students, along with high school sophomores. It covers basic subjects like math, language arts and science.

It is truly a performance evaluation, disguised as a test.

And don't take my word for it. It's actually the law.

According to Indiana statute, the ISTEP must be used "to assess the strengths and weaknesses of school performance" and "to compare the achievement of Indiana students to achievement of students on a national basis."

Starting in early June, test-graders will be hired for about six weeks to grade the ISTEP+ in Indianapolis. Scorers must have at least a bachelor's degree to be considered. CTB McGraw Hill contracted with Kelly Services to handle the hiring.

According to a CTB McGraw Hill spokesman, the scorers are trained to score specific items on the ISTEP. Graders can be assigned math, language arts or science.

After training is complete, scorers must pass a qualification test where the test reader is required to score a set of papers.

"The [Indiana Department Of Education] closely reviews the process by which we train, evaluate and qualify scorers for ISTEP+," Brian Belardi, Director of McGraw Hill Education, said in an email.

Many graders, he said, return each year.

Under terms of a four-year contract that expired this year, Indiana paid CTB McGraw Hill more than $95 million to design and implement the ISTEP.

Given the money spent, Belardi was asked why the company could not afford to have trained test scorers on staff full-time.

"We recognize the considerable investment that Indiana has made in ISTEP+," he said. "Quality is always our first objective."

"However, as scoring summative assessments is a seasonal activity that takes place in a relatively short window of time, training qualified scorers who are not full-time employees provides high quality that meets the state's budget for assessments."

A spokesman for the Indiana Department of Education declined comment.

Test graders acknowledge that the task is daunting.

A recent college graduate who graded a sixth-grade math exam for Pearson assumed it would be relatively objective.

It wasn't, according to an essay he wrote on Tumblr about his experience.

"Students are graded on a four-point scale on three dimensions: the actual answer they gave, the steps they took to get the answer, and the written explanation of why they took those steps. The first two dimensions were straightforward enough for the scorers, but the third one, the explanation, was a nightmare to score.

"The explanation score often ended up being a total judgment call, as there was often no clear, objective score. In a subject like math, where objectivity should be a priority, this is a problem. During my time at Pearson, the grading criteria for the explanation section changed three times."

Recently, HBO's "Last Week Tonight With John Oliver" took on the topic of student testing. The segment included interviews with test graders who said supervisors told them that they were seeing too many high scores and that the workers needed to be more critical in their grading methods.

A college education professor told me about one grade school child who filled an entire page with her answer on an ISTEP language arts question.

The student received the highest possible score for her answer.

The problem was that the answer made no sense. It was simply filled with random words and thoughts.

School administrators were left to assume that the test scorer, who had been likely grading the same problem for hours, simply saw the answer sheet filled with sentences and issued a passing grade.

The complexity, or at least the quality, of the questions is also an issue.

Dr. Phyllis M. Gilworth, an assistant superintendent for the School Town Of Munster, recently told a group of parents that she was part of a working group that was reviewing language arts ISTEP questions.

In particular, one question asked elementary school kids for the "main idea" for a short story.

She acknowledged that the group, all of whom had a least a master's degree in education, could not agree on a correct answer.

Students, not just those in Indiana, are spending hours--sometimes up to 12 hours or more--taking these exams. Teachers and administrators spend even more time preparing their students. Even parents are reminded to make sure their kids eat a healthy breakfast on test days.

Yet the students are being evaluated by temporary workers who are being asked to grade questions that even education experts can't answer, for a wage that is less than the average salary for a full-time worker at Wal-Mart.