CPS agrees to overhaul standardized test practice after OIG finds ‘unusual patterns’

Chicago Public Schools officials have agreed to overhaul testing procedures after the CPS inspector general pointed out “unusual patterns” and “irregularities” in some standardized test results, including unusually long test times and high numbers of pauses in the computerized exams.

In a news release issued Friday afternoon — traditionally when bad news gets dumped — CPS announced major changes to the rules and security around its highest-stakes test — the Northwest Evaluation Association, or NWEA test, used to evaluate teachers and principals, screen for selective enrollment admissions and determine each school’s rating.

Inspector General Nicholas Schuler’s investigation didn’t necessarily find provable cheating, but it found holes in the system — ones that allowed testing procedures to be “repeatedly violated” — that could be exploited for a student’s advantage.

Contrary to normal practice, schools officials were the ones to release the findings of the outgoing inspector general, whose last day on the job is next week and who typically publicizes his own reports. Asked about the atypical manner of the release, CPS spokeswoman Emily Bolton said, “The OIG is heading out soon, and we wanted to make sure this was out there for the public to see prior to his departure.”

Also unusual is how CPS agreed to accept all of the Office of the Inspector General’s eight recommendations to tighten the integrity and security around how the untimed computerized test is issued to second- through eighth-grade students. A testing security expert is to be hired, for example, and will keep track of who proctors each student’s test to ensure it isn’t a teacher whose evaluation depends on the results.

CPS board member Lucino Sotelo emphasized this is “not a cheating scandal” and is simply an opportunity to shore up the district’s testing protocols.

“The reality is that there was no correlation between the extended duration times and pause times to the growth that we’ve seen with the students’ performance,” Sotelo said.

Sotelo went on to call Schuler’s recommendations “very valuable” in placing the right “processes, controls and procedures to make sure that cheating does not occur.”

He also said it’s worth a deeper look at how the district uses the test for high stakes decisions. For the time being, CPS will continue to use the scores in that way.

“That’s definitely a fair question, for sure,” Sotelo said. “Is that the right thing to be doing with this kind of test? Was it designed to be doing that?

The findings come during Schuler’s last days as the school system’s watchdog. Mayor Lori Lightfoot asked Schuler to resign in the wake of accusations that he fostered a toxic workplace, allegations CPS hired a law firm to investigate.

In a statement, Schuler said Friday’s news release “took us by surprise, as CPS leadership had previously asked us to release our findings via a joint presentation with them at next Wednesday’s Board meeting.

“We also are puzzled as to why CPS did not redact the names of schools from our report. It was always our position and understanding that schools would not be identified publicly because no findings of impropriety were made about individual schools,” added Schuler, whose last day is Feb. 29.

“We urge the public not to jump to conclusions about individual schools based on our report.”

In the 68-page report handed to Chicago Board of Education members on Sept. 26, the OIG’s data analysis unit described “a concerning level of unusually long test durations, high pause counts and other irregularities” during the spring 2018 testing period.

Starting in second grade, CPS students typically take the NWEA Measures of Academic Progress or MAP test in the fall and the spring, and on average they took twice as long as the national average to complete the test, designed to last about an hour. The NWEA MAP test has no time limit, a feature some students told investigators they took advantage of by waiting out difficult questions until new questions would pop up.

In a small cluster of schools, students also had paused the test an abnormally high number of times, which could occur “for benign reasons” or could “be indicative of attempts to game the test to win higher scores or gains,” according to the report.

OIG investigators reported a correlation between the handful of schools that had the longest test-taking times and the highest amount of growth — or improvement made since previous tests — a correlation CPS denies.

At Dixon Elementary School, five of the students with the highest growth between 2017 and 2018 took a combined seven hours to take the test and had paused it between nine and 16 times.

However, Schuler added — and CPS emphasized — that his office’s analysis didn’t substantiate wrongdoing nor were the test’s results invalidated.