Let’s pay attention when students, teachers tell us what they think about their school

School climate surveys are one of the most promising tools we have to create school environments that help students succeed. When we understand where school climate needs improvement, we can address the fundamental conditions that create great schools and help all students succeed. Yet, therein lies the rub. As school climate surveys have gained traction as improvement tools, real challenges exist in using that data to change and improve practice in ways that don’t undermine their credibility.

Student and teacher reports about their school are sometimes seen as inaccurate or untrustworthy because they reflect the respondents’ subjective views. However, the perceptions of students and teachers of their school environment and experiences are valid data that are highly predictive, showing where students are most likely to have academic success. Surveys allow us to capture a consistent snapshot of the lived experiences of students and teachers across a school or district. They also illuminate the underlying elements of successful learning environments — such as safety, sense of belonging, sense of challenge — in ways that a test score or grade cannot.

The University of Chicago Consortium on School Research created the 5Essentials Survey in the 1990s, and Chicago Public Schools has used it (as the My Voice, My School survey) for decades. So Chicago is rich in data about school climate. This allows for a deep understanding of the impact of school policy and practices that is not available in other places. However, we have heard concerns from teachers, principals, and school officials who are unsure they can trust the data, because the survey has changed significantly since it was developed and is now part of CPS accountability metrics. So can we trust the data and is the survey still predictive of a school’s ability to improve? And how are schools using the data for improvement?

The answer to the first question is yes. Teacher and student voice, as measured by the survey, remains highly predictive of a school’s effectiveness and its ability to improve on an array of academic outcomes, and this is true in schools serving students from both higher- and lower-economic backgrounds. It remains predictive despite concerns that the survey ratings are part of CPS’ accountability system.

For our new study on the 5Essentials Survey, we interviewed CPS officials and school staff, who acknowledged the challenges of responding to surveys that are used for both improvement and accountability, since being honest about a school’s challenges can lead to a lower rating.

While we received varied responses, we did not find evidence of wide-spread attempts to game the survey. Rather, we found that the survey is still very predictive of both a school’s effectiveness and its capacity to improve.

To understand and support schools, listen first

Through interviews with district officials and school staff, we learned how difficult it can be to use the survey data to improve schools. We heard a clear desire for surveys to guide practices, but also that there is no cohesive vision about how to use the data nor adequate support for schools to do so. These challenges are not unique to surveys on school climate — they exist with student assessment data too. The challenges are reminiscent of CPS’s initial roll-out of the Freshman OnTrack indicator, which uses students’ performance in their freshman year to accurately predict their likelihood of graduation. The indicator was first used strictly for accountability, and there was no change in student outcomes.

But when CPS began to support high schools in using the data to improve, their on-track and graduation rates began to rise and have continued to do so.

Our study’s findings present an opportunity for the state and the district to do with 5Essentials data what it did with Freshman OnTrack: match the strong desire for change and improvement with strong support and tools for using the data.

To accomplish this goal, district and state leaders will need to consider, and address, how using the survey for both accountability and school improvement creates conflicting incentives.

And it’s critical to listen to teachers, principals, and school staff — to understand what they need, so the data can be used to support their students’ success.

Elaine Allensworth, PhD, is the Lewis-Sebring Director of the University of Chicago Consortium on School Research