Guest Blogger: Dr. William L. Heller, Using Data Program Director, Teaching Matters*

There are often revelatory moments in the data inquiry process, where your analysis will lead to great insight and discovery in a way that challenges your assumptions and changes the way you think about teaching and learning in your school. There are other times when the data shows exactly what you werePen pointing to detail of bar graph showing flat results expecting, confirming your predictions and giving you valuable evidence in making your case to others. Many times, however, the data doesn’t show anything at all.

This can be somewhat dispiriting to an enthusiastic data team, but it doesn’t need to be. Sometimes the data may show nothing, but that’s still valuable information that puts you ahead of where you were before you looked. We don’t complain when our dentist finds no cavities, when the mechanic finds nothing wrong with our car, or when a medical test comes back negative. Similarly, in data inquiry, even a finding of nothing can really be something, if you know how to interpret what it means.

Bar graph showing no achievement gap between males and femalesWhat does it mean when you find no achievement gap?
Counterintuitively, teams can be excited to find a significant difference in performance between different sub-groups within their schools. They’re not happy that a gap exists; they’re just glad that they’ve found it. But when you do the analysis and find no gap, you should be just as pleased, if not more so. Keep disaggregating the data using different criteria, but if you don’t find any achievement gaps in the end, enjoy the discovery that your school seems to be providing relatively equitable educational opportunities for all of your students.

What does it mean when testing data shows even performance across strands?
It’s a clear path to drill down into a content strand where students are clearly showing deficiency. But when strand data doesn’t provide easy answers, it’s time to start looking for pervasive problems that reach across strands. Could students be having trouble with multi-step problems in Algebra, Geometry, and Measurement alike? Are students having vocabulary issues, regardless of whether the question is asking for a literary response or critical evaluation? A deeper look within the individual strands can usually reveal more fundamental problems.

What does it mean when students leave answers blank?
While a telling distractor answer might give you insight into why students got the wrong answer, a significant number of blank answers can signal that students didn’t even understand the question. It also might indicate a problem with stamina, if students are leaving a lot of questions blank towards the end of the exam. A student guessing blindly has a one in four chance of making a lucky choice, but blank answers are often deafening in their silence.

What does it mean when surveys show ambivalence from respondents?
Surveys often measure participant attitudes using a Likert scale, a series of statements that the survey-taker can agree or disagree with to different degrees. Many researchers prefer to offer an even number of responses, so the respondent is forced to choose one side or the other. But if you neglect to include a middle option—the “No Opinion” choice—you are missing out on valuable data. The idea of a survey is not to force anyone to generate results that will be more interesting to you; the survey should be designed to collect the most accurate data possible, and that includes allowing respondents to express their ambivalence if that’s how they feel.

A data analysis that shows nothing is not the same thing as a lack of data or an inconclusive result. So the next time your hard-earned data analysis shows no results, go ahead and make a big deal out of nothing!

*Teaching Matters is a non-profit organization that partners with educators to ensure that all students can succeed in the digital age. They are an official TERC Using Data partner organization, conducting the Using Data for Meaningful Change institute for New York City schools.