Can We Influence Public Policy? Shifting Data Resources from Systems to People

In early May, TERC’s Using Data Director, Diana Nunnaley, was invited to attend an important national meeting that can have future influence on public awareness, policy, and pre-service and in-service teacher preparation related to data literacy for teachers.

Diana was selected because of the groundbreaking work TERC initiated over ten years ago, developing a process of collaborative inquiry that engages teachers in cycles of data analysis and root cause analysis to inform instructional changes. Using Data currently works in districts and schools nationwide, building teacher-led data teams and facilitating a proven process of data analysis, instructional improvement, and increased student achievement—all leading to successfully narrowing achievement gaps among student population groups.   

The meeting was coordinated by WestEd and Education Northwest, and supported by the Bill and Melinda Gates Foundation. It brought together 50 nationally recognized experts who have studied the meaningful use of education data to improve instruction. They represented several universities, education research organizations, professional development providers, and foundation leaders.

Diana shares a glimpse of the discussions that ensued at the meeting and the musings they spurred. She concludes with a call to action for all who are committed to excellent education for all children…

 

March Madness: From Basketball Courts to Classrooms

By Diana Nunnaley, Director, TERC’s Using Data

March Madness annually takes over the country, or at least the media and the minds of U.S. college basketball fans who give it their frenzied attention each spring. At the same time, another March Madness is going on that does not garner the same enthusiasm and  does not make national news in quite the same way. It’s the March Madness going on in schools across the country as teachers and administrators ready for spring, state-initiated student accountability assessments. These tests are considered by some to definitively provide feedback on how much students have learned this year, and correspondingly – how effective their teachers are. (That second-tier “madness” could fill volumes, and I chose to let the pundits continue to hash out that one.)

 

Using Data Tip #9: Disaggregating Data Makes the Invisible Visible

GUEST BLOGGER: Mary Anne Mather, Using Data Senior Facilitator & Social Media Liaison on Twitter & FaceBook

If you want to tap one of the most powerful uses of data, disaggregate! Disaggregation means looking at how specific subgroups perform. Typically, formal student achievement data come “aggregated,” reported for the population as a whole—the whole state, school, grade level, or class. Disaggregating can bring to light critical problems and issues that might otherwise remain invisible.

For example, one district’s state test data indicated that eighth-grade math scores steadily improved over three years. When the data team disaggregated those data, they discovered that boys’ scores improved, while girls’ scores actually declined. Another school noticed increased enrollment in their after-school science club. However, disaggregated data indicated that minority students, even those in more advanced classes, weren’t signing up. These are just some of the questions that disaggregated data can help answer:

• Is there an achievement gap among different demographic groups? Is it getting bigger or smaller?

• Are minority or female students enrolling in higher-level mathematics and science courses at the same rate as other students?

• Are poor or minority students over-represented in special education or under-represented in gifted and talented programs?

 

One Teacher Talks About Using Data to Drive Instruction in the Classroom

GUEST BLOGGER: Mary Anne Mather, Using Data Senior Facilitator & Social Media Liaison on Twitter & FaceBook

I very much enjoyed Part I of Jill Thompson’s blog series about “Using Data to Drive Instruction in the Classroom.” According to her bio, Jill is an elementary math and science facilitator.

I applaud her for sharing her insights and passions about this subject. As a former classroom teacher, and currently as a facilitator for TERC’s Using Data process, I find myself in step with her thinking. Regularly integrating formal and informal assessments into the instructional planning process is a must. It’s not adding more to the plate — it IS the plate…understanding the impact of the teaching process on student learning and using that information to plan the necessary next steps—not only what to teach, but how to engage kids in the learning.

These days there is so much negative emphasis on testing, and I understand the rub when I see test scores being used to punish teachers and categorize kids. But let’s be clear that using data and testing are not the same thing. Data comes in many shapes and forms, well beyond test results and grades (these are just one data point). Teachers have the opportunity to use data as a valuable resource to guide a teaching and learning approach that can ignite learning for all students. As Jill notes–it just takes time and know-how (and an understanding that it’s a non-negotiable).

I plan to follow Jill’s blog series on this topic, and I recommend it to you. Thank you, Jill, for sharing your experiences and helping those who might be uncertain about how to put their data to work as an instructional tool. Your ideas illuminate understanding of a process for using data that can profoundly impact student engagement and achievement.

 

Using Data Tip #6: Beginning a Causal Analysis Process: Why Asking “Why” Once is Not Enough

GUEST BLOGGER: Mary Anne Mather, Using Data Senior Facilitator & Social Media Liaison on Twitter & FaceBook

“Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning.” Albert Einstein

Once a school or grade-level data team has analyzed several data sources to pinpoint a student learning problem, they often feel ready to leap into action and solve it. To ensure that the solution pursued produces the hope-for results, it’s essential to engage in a collaborative process of causal analysis to identify the “root” cause of the problem.

There are many tools that support root cause analysis, one of them is referred to as Why-Why-Why—a question-asking technique used to explore cause and effect relationships. Why-Why-Why helps a group look beyond symptoms to underlying causes by taking the identified problem and asking why it exists at least three times—each time probing more deeply.

 

Even Nothing Can Be Something

Guest Blogger: Dr. William L. Heller, Using Data Program Director, Teaching Matters*

There are often revelatory moments in the data inquiry process, where your analysis will lead to great insight and discovery in a way that challenges your assumptions and changes the way you think about teaching and learning in your school. There are other times when the data shows exactly what you were expecting, confirming your predictions and giving you valuable evidence in making your case to others. Many times, however, the data doesn’t show anything at all.

This can be somewhat dispiriting to an enthusiastic data team, but it doesn’t need to be. Sometimes the data may show nothing, but that’s still valuable information that puts you ahead of where you were before you looked. We don’t complain when our dentist finds no cavities, when the mechanic finds nothing wrong with our car, or when a medical test comes back negative. Similarly, in data inquiry, even a finding of nothing can really be something, if you know how to interpret what it means.

 

Using Data Tip #5: Getting to the Bottom of “Why” Before Investing in Solutions — Make Inferences and Question Your Data’s Story

GUEST BLOGGER: Mary Anne Mather, Using Data Facilitator & Social Media Liaison on Twitter & FaceBook

“Make data observations. Then generate possible explanations that inform next-steps to finding the best teaching and learning solutions.”
(from: Love, Nancy et al. The Data Coach’s Guide to Improving Learning for All Students, 2008.)

Data analysis is more effective, and more on-target for getting student achievement results, if a team of stakeholders first observe and list as many details as possible about what the data reveal, followed by making inferences about these observations, and then asking “why is this happening?” “what else do we need to know to be sure?”.

Infer/Question is the fourth stage in a team-based, 4-phase dialogue process* that guides deep discussion toward deriving accurate meaning from performance data. (See more information about Step 1: Predict, Step 2: Go Visual, and Step 3: Make Observations.)

These action steps will help you and your data team share inferences about the story the data reveal—inferences that will inform important next-steps toward identifying a valid student learning problem and its true causes.

 

Taking Standardized Testing’s Higher Road

GUEST BLOGGER: Mary Anne Mather, Using Data Facilitator & Social Media Liaison on Twitter & FaceBook

Since the story broke, the media and bloggers have actively covered the details about a recent test-score-fixing fiasco—and the news continues (links provided below). There have been commentaries about who really suffers from a scam of this magnitude (students robbed of remedial opportunities) and who is to blame—their motivations…and their motivators (“…targets were implemented…in such a way that teachers and administrators believed that they had to choose between cheating to meet targets or failing to meet targets and losing their jobs.” Volume 3: Conclusions: Why cheating occurred and cover-up allegations, p. 4). I’m particularly taken with “the data-made-me do-it” explanation. There’s even talk of legal prosecution. This whole mess feels like a major attraction in a very tawdry sideshow of the school reform circus. And sadly, although the media is focused on one district at the moment, test score and data manipulation is not new news.

Yes, there’s explaining to do, and some one (or many) will need to be held accountable, but wouldn’t it be great if the lion’s share of the energy fueling our collective indignation, disbelief, and need for retribution could be channeled to establishing more positive, long-term improvements to a testing and assessment system that has surely gone awry. I’m not yet so tainted that I can’t believe we (saints and cheaters alike) all really want the same thing: exemplary schools, highly qualified teachers, and well-educated students who are life-long learners ready to succeed in their adult lives.

I weigh in with Diane Ravitch on this one,

 

Data Blind Spots—Data Use Continuum Tool Can Improve a School’s Vision

By Diana Nunnaley, Director, TERC’s Using Data

Depending on where you sit, and which frame of reference shapes your work, you either celebrate charter school efforts or think charters reflect a “right” wing or “left” wing  (take your pick) conspiracy to undermine the role of public education in the United States.

A blog post is too short a space to weigh into the considerable arguments both pro and con that can be made regarding the place for charter schools in America. To my thinking, charters are a natural consequence of Americans seeking a solution to a social problem. We may not agree on the substance of the problem or the direction of the solution, but in a society that values and applauds entrepreneurial efforts, charters are here to stay. That is, they have a place until we learn more about the experience (hopefully by examining the data) or, have a collective epiphany about the impact of poverty on kids’ success in learning and activate the collective will to change the way we fund and support local education.

Charter School Vision Equally Blurred

Based on my experience working in schools across the country, the reality is that teachers in charter schools bring the same passion and desire to help children learn as teachers in any other public or private setting. They face the same staggering challenges and then some. And they bring the same blind spots to the table when examining their student learning data.

 

Measure What You Treasure: Soccer Goals, Factory Lighting, and Monitoring Plans

GUEST BLOGGER: Dr. William L. Heller, Using Data Program Director, Teaching Matters*

There is an important lesson to be learned from a soccer game played between Barbados and Grenada at the 1994 Shell Caribbean Cup. In this tournament, tied games would go to sudden death overtime, and any subsequent goal scored would be a “Golden Goal” worth two points. Barbados needed to win by two to progress to the next round, and in fact they were ahead 2-0, when Grenada scored.

With just minutes left in the game, a quick-thinking Barbadian player scored on his own goal, tying the game in order to invoke sudden death and buy some time for his team. Grenada’s players then tried to score on their own goal, hoping to lose by one, but Barbados was able to successfully defend Grenada’s goal. The game went into overtime, and Barbados won 4-2.

You would think that it would have taken a great deal to get these players to go against years of training and experience to want to score on their own goals, but all it really took was a momentary change in their accountability system.

We’ve seen a similar effect this year in New York City middle schools.