Group of people standing on a graph line that is pointing upwardIn early May, TERC’s Using Data Director, Diana Nunnaley, was invited to attend an important national meeting that can have future influence on public awareness, policy, and pre-service and in-service teacher preparation related to data literacy for teachers.

Diana was selected because of the groundbreaking work TERC initiated over ten years ago, developing a process of collaborative inquiry that engages teachers in cycles of data analysis and root cause analysis to inform instructional changes. Using Data currently works in districts and schools nationwide, building teacher-led data teams and facilitating a proven process of data analysis, instructional improvement, and increased student achievement—all leading to successfully narrowing achievement gaps among student population groups.   

The meeting was coordinated by WestEd and Education Northwest, and supported by the Bill and Melinda Gates Foundation. It brought together 50 nationally recognized experts who have studied the meaningful use of education data to improve instruction. They represented several universities, education research organizations, professional development providers, and foundation leaders.

Diana shares a glimpse of the discussions that ensued at the meeting and the musings they spurred. She concludes with a call to action for all who are committed to excellent education for all children…

*******************************************

Can We Influence Public Policy?
Shifting Data Resources from Systems to People

By Diana Nunnaley, Director, TERC’s Using Data

I am a strong proponent of the power of professional collaborative inquiry and knowledge sharing. It’s part of my lifelong work as an educator and professional development facilitator—something I passionately lobby for as part of a teacher’s day. Therefore, with pleasure I accepted a recent invitation to attend a meeting convened by WestEd and Education Northwest to engage in a national dialog about data. What an honor to meet in San Francisco and be among a group of nationally recognized peers who have studied the meaningful use of education data to improve instruction! This was not a group of “outsiders” who raise banners for standardized testing, rather a group dedicated to assessment intended to improve classroom instruction and student achievement. My element!

Teachers in Rowland, CA engage in collaborative inquiry, using data to inform changes in practice and curriculum.

The central focus for convening this meeting was to engage the group in pooling their experience in order to specifically define what it means to be a data literate teacher, building administrator, or district leader. What do data-literate educators know and are able to do?

Prior to the meeting, participants were asked to submit their own definition of data literacy as it applies to teachers’ use of data. Definitions ranged from one sentence to a full page of attributes and descriptors. Regardless of the length or details in the definitions, all pointed to the fact that data literacy is more than looking at numbers on a spreadsheet and rank-ordering schools or the teachers in them. And to develop a school-wide data literacy culture takes time and resources.

At the outset of our conversations, we agreed that the definition can vary depending on the purpose for using the data and that it is difficult to separate data literacy—knowing how to use the data you have, from assessment literacy—knowing how to effectively assess student learning.

We were able to get into a rich discussion through a hands-on activity in which we were asked to position two circles, one labeled “data literacy” and the other labeled “assessment literacy,” such that the arrangement reflected the relationship between the two. The group brought their experience and creativity to the task. After the large-group task, we formed three small discussion panels to collaboratively define the skills, knowledge, and practices that relate to data literacy vs. assessment literacy. One panel focused on teachers at the classroom level, another on the building administrator’s level and the third on the district leadership level. Then, we compared our ideas and discovered similarities and differences.

Then the large-group conversation shifted back to a the more global view of using data. We overwhelmingly agreed that aside from a set of discrete skills and knowledge, effective data use is primarily the ability to ask good questions about teaching and learning, to find and follow the evidence leading to the answers—and to do this as a team.  In other words, it requires what scientists call “Scientific Method” and we at TERC call “Collaborative Inquiry.” This ability to observe what the data say and ask specifics about the “whys”—even when those questions might be uncomfortable or difficult—is at the heart of data literacy, and ultimately what is called data-driven decision-making.

From this activity at the meeting, I found my own thinking crystallizing around the complex set of skills, knowledge, and beliefs it is necessary to develop in order to use data effectively—and by that I mean: to use multiple data points (not just tests) to pinpoint student learning challenges and inform improved school and curriculum organization, improved instruction, and ultimately, improved student achievement.

Ironically, TERC frequently faces a frustrating dilemma when trying to respond to requests from state, district, and school leaders who ask us to provide professional development that will make their teachers data literate. They want us to deliver a professional development offering that conveys this complex set of skills and behaviors to teachers in one afternoon release day or through a 30-minute self-paced online module. Their rationale for this abbreviated training is always attributed to budget and time constraints. Their intentions are sincere, but I fear they don’t fully understand the complexity of meaningful data analysis.

Here’s a further irony: Right now, as a nation, we are focused on education data. National leaders are capable of generating millions of dollars to support the building of powerful data systems under the banner of high stakes accountability. States and large districts, in turn, are spending additional millions paying for sophisticated assessment systems. Underlying these expenditures is a very sincere desire to improve student achievement, but the resources are allocated from an uninformed assumption that all it takes to use these sophisticated systems well is a simple introduction to their data report generators. Instead, what it really requires to move from just a lot of numbers to achievement is a shift from spending millions to generate data to assigning adequate resources for building data literacy. This investment in human capitol is actually what it takes to realize the potential of the data to change instruction and schools.

For years TERC has been working with schools to help teachers learn how to analyze their own student learning data, pinpoint student learning problems, validate the causes of these problems, and identify changes in curriculum and instruction that directly solve them. In an ideal situation, TERC’s Using Data facilitators engage teams of teachers and their leaders over an extended period of time, teaching them a structured process that helps them extract the type of information from their data that can have profound impact on instructional practice and curriculum. The professional development is hands-on and job-embedded. Teachers work collaboratively to effectively analyze multiple data sources and discuss the implications of their findings. These sources move well beyond test scores and get into analysis of student work samples, attitudes, and school cultures. The work is transformational, and the results are empowering. For many teachers, it is their first entry into systems-level thinking, as they use what they have learned from their data-informed conversations to implement instructional changes and deepen their own content knowledge.

Professional learning activities that enable teachers to achieve this level of success, through ongoing data analysis and reflection on practice, require time and resources. In fact, using data well requires an investment equal to the millions currently invested in data infrastructure. What will it take for policy makers and those who control the purse strings in districts and schools to realize that this type of deep learning does not result from one sit-and-get or make-it-and-take-it session? How can they begin to think about shifting resources from data to people?

U.S. Secretary of Education Arne Duncan has been one of the most visible champions for the use of data by schools, and he is not alone. There are many other public figures and public policy organizations speaking out. What I’m thinking about is: As an informed community of educators, can we influence their enthusiasm for finding resources to create data (testing) and house data (data systems) to assigning resources to support professional development, collaborative inquiry time, and processes that result in data-informed practice? I sat with a roomful of colleagues in San Francisco who are trying to do just that. Our respective organizations will be vocal on these issues. We hope you will join with us in raising public awareness that data literacy is more than numbers. And locally, as education leaders, begin to creatively allocate the time and resources needed to support your teachers in becoming data-informed practitioners.