By Mary Anne Mather, Managing Editor
TERC’s Using Data for Meaningful Change Blog

Teachers meet across schools to share data use insights.

A team of Using Data Facilitators recently completed the training and technical assistance segment of a three-year professional development initiative in Duval County, Jacksonville, Florida. As part of an IES-funded, randomized control study looking at fourth and fifth grade mathematics achievement, 30 elementary schools set out on a journey to learn what it means to be truly data informed. Twenty-eight completed the long-term, job-embedded training that focused on enacting a process for understanding and analyzing data linked directly to student outcomes and classroom practice. The ultimate goal—to build a culture of collaborative data use that informs an ongoing pathway to improvement in any subject area.*

After two and a half years of work, these 28 schools came together on May 16, 2013 at the Schultz Professional Development Center in Jacksonville to showcase their results.

Big ideas that participants shared centered on student use of data, collaborative inquiry, and taking time for content-specific data analysis and drill down to strand and item data..

Many told stories about improved student scores, students engaged with their own data and learning needs, increased teacher collaboration time, and shared responsibility across grades for student outcomes. Some uncovered sensitive problems that they are still in the process of grappling with. And as a result of their journey together, they are comfortable sharing the challenges, as well as the successes, and seeking solutions from research and from one another.

Because the Using Data work is a process, some schools were further along than others, but all had something to share and something to learn. And all recognized that even though the training is over, the work is ongoing, iterative, and IMPORTANT.

Some expressed hopes that the Using Data training might be offered to other Duval schools because they recognize their own leg-up with meeting new district expectations tied to achievement goals. They also recognize that having learned the process and putting a Data Team in place is one thing, but continuing the collaborative inquiry and integrating the process into day-to-day classroom planning is the ultimate desired outcome. It’s not over just because the training is over.

Likewise, although the treatment portion of the study is complete, the Using Data team feels a desire to continue the conversation and stay connected with a cohort of schools that have demonstrated great insight and promise. With that in mind, the Using Data Project Director shared some parting thoughts in hopes of bolstering continued growth and professional interaction.

In truth, these parting thoughts are offered to all schools that have committed to long-term, job-embedded professional development and have done the hard-work required to uphold that commitment. Substitute your own context, tools, and processes for the specifics in the letter below and keep the work alive!  (We also invite you to take advantage of the resources mentioned in the letter.)

********************************

Dear Principals, Data Coaches and Data Teams,

Saying that your work last week was impressive doesn’t begin to capture the energy, the information, and the knowledge at work in the Schultz Center on May 16. Your Using Data Facilitators applaud the accomplishments you have all made during the past two and a half years of work. Regardless of where you are on implementing continuous inquiry into student learning, you all have the tools and processes that will continue to open doors to student learning. Based on the level of questioning and the exchanges of information and practical advice shared during the School Data Exhibits, many of you now have new ideas to help you deepen your work with data.

As a parting message, let me encourage all of you to continue creating the structures (teams and time to meet) that are the first layer of implementation needed to support collaborative inquiry. And as your teams continue to exam student results (FCAT, Interim Assessments, student work samples), keep in mind that we must be prepared to push beyond the experience and knowledge that we have, whether individually or collectively across teams, to continually seek new knowledge from research about what works, from content specialists (your mathematics and literacy coaches), and from best practice evidence.

To impact change, we must be prepared to push beyond our comfort zones as we use data to illuminate learning challenges. We need to commit to continually learn more about our craft. Don’t settle for “good” or good enough—keep reaching for “great”— even if that requires abandoning a long-held practice or belief.  Be willing to rock the boat! Use your Short Cycle Action Plans to try new practices. And take the time to collaboratively analyze the results to know what’s working!

Our formal time together is over, but we are still your team! Don’t hesitate to contact us for assistance in the coming weeks, months, and years. We have invested with you in your progress over the past two years, and we are invested in your continued success. We encourage you to sign up for the Using Data for Meaningful Change blog. Some of your own Duval stories are being shared, along with other thoughtful reflections about teaching and learning.

We leave you with these resources for new data tips and pointers to best practice that can continue to inform your good work.

Subscribe to our Using Data blog: http://usingdata.wordpress.com
Twitter: http://twitter.com/TercUsingData
FaceBook: http://on.fb.me/9vrDOk

Warmest regards,
Diana
Diana Nunnaley, Director
Using Data Projects

*************************************************

*The Using Data process emphasizes a structured process to consult multiple data sources (looking beyond just test data), the development of school-based data coaches and data teams, and taking time for collaborative inquiry leading to the identification of specific areas where students are not achieving. Once problems are uncovered, rather than jumping right to possible solutions, Data Teams engage in a causal analysis process. They delve into “What is causing our problem?” “How do we know?” They explore potential causes and consult additional evidence to verify their existence. Verified causes can range from scheduling and curriculum alignment issues to very specific student misconceptions to teachers needing content-focused professional development. Only then are solutions enacted and action plans developed. Ongoing data consultation drives close progress monitoring to highlight results tied back to action plans.

TERC’s Using Data facilitators have been working for the past two years with 30 elementary schools in Duval County Florida. This is possible through funding from a U.S Department of Education, Institute of Education Sciences (IES) grant to study the efficacy of the Using Data for Meaningful Change processes.

As our time together winds down, the Duval schools are sharing stories about transformation in practice, focus, and student achievement. In May, the schools will host a local conference to share their successes, collaborate to address ongoing challenges, and plan for how their data cultures will serve them as they tackle new district mandates and initiatives. We hope to share their individual stories with you – both as inspirations and mini-tutorials about how they have developed a data culture that doesn’t relegate students to a set of numbers. Instead, it pinpoints specific student needs, gets students invested in their own learning successes, and in some cases has impressively raised school ratings.

In the meantime, we share this glimpse of the outcomes that were already surfacing last spring. This article just appeared on the SEEN* website

Putting Data Back Into the ‘Palms’ of Duval County’s Teachers

For a related story about Duval’s San Mateo Elementary School see, The Journey to Create a Data Driven School Culture.

*SEEN—the Southeast Education Network’s mission is to reinvigorate the spirit of American education. SEEN Magazine and http://www.SEENmagazine.us, presents resources, ideas, and techniques to help educators become more effective while growing personally and professionally. SEEN is the only organization that addresses the unique needs of educators who live and work in the southeastern United States.

]]> http://usingdata.wordpress.com/2013/04/04/putting-data-back-into-the-palms-of-duval-countys-teachers/feed/ 0 diananunnaley LogicModelDuval http://usingdata.wordpress.com/2013/03/18/the-journey-to-create-a-data-driven-school-culture/ http://usingdata.wordpress.com/2013/03/18/the-journey-to-create-a-data-driven-school-culture/#comments Mon, 18 Mar 2013 23:31:49 +0000 diananunnaley http://usingdata.wordpress.com/?p=918 ]]>

Collaborative inquiry posters showing causal analysis based on San Mateo Elementary School data

The Data-Aware Principal: Reflection #1
Guest Blogger: Lindsay P. Sharp, Principal, San Mateo Elementary School, Duval County Public Schools, Jacksonville, FL

As a principal, it’s clear to me that I need to be data informed. My job depends on it—literally, since I am evaluated by my school’s achievement. More importantly, though, my heart depends on it—I am committed to seeing data not as just numbers, but connected to the success of the students and teachers in my school.

As the school’s leader, my thoughts turn to the best way to translate my own state of “data informed-ness” into meaningful action, and I have come to understand the key lies in putting my efforts into creating data leaders beyond the principal’s office. My Using Data colleagues are now in every classroom in my school! Accomplishing this level of a “using data school culture” depends on a process that involves professional development, support, and dedication over time. We work at it every day.This journey started when I selectively invited third, fourth, and fifth grade teachers who were naturally analytical and collaborative to form a Data Team. As a small team, we shared a common vision and passion to create a culture where data would drive instructional practice and result in student improved student achievement.

Our vision focused on a commitment to collaborative inquiry. We use a process called data driven dialogue to analyze data together, versus individual teachers looking at their own data in isolation. This process has helped us to uncover critical student learning problems—some of which we discovered were systemic.  For example, are the standards for mastering fractions only challenging at the fourth grade level, at which time they are specifically tested and we see deficiencies? Or is there a vertical trend in this unit of study?

What happens within our Data Team work is magical. I have become just another member of the team. As principal, I am not the facilitator, but instead, an equal contributor and participant in our growing school data culture. True ownership and connections have evolved to the point where team roles have become fluid. Our team meetings always include a recorder, but no single designated facilitator. Instead, we start each meeting with a specific goal/focus. Based on related teaching experience, passion for the topic, and content knowledge, different members take the lead as different times.

This initial vertical Data Team of third through fifth grade teachers meets monthly to analyze aggregate and disaggregate state, district, and school data. We look for patterns and trends and then drill down through strand and specific item data, including student work samples. This approach allows data analysis to inform us on a non-personal level, and the data is intended to inform our practice, never to find fault. As we analyze data together, our ultimate goal is to create a plan to address a student-learning problem that team members agree on because it is backed up by multiple data sources. We set a target goal and devise action steps to achieve it. Following each meeting, team members return to their grade levels to report our findings and discuss next steps. Together, the teachers own the problems AND the solutions.

Once this initial Data Team was well established, I wanted to roll out the same process in kindergarten through grade 2. I met with the lower grade levels during their common planning time to demonstrate the same protocols the upper grades use. I discovered that flexibility was key to success, since my primary teachers had less exposure to formal data analysis than the upper-level teachers. Historically, they rely instead on informal observations, rubrics, and limited formal assessment data. I found I had to differentiate and scaffold the process for each grade level—just like for students with different learning styles in a classroom.

It has been almost two years since my school’s Using Data initiative began. In addition to the collaborations I’ve already mentioned, we currently hold vertical content meetings (K-5) each month. This is another scheduled opportunity for teachers to talk about and analyze data, and collaboratively look for trends in student learning problems and their solutions across grade levels.

This dedication to process and devoting time to regular collaborative inquiry has led to a growing data-driven school improvement culture at San Mateo Elementary.

In future posts, I will guide you through my journey of going even farther with data. I’ll share our experience of extending this process to involve students with goal setting and then holding student-led Family Data Nights. I’ll share how the triangulation of data, grades, and student work has created a deeper dialogue with families and our support staff—leading to increased student growth as a result of more targeted RtI (Response to Intervention) strategies and safety nets. I’ll also expose our two-year process to create and protect true collaborative inquiry and planning time and the impact this journey has had on content-driven professional development.

San Mateo Elementary School was one of 30 treatment schools that participated in an IES grant to study the effectiveness of TERC’s Using Data model and protocols during the 2012-2014 school years.

Source: Rowland School District, CA

We don’t learn from experience. We learn from reflecting on experience.
John Dewey

States, districts, education reform pundits…they force our reaction to standardized test scores. But there is more to data than those somewhat controversial numbers currently being used in many places as the primary data point to inform changing policy and practice, and even to rate teachers. Unfortunately we don’t hear many talking about “reflection data.” Let’s take the time to collect data via reflective practice…and teach it to students. Then we’ll all be on sounder ground for tempering test scores with meaningful data that can potentially drive sustained changed.

In his blog entry, Mark Clements focuses on teaching reflection to students, but it’s a lesson to take to heart as professionals, too.

The Importance of Reflection in Education

Mary Anne Mather is also a Using Data Senior Facilitator & Social Media Liaison on Twitter & FaceBook

I was bolstered by this bit of news from Tennessee via Learning Forward about the efficacy of teacher teams that meet regularly to share data and strategies. The article is a sound-bite about the good news for student achievement in Wilson County that leaves me hungry for the details about how their meetings are structured, what data they look at, and how that data inform practice. From the published results, they seem to have discovered the perfect storm where collaboration, data, and strategies/solutions meet to make a difference. I, for one—as a facilitator of processes to help conjure similar storms, applaud them!

But the news item also reminded me that there’s more to this kind of success than simply meeting as a team and sharing “what works.” Surely, many schools and districts provide common meeting time for PLCs (professional learning communities). Often, the members select a worthy professional reading and then discuss it together. Excellent knowledge expansion!

In most schools, grade level teams have time to meet weekly. These meetings generally include earnest informal sharing of strategies and tips for teaching concepts in new, interesting, and lively ways. As a former teacher, I identify with that mindset of “what’s mine is yours if it helps.” It feels good to be supported!

And undoubtedly, various teams (grade levels, vertical teams, content areas, improvement and instructional teams) meet to look at student data to determine if scores are heading up or down.

However, it’s the confluence and inter-relatedness of these three activities, driven by a collaborative inquiry process for analyzing multiple data sources—from state assessment tests to the work students produce in classrooms—that stirs up results like the ones reported in Tennessee.

Having just returned from Jacksonville, Florida where Using Data is working with teacher teams in Duval County as part of a study funded by IES to investigate the efficacy of a collaborative data-use process to drive improvement, I was heartened when one participant shared an ah-ha moment. “Before we created our Data Team and started the Using Data process, we already had PLCs and grade-level meetings where we shared classroom strategies.” She went on to explain that now it’s different. The analysis of multiple data sources is centered on a formalized collaborative inquiry process. This process helps the team to pinpoint the exact nature of student learning problems, which, in turn, leads to strategic selection of readings for their PLCs to investigate the best-practice research for addressing the identified problem. And when grade level teams meet to share strategies, they target classroom practices linked to specific student needs defined in the data that can be applied across the grade and even across grade levels. A key element of their process is progress monitoring to assess impact and adjust, adjust, adjust until results are achieved.

Bravo to Wilson County, Tennessee. Bravo to Duval County, Florida. And, if you’re hungry for more information about the elements for creating a perfect storm in your district, have a look at, Using Data: It’s the process that leads to improvement (scroll down to “Articles by Using Data” and click on the first article). You might also find Using Data’s free data tips helpful. The series, taken together, outlines a continuous collaborative inquiry process for analyzing data and taking action toward results.

In early May, TERC’s Using Data Director, Diana Nunnaley, was invited to attend an important national meeting that can have future influence on public awareness, policy, and pre-service and in-service teacher preparation related to data literacy for teachers.

Diana was selected because of the groundbreaking work TERC initiated over ten years ago, developing a process of collaborative inquiry that engages teachers in cycles of data analysis and root cause analysis to inform instructional changes. Using Data currently works in districts and schools nationwide, building teacher-led data teams and facilitating a proven process of data analysis, instructional improvement, and increased student achievement—all leading to successfully narrowing achievement gaps among student population groups.   

The meeting was coordinated by WestEd and Education Northwest, and supported by the Bill and Melinda Gates Foundation. It brought together 50 nationally recognized experts who have studied the meaningful use of education data to improve instruction. They represented several universities, education research organizations, professional development providers, and foundation leaders.

Diana shares a glimpse of the discussions that ensued at the meeting and the musings they spurred. She concludes with a call to action for all who are committed to excellent education for all children…

*******************************************

Can We Influence Public Policy?
Shifting Data Resources from Systems to People

By Diana Nunnaley, Director, TERC’s Using Data

I am a strong proponent of the power of professional collaborative inquiry and knowledge sharing. It’s part of my lifelong work as an educator and professional development facilitator—something I passionately lobby for as part of a teacher’s day. Therefore, with pleasure I accepted a recent invitation to attend a meeting convened by WestEd and Education Northwest to engage in a national dialog about data. What an honor to meet in San Francisco and be among a group of nationally recognized peers who have studied the meaningful use of education data to improve instruction! This was not a group of “outsiders” who raise banners for standardized testing, rather a group dedicated to assessment intended to improve classroom instruction and student achievement. My element!

Teachers in Rowland, CA engage in collaborative inquiry, using data to inform changes in practice and curriculum.

The central focus for convening this meeting was to engage the group in pooling their experience in order to specifically define what it means to be a data literate teacher, building administrator, or district leader. What do data-literate educators know and are able to do?

Prior to the meeting, participants were asked to submit their own definition of data literacy as it applies to teachers’ use of data. Definitions ranged from one sentence to a full page of attributes and descriptors. Regardless of the length or details in the definitions, all pointed to the fact that data literacy is more than looking at numbers on a spreadsheet and rank-ordering schools or the teachers in them. And to develop a school-wide data literacy culture takes time and resources.

At the outset of our conversations, we agreed that the definition can vary depending on the purpose for using the data and that it is difficult to separate data literacy—knowing how to use the data you have, from assessment literacy—knowing how to effectively assess student learning.

We were able to get into a rich discussion through a hands-on activity in which we were asked to position two circles, one labeled “data literacy” and the other labeled “assessment literacy,” such that the arrangement reflected the relationship between the two. The group brought their experience and creativity to the task. After the large-group task, we formed three small discussion panels to collaboratively define the skills, knowledge, and practices that relate to data literacy vs. assessment literacy. One panel focused on teachers at the classroom level, another on the building administrator’s level and the third on the district leadership level. Then, we compared our ideas and discovered similarities and differences.

Then the large-group conversation shifted back to a the more global view of using data. We overwhelmingly agreed that aside from a set of discrete skills and knowledge, effective data use is primarily the ability to ask good questions about teaching and learning, to find and follow the evidence leading to the answers—and to do this as a team.  In other words, it requires what scientists call “Scientific Method” and we at TERC call “Collaborative Inquiry.” This ability to observe what the data say and ask specifics about the “whys”—even when those questions might be uncomfortable or difficult—is at the heart of data literacy, and ultimately what is called data-driven decision-making.

From this activity at the meeting, I found my own thinking crystallizing around the complex set of skills, knowledge, and beliefs it is necessary to develop in order to use data effectively—and by that I mean: to use multiple data points (not just tests) to pinpoint student learning challenges and inform improved school and curriculum organization, improved instruction, and ultimately, improved student achievement.

Ironically, TERC frequently faces a frustrating dilemma when trying to respond to requests from state, district, and school leaders who ask us to provide professional development that will make their teachers data literate. They want us to deliver a professional development offering that conveys this complex set of skills and behaviors to teachers in one afternoon release day or through a 30-minute self-paced online module. Their rationale for this abbreviated training is always attributed to budget and time constraints. Their intentions are sincere, but I fear they don’t fully understand the complexity of meaningful data analysis.

Here’s a further irony: Right now, as a nation, we are focused on education data. National leaders are capable of generating millions of dollars to support the building of powerful data systems under the banner of high stakes accountability. States and large districts, in turn, are spending additional millions paying for sophisticated assessment systems. Underlying these expenditures is a very sincere desire to improve student achievement, but the resources are allocated from an uninformed assumption that all it takes to use these sophisticated systems well is a simple introduction to their data report generators. Instead, what it really requires to move from just a lot of numbers to achievement is a shift from spending millions to generate data to assigning adequate resources for building data literacy. This investment in human capitol is actually what it takes to realize the potential of the data to change instruction and schools.

For years TERC has been working with schools to help teachers learn how to analyze their own student learning data, pinpoint student learning problems, validate the causes of these problems, and identify changes in curriculum and instruction that directly solve them. In an ideal situation, TERC’s Using Data facilitators engage teams of teachers and their leaders over an extended period of time, teaching them a structured process that helps them extract the type of information from their data that can have profound impact on instructional practice and curriculum. The professional development is hands-on and job-embedded. Teachers work collaboratively to effectively analyze multiple data sources and discuss the implications of their findings. These sources move well beyond test scores and get into analysis of student work samples, attitudes, and school cultures. The work is transformational, and the results are empowering. For many teachers, it is their first entry into systems-level thinking, as they use what they have learned from their data-informed conversations to implement instructional changes and deepen their own content knowledge.

Professional learning activities that enable teachers to achieve this level of success, through ongoing data analysis and reflection on practice, require time and resources. In fact, using data well requires an investment equal to the millions currently invested in data infrastructure. What will it take for policy makers and those who control the purse strings in districts and schools to realize that this type of deep learning does not result from one sit-and-get or make-it-and-take-it session? How can they begin to think about shifting resources from data to people?

U.S. Secretary of Education Arne Duncan has been one of the most visible champions for the use of data by schools, and he is not alone. There are many other public figures and public policy organizations speaking out. What I’m thinking about is: As an informed community of educators, can we influence their enthusiasm for finding resources to create data (testing) and house data (data systems) to assigning resources to support professional development, collaborative inquiry time, and processes that result in data-informed practice? I sat with a roomful of colleagues in San Francisco who are trying to do just that. Our respective organizations will be vocal on these issues. We hope you will join with us in raising public awareness that data literacy is more than numbers. And locally, as education leaders, begin to creatively allocate the time and resources needed to support your teachers in becoming data-informed practitioners.

There is a growing philosophy that every teacher is a literacy teacher, a view that is becoming increasingly important as states prepare for the Common Core State Standards, which place an emphasis on content literacy.

But what does “every teacher is a literacy teacher” actually mean? Will science teachers be expected to put away the Bunsen burners and take out the Balzac? Will social studies teachers be responsible for teaching contractions alongside the Constitution? If we misunderstand the idea, we may misapply it, and it may even lead to resentment among teachers who feel they are being asked to take on another’s responsibility.

Part of the confusion may stem from the tendency to refer to the English Language Arts (ELA) class as Literacy class. I’ve done it myself. After all, that is the class where students ultimately learn how to read and write. But as we continue to examine the demands of college and the workplace, we are discovering the need to expand our understanding of literacy as a set of essential skills that are critical for success in every subject area. Teaching literacy in isolation misses the point of why we need to be literate in the first place.

The idea of reading and writing across the curriculum isn’t new. We already know that having students write in their content classes, say math, strengthens their performance in ELA assessments. But the critical shift in rethinking the idea of literacy is that we want students to read and write in math because it also makes them achieve better results in math. If they can construct a viable argument and critique the reasoning of others, they will be doing the real work of mathematicians. If they can’t, it doesn’t matter how good they are at calculating or memorizing facts.

There is no question that students need to learn a complex range of skills in reading and writing. But they also need to learn how to learn through reading and writing. When students in Social Studies class conduct research on current events issues and write letters to their elected representatives to express their opinions, they are exercising critical literacy skills. And yet, we would not deny that these activities are appropriate for the social studies classroom.

To further illustrate this point, imagine a hypothetical scenario in which we have the specific goal of only preparing students to be scientists. The purpose of K-12 education, in this scenario, is to make our students ready for a college experience where they will only take science courses, so that they can graduate and become scientists. They would still need to know how to cite evidence from informational texts to support an argument. They would still need to know how to write explanatory texts to convey complex information. They would still need to know how to prepare and deliver oral presentations and communicate with other scientists. World progress in science depends on literate scientists.

So if we can agree that content-specific literacy skills are vital to the work of the scientist, the historian, and the mathematician, we must then ask who is best prepared to teach these skills. Who should teach students how to write a story proof to solve a math problem? Who should teach students how to dissect primary source documents to learn about a historical period? Who should teach students how to use experimental data to construct an argument about a scientific principle? Confining literacy skills to the ELA classroom makes about as much sense as allowing students to use wooden pencils only in wood shop.

As a Using Data facilitator, I once conducted a session for all of the Data Teams in one New York City school. Each Data team represented a different academic discipline. During this session, the teachers themselves answered the question about who has the responsibility to teach literacy. As we drilled down into the data for each subject area, every team except one independently discovered the exact same problem within its own content area—students were weak in the academic vocabulary of that discipline, which has a profound effect on content comprehension. And who better to teach the vocabulary of the discipline than the content experts, themselves? Interestingly, the only department to select a different student learning problem was ELA.

If students start at an early age to learn not only how to read and write, but also how to learn through reading and writing, their learning in all content areas will improve. Furthermore, they will be more prepared for college and the workplace; they will be more informed citizens and critical consumers; and they ultimately will be more prepared to become life-long learners. Is every teacher a literacy teacher? The writing’s on the wall!

Teaching Matters is a non-profit organization that partners with educators to ensure that all students can succeed in the digital age. They are an official TERC Using Data partner organization, conducting the Using Data for Meaningful Change institute for New York City schools.

Education reform is an ongoing topic of public comment and debate in many states. Our guest blogger, Kevin Dwyer, a long-time education consultant and Connecticut resident, fills us in on the Connecticut news that he’s been following…

A fiery public debate about education reform in Connecticut has been ignited by first-term Governor Dannel Malloy. Twitter is alive with back and forth 140-character points and counterpoints (see #ctedreform and #wecantwaitct). Data is at the heart of arguments on both sides.

The driver for the debate is a fact debated by no one: Connecticut has the highest achievement gap in the country. The purported excellence of its suburban schools serve to highlight the gross inadequacies of Connecticut’s urban districts. Students in urban districts are simply not being given the same access to quality education as their suburban neighbors.

Compounding the problem is that Connecticut has failed three times to secure Race to the Top (RTTT) money. Billions of dollars have been awarded in three rounds of funding. Connecticut has yet to earn a dime. Lack of an adequate evaluation system to promote effective teaching practices has been a key shortcoming in their RTTT applications.

After the most recent RTTT application failure, the Governot drew a line in the sand. At the State of the State address in February, Malloy said, “Let’s be honest with ourselves, and let’s speak bluntly: many parts of our system of public education are broken.” In essence he said that schools in Connecticut must change—not just urban schools, ALL schools.  He added, in reference to the issue of equitable teacher quality, “In today’s (public education) system, basically the only thing you have to do is show up for four years. Do that, and tenure is yours.”

Opponents and proponents of the Governor’s comprehensive school reform plan have readily lined up on either side of the issues of teacher quality, funding for charter schools, and definitions of education reform. Interestingly, both camps reference common data sources and are able to make data interpretations that selectively support their opposing views.

The Connecticut Education Association (CEA) leaked a memo that outlined their strategy to delay the Governor’s reforms for at least another year. The CEA has launched a television ad campaign which says that the Governor doesn’t get reform right. They attest that the Governor ‘s plan “takes away district control and places it in the hands of the state education commissioner; allows principals to decide which teachers are certified and; and siphons tax dollars from neighborhood schools.”

On the other side of the argument, reform advocacy group ConnCAN  has taken dead aim at the union—telling them to “Come Clean” with their membership about the role of student achievement data and teacher evaluations. ConnCAN is one of a host of groups representing businesses (CT Business and Industry Association), Superintendents (CT Association of School Superintendents), School Boards, and principals who are supporting Governor Malloy’s vision for school reform.

In the end, the question of what’s right for students and educators is being lost. Everyone agrees that something needs to be done. But the message is being buried by claims and counter claims buttressed by the same data—often manipulated to support opposing viewpoints. It leaves the public increasingly polarized about education funding and teacher performance, and wondering whose data interpretations to trust. The REAL challenge remains: figuring out how to move beyond special interests to how to meaningfully and accurately use the data to move together towards educational excellence.

March Madness annually takes over the country, or at least the media and the minds of U.S. college basketball fans who give it their frenzied attention each spring. At the same time, another March Madness is going on that does not garner the same enthusiasm and  does not make national news in quite the same way. It’s the March Madness going on in schools across the country as teachers and administrators ready for spring, state-initiated student accountability assessments. These tests are considered by some to definitively provide feedback on how much students have learned this year, and correspondingly – how effective their teachers are. (That second-tier “madness” could fill volumes, and I chose to let the pundits continue to hash out that one.)

There are multiple reasons to call this time during the school year “madness.” For one, the stakes are a lot higher compared with other types of student assessment cycles. Additionally, the outcomes actually drive a critical part of our national dialogue about how to help schools increase student learning and close persistent gaps between groups of students. This single spring data point, unfortunately, influences the public’s understanding (or misunderstanding) of schools and their work. And just as in the sport’s world, there is no shortage of armchair coaches, fueled by the spring test results, who feel qualified to tell schools what they should be doing to perform better.

Cultivating a Dream

Each basketball team in the National Collegiate Athletic Association begins each season with one dream: winning the national championship. Not all succeed. In fact, not all even get to try. Prior to March Madness, an intricate system takes into account regions of the county, local sports program guidelines, and game wins to build 16-team “brackets” that define four regional, rank-ordered lists of noteworthy teams eligible to compete for that dream.

In my education analogy, rather than selecting 16 noteworthy teams, I’ve identified 16 noteworthy factors to consider as a single “bracket” of must-have elements for making meaningful data-informed decisions. Students always rank at the top of the list. The rest can be ordered depending on your context and how successfully you have already integrated each factors into a winning school improvement scenario. They are all important. What areas immediately need more attention in your school or district?

1. STUDENTS – who are excited about learning & reaching their potential
2. Teachers – actively engaged in a culture of meaningful data-driven dialogue
3. Principals – who provide resources to support effective assessment practices
4. Data Coaches – with skills to facilitate teacher-level collaborative inquiry
5. Professional Development to help teachers and leaders engage collaboratively with data
6. The right data reports – disaggregated in multiple ways
7. Item level test data – including state and district test items
8. Formative assessments embedded into lessons
9. Quick turnaround of assessments into results
10. Open-ended questions – and the ability to analyze for understanding
11. Time for teachers to analyze data every week
12. Time to share findings, reflect, and act on implications of the findings
13. Time to revise curriculum
14. Time to create more rigorous lessons
15. District support and leadership
16. A shared vision for improvement and how to monitor success toward reaching the vision

Celebrating the Winners

In the NCAA March Madness, it comes down to one team who takes home the trophy. In my dream, there are no losers. In the world of teaching and learning, it takes all 16 factors to produce winners. The ones we want to see come out on top are all our kids and their teachers. They are the pre-chosen champions.

I’m happy to report that well-coached, school-level Data Teams can help create these winners—high-achieving schools where students reap the benefits. They increase achievement by careful and varied data analysis that reaches well beyond one spring test. They compare state tests results with district benchmark assessments, student work samples, informal classroom assessments, student observations, and more. They use the accumulated evidence to focus on students’ strengths and challenges in ways that produce more rigor, better curriculum alignment, vertical grade and cross-course alignments, deeper content and pedagogical knowledge, and bigger scores. And, scores aside, this work ultimately leads to more meaningful learning and more engaged students—the real measures of success.

Here’s to eliminating March Madness in education and supporting an effective culture of teacher-led improvement that helps every student achieve the dream.

If you want to tap one of the most powerful uses of data, disaggregate! Disaggregation means looking at how specific subgroups perform. Typically, formal student achievement data come “aggregated,” reported for the population as a whole—the whole state, school, grade level, or class. Disaggregating can bring to light critical problems and issues that might otherwise remain invisible.

For example, one district’s state test data indicated that eighth-grade math scores steadily improved over three years. When the data team disaggregated those data, they discovered that boys’ scores improved, while girls’ scores actually declined. Another school noticed increased enrollment in their after-school science club. However, disaggregated data indicated that minority students, even those in more advanced classes, weren’t signing up. These are just some of the questions that disaggregated data can help answer:

• Is there an achievement gap among different demographic groups? Is it getting bigger or smaller?

• Are minority or female students enrolling in higher-level mathematics and science courses at the same rate as other students?

• Are poor or minority students over-represented in special education or under-represented in gifted and talented programs?

• Are students at certain grade levels doing better in core subjects?

• Are students whose teachers participate in ongoing professional development

in their content areas doing better than students whose teachers do not?

• Are the school’s most recent curriculum and instruction adjustments improving the performance of students in the lowest quartile?

To answer these or other questions, carefully consider what disaggregated data is already available and what additional data is needed. Develop a data collection plan that includes a wide variety of data. Consider state and local performance assessments, samples of student work, enrollment in advanced courses, participation in special programs, participation in professional development, student and teacher survey results, and more. All these can and should be disaggregated.

Some tips to help you get started with disaggregating test data:

• Thoroughly understand your school’s demographics in order to select the most relevant variables for disaggregation. NOTE: Some schools benefit from disaggregating data within demographic groups, such as Hispanic students born in the continental U.S. compared to those who are foreign born.

• Request state and district test data reports that are disaggregated relevant to your student population.

• Explore technology tools that will help collect, analyze, and report disaggregated data more easily.

• Note relevant demographic data as you collect other information about student learning.

• Ask for support from district data experts or the companies that provide your data system.  Let them know the types of disaggregated reports that will best serve your needs.

• Get your hands dirty—dive into the data using the four-phase data-driven dialogue process described in our previous tips (http://usingdat.terc.edu/tips).

What Lawrence Lezotte and Barbara Jacoby noted in 1992 in their publication, Sustainable School Reform, still rings true today, “Disaggregation is a practical, hands-on process that allows a school’s faculty to answer the two critical questions: ‘Effective at what? Effective for whom?’ It is not a problem-solving but a problem-finding process.”

* Segments excerpted from Love, N. Using Date/Getting Results: A Practical Guide for School Improvement in Mathematics and Science. (2002). Christopher-Gordon Publishers, Inc., p.39-42.