data analysis


We’ve written here multiple times about collaborative inquiry and how it is the most essential ingredient in using data to achieve real results. The two foundational books that came from the Using Data Project at TERC, Using Data Getting Results* and The Data Coach’s Guide, created a great number of pages, templates and tools describing step-by-step processes to guide instructional coaches, administrators, grade level team leaders and others. The guidance focused on how to begin developing the trust, confidence and skill of practitioners that can bring about collaborative inquiry in the service of learning to use data to guide instructional changes.

Our work began In the late 90’s just as data began it’s ascendency as flash points for improving educational outcomes. The underlying research was initially sparked by requests from schools, districts, and state department of education leaders, who themselves were overwhelmed and seeking help to understand how best to use data in schools. The growing demand for technical assistance led, Nancy Love, then at TERC, to wide-ranging research of a nascent body of literature and a very short list of other educators working in the field – Bob Garmston and Bruce Wellman, Ruth Johnson, Mike Schmoker, Victoria Bernhardt. But in fact, much of what was developed and disseminated through professional development came from schools themselves, where teams of early explorers worked with the Using Data Project team in partnership with WestEd,  to shape the ideas and build the tools.

One result, nearly twenty years later, is that you can’t pick up a book from the educators’ book list without encountering the essential ingredient for any change process to take hold which is collaborative inquiry. And it matters little what the main topic of the book is, to get the work done, it starts with teachers shifting their culture from one based on being individually “highly qualified”, to being part of a collaborative team where inquiry uses multiple perspectives and sets of skills to craft more rigorous lessons, differentiate instruction to assist all children to grow from their own starting point, use multiple forms of formative and summative assessments and other data to measure growth, provide timely feedback to students, and identify new areas of content and pedagogy needing professional development or coaching and peer feedback.

We know better than ever how powerful collaborative inquiry is in the hands of classroom teachers. We have hands-on professional development, online courses and tools, videos from the Teachers’ Channel and YouTube to assist schools in shifting their culture toward one of collaborative inquiry. We have rubrics to help us gauge our progress to achieving the goal and showing us what the end goal will look like. What we don’t have, perhaps, is a clear idea about what it looks like when schools don’t make this journey.

Based on our work over the past twenty years, we recognize what practice without collaborative inquiry looks like, and what it sounds like in schools with no culture of collaborative inquiry at the heart of how teachers go about the business of teaching our children.

Without collaborative inquiry, some if not all of the following attributes are in clear evidence. Same old, same old…

  • There is no weekly schedule that devotes common planning time to teacher teams who need to bring their student work or other assessment results to the table for investigation.
  • Administrators rarely sit in on team meetings to learn and to support teachers’ work.
  • Grade level and subject level team conversations sound like past years’ “teachers’ room talk” where students or their parents are routinely blamed for students’ poor behavior or performance.
  • Cursory examination of end-of-year or interim assessment results is focused on what’s wrong with the wording of the items.
  • From grade to grade, subject to subject, there is no common vocabulary related to assessments, content or pedagogy.
  • Interventions and specialists’ work with students is not aligned or in step with classroom instruction – no coordination of scaffolding to support students’ needs.
  • Students have no idea where their own learning is in relation to the learning standards. Grades are just marks on their work.
  • Instruction is focused on helping students learn “basic skills”, develop fact fluency, use “key words” to determine what algorithm to use.
  • Rigorous instruction is restricted to high achieving students until the rest master the previous bullet items.
  • Team talk rarely if ever digs into the concepts embodied in learning standards, to explore their own content knowledge or the pedagogy that could support different learners.
  • Instructional coaches or assistant principals meet individually with teachers to tell them about the data they are seeing and to suggest needed changes.
  • There is no coherence of content vertically (or horizontally) and little knowledge of what instruction in previous grades looked like relative to current instruction.
  • There is little evidence that current research about best practices or case studies are brought to the table to provide new insights.
  • Administrators select professional development for staff based on what other schools are doing near by.
  • There is no routine monitoring to determine the level of impact of newly implemented “fixes”.
  • No time is devoted to enabling teachers to regularly reflect both as individuals and collectively about their practice, about what excellent instruction would look like to help their students learn this, what would it take, how far are we from that.

We have spent years and years in schools recognizing the attributes above at the outset of the work, and we’ve experienced the satisfaction and real joy that comes when we see these characteristics disappear as teachers themselves begin to surface the important questions and go after the data to help them find the answers.

Collaborative Inquiry is happening! Read about the experience in Metro-Nashville Public Schools working with REL Appalachia to support teachers building their craft together through Collaborative Inquiry.

Resources

Johnson, M. (2018). Empowering educators to make data-informed decisions: A district’s journey of effective data use.   In E. G. Mense & M. Crain-Dorough (Eds.), Data leadership for K-12 schools in a time of accountability, 158-183. Hershey, PA: IGI Global, Inc. MNPS Collaborative Inquiry Toolkit website at www.mnpscollaboration.org/about.html

 

*Using Data Getting Results was published by Christopher Gordon Publishers which no longer exists and the book is no longer in print.

*The Data Coach’s Guide is still published and available at Corwin Press a Sage Publishing Co.

In facilitating teams of teachers who are focused on using their data to figure out next steps for instruction (or school level teams focused on teaching and learning), Using Data facilitators introduce processes and protocols to support genuine inquiry.  There are the 5 phases of continuous improvement (or the 6 or the 8). And frequently schools implement cycles of improvement.  What they so frequently miss is one element that makes it work.  In music, it’s “all about the bass”.

In data analysis it’s all about discovery,  being open, being in exploration mode, which means leavimultiple pieces of large chart paper displaying data analysis that creates a hand-drawn data wallng assumptions at the door. The tension here is that as humans, we aren’t that comfortable with holding out in uncertainty.  We want to solve problems quickly. We want to feel confident that we know what we’re doing. And any suggestions to the contrary, render us incapable to doing anything but sticking to what is familiar instead of taking the risks that high performing schools have come to relish.

If we extend the notion of being open a little further, it isn’t too far a stretch to realize that  along with discovery and exploration goes one of the 7 Norms of Collaboration – screen-shot-2016-12-01-at-10-21-07-am“Presuming Positive Presuppositions”. In other words, assume that everyone at the table only wants what’s best for our students. And most importantly, when looking at our students’ results, presume that every student wants to learn and to be successful. If we can presume positive presuppositions about them while we stay in discovery mode to learn more about their strengths, their sometimes hidden or buried aspirations, we can figure out how to design instruction that overwhelms the effects of poverty, learning disabilities and language differences.

In other words, explorers don’t let students’ historical and demographic profiles bias their instruction. Instead they are continuously open to the possibilities that are within every student we teach. Teacher teams who have learned how to confront their low expectations for student learning use the data to surface the questions leading to the next great discovery rather than jumping to premature conclusions that typically result in same old, same old – cycles of reteaching, assigned interventions and test prep.

On another note, with this week’s announcement by President-Elect, Donald Trump that his nomination for the Secretary of Education position is Betsy DeVos, a strong advocate of education vouchers and charter schools in Michigan, perhaps we could slow down any rush to judgement and instead, benefit by using some of the same processes for using data effectively (be in discovery mode, triangulate the data, search for root causes, monitor progress toward goals)  before we draw conclusions about the implications of this appointment.

By Mary Anne Mather, Managing Editor
TERC’s Using Data For Meaningful Change Blog
TERC Using Data Senior Facilitator

Many districts are heading into spring state-level testing. It’s irrefutable that the opinions surrounding the pros and cons ofthree teachers collaboratively analyzing student work samples such assessments make for heated discussions in many circles. Not the least among the disputes is the time spent on what some call “teaching to the test.” The high stakes value placed on these tests can make even the best of us do things we don’t really embrace as best practice.

At TERC, we try to look at it from a different angle. What if our day-to-day work as professional-level instructors set the stage for students to perform better on the standardized tests because we intricately understood the ins and outs of what students do and do not know? Armed with that knowledge, we can plan classroom instruction that closes the gap between misconception and success. It’s most likely going to influence test scores, while addressing essential grade-level learning goals. That’s where looking at student work samples comes in! (more…)

Guest Blogger, Jennifer Ungermultiple pieces of large chart paper displaying data analysis that creates a hand-drawn data wall

When TERC’s Using Data facilitators work with schools and districts, we assist with establishing a continuous improvement culture that is grounded on collaborative inquiry. In the process, a lot of chart-paper-sized posters are created. There’s a sound and productive reason for this large-format paper trail!

Anyone who has engaged in data analysis with Using Data’s protocols and processes knows that we value public recording on chart paper because it gets everyone literally “on the same page” and talking together as they uncover learning challenges, own them, and identify strategic solutions. (more…)

Guest Blogger: Jennifer Ungermany colored 3-D question marks

I have worked with so many districts and schools where the leadership proudly points to their “data binders”—most recently I recall a three-inch D-ring binder. Not that binders filled with data aren’t helpful or good, but I caution that if they are not being used to guide instructional and programmatic decisions, well, then they can be a waste of precious time and money. More importantly, if they are not connected to a shared ownership of the questions a group of educators has about instruction and programs and similar concerns, then they can serve no meaningful purpose.

So how do we get from just having data to using data for meaningful change and improved results? (more…)

Introduction by Mary Anne Mather, Managing Editor
TERC’s Using Data for Meaningful Change BlogGroup of teachers analyzing and charting data using 4-pahse dialog
…with a link to Data Quality Campaign’s Flashlight blog on
How Educators Use Data: A Four Step Process
*
****************

Effective Use of Classroom Data: It’s a topic that weighs on the minds of many educators these days. It’s also the title of a workshop that TERC Using Data recently facilitated at MESPA (Massachusetts Elementary Principals’ Association). The educators who attended were seeking strategies and resources to bring back to their schools that would help them build a culture of data use that is continuous, meaningful, manageable, sensible, and effective. Who isn’t?

There is little doubt that, in the news, education-related data are routinely discussed, bandied about, and sometimes applied in ways that are not efficacious for supporting effective teaching and learning. TERC is dedicated to making data a sweet and welcomed word, not a dreaded mandate. That’s why we were so excited that Rebecca Shah (@rebecca_shah) from Data Quality Campaign was a surprise workshop attendee! Rebecca took one of the teacher-level data analysis processes shared during the workshop and used it to reflect on the session and its outcomes. Her thoughts and related resources are posted on the Flashlight, Data Quality Campaign’s blog: How Educators Use Data: A Four Step Process. Enjoy!

And if you’d like to learn more about Four-Phase Data Dialogue, visit our Data Tips (see Tips 2-5).

 

GUEST BLOGGER: Kevin Dwyer, Education Consultant, LearningDesignsheadshot of the author, Kevin Dwyer
Email: kevin@learningdesigns.net          
Twitter: @marketeducate

Education reform is an ongoing topic of public comment and debate in many states. Our guest blogger, Kevin Dwyer, a long-time education consultant and Connecticut resident, fills us in on the Connecticut news that he’s been following…

A fiery public debate about education reform in Connecticut has been ignited by first-term Governor Dannel Malloy. Twitter is alive with back and forth 140-character points and counterpoints (see #ctedreform and #wecantwaitct). Data is at the heart of arguments on both sides.

The driver for the debate is a fact debated by no one: Connecticut has the highest achievement gap in the country. The purported excellence of its suburban schools serve to highlight the gross inadequacies of Connecticut’s urban districts. Students in urban districts are simply not being given the same access to quality education as their suburban neighbors.

Compounding the problem is that Connecticut has failed three times to secure Race to the Top (RTTT) money. Billions of dollars have been awarded in three rounds of funding. Connecticut has yet to earn a dime. Lack of an adequate evaluation system to promote effective teaching practices has been a key shortcoming in their RTTT applications.

After the most recent RTTT application failure, the Governot drew a line in the sand. At the State of the State address in February, Malloy said, “Let’s be honest with ourselves, and let’s speak bluntly: many parts of our system of public education are broken.” In essence he said that schools in Connecticut must change—not just urban schools, ALL schools.  He added, in reference to the issue of equitable teacher quality, “In today’s (public education) system, basically the only thing you have to do is show up for four years. Do that, and tenure is yours.”

Opponents and proponents of the Governor’s comprehensive school reform plan have readily lined up on either side of the issues of teacher quality, funding for charter schools, and definitions of education reform. Interestingly, both camps reference common data sources and are able to make data interpretations that selectively support their opposing views.

The Connecticut Education Association (CEA) leaked a memo that outlined their strategy to delay the Governor’s reforms for at least another year. The CEA has launched a television ad campaign which says that the Governor doesn’t get reform right. They attest that the Governor ‘s plan “takes away district control and places it in the hands of the state education commissioner; allows principals to decide which teachers are certified and; and siphons tax dollars from neighborhood schools.”

On the other side of the argument, reform advocacy group ConnCAN  has taken dead aim at the union—telling them to “Come Clean” with their membership about the role of student achievement data and teacher evaluations. ConnCAN is one of a host of groups representing businesses (CT Business and Industry Association), Superintendents (CT Association of School Superintendents), School Boards, and principals who are supporting Governor Malloy’s vision for school reform.

In the end, the question of what’s right for students and educators is being lost. Everyone agrees that something needs to be done. But the message is being buried by claims and counter claims buttressed by the same data—often manipulated to support opposing viewpoints. It leaves the public increasingly polarized about education funding and teacher performance, and wondering whose data interpretations to trust. The REAL challenge remains: figuring out how to move beyond special interests to how to meaningfully and accurately use the data to move together towards educational excellence.

Next Page »