Data-Driven Crazy
I know that what I am about to say is akin to pedagogical blaspheme but hear me out. Being "data-driven" is starting to drive us nuts! There...I said it. First there was an emphasis on data-driven decision-making, which is a necessary component of effective schooling. As we began to delve into "data-driven instruction" and "data-driven testing," the slippery slope we were on should have become apparent. Last week, the phrase "data-driven results" stopped me in my tracks. I mean, what does that mean? A careful look reveals that the phrase is either redundant -because data is the result upon which we base our actions- or it is hypocritical because -if we recognize that a single data point is not synonymic to a result- we should approach formative assessment data much differently. At that moment I knew that "data-driven" had joined the long list of industry buzzwords that no longer holds much meaning and certainly gives no clear indication about how to fix our failing schools. So here are some alternatives.
WHAT ABOUT BUILDING "DATA-FRIENDLY" CULTURES...
A few years ago an administrator sat down to review the baseline assessment data from each of my two English language arts classes with me. The September screening revealed that only 8% of the students in one class was proficient and just thirty percent of my accelerated class was proficient. The administrator asked if I was concerned by these results. My response was that I was aware of the data but not concerned by it. I produced my own copy of the report complete with scribbles, hand-drawn graphs, and a detailed plan centered around seven groups of students. I knew just where my students were and where they were going. Not only did the administrator accept this response, she seemed pleased. Her goal was really to make sure that I was responsive to the data. She initiated an open-ended conversation and allowed me to describe my plan to meet our common goals. Because we had built a relationship of trust and communication, my initial reaction was one of optimism and ownership rather than fear.
Fear of the potential consequences of poor results is not an effective motivator for educators or students. An environment that places educators at odds with data activates the fight or flight response. The natural teacher response to this kind of environment is to be combative towards administrators, to be dismissive of data, and to be reluctant to teach tested subjects. We witness the consequences of misunderstanding and misusing data when schools succeed only in driving students to drop out while teachers and talented administrators opt to leave the profession.
OR "DATA-FORAGING" FOR INFORMED DECISION-MAKING...
Successful organizations know that every bit of information gives them an edge as long as they know how to use it. They rely on the ability of their employees to process the volume, variety, and velocity of data necessary to make informed decisions and adhere to current best practices. School and district administrators should provide training that empowers educators to routinely and easily dig-in when presented with data. Equipping teachers with basic data-intelligence skills like visualizing, interpolating, and extrapolating data allows them to ask smarter questions and get better results. This involves searching for patterns and trends that filter through the volume of data to clearly point the way forward. In this way, schools and organizations can begin to be more intentional in setting goals for teachers who are also more receptive to evaluation.
It is equally important to give thought to the variety of data that schools collect. While schools are beginning to embrace different formative and summative assessments of learning, some still struggle to use assessments for learning or as learning. But being data-driven should mean more than just measuring student achievement. It is impossible to be data-driven while choosing to ignore other metrics that matter. Educators have access to a wealth of available data about the factors that influence student achievement. Data from researchers shows the importance of parent and community engagement. Collecting data from teacher surveys reveals barriers to learning such as constant interruptions or maintenance issues. Even acheivement data should be examined as a measure of the efficiency of the programs schools purchase and to set criteria for the educational materials being used. The culture of accountability only works if every person who is a part of a system is accountable for their role within it. Think of education as a bed of nails. When the weight of educating our children is shared by all stakeholders from administrative assistants to custodians, each person can easily handle their fair share of pressure. Ultimately, being responsive to every piece of available data helps organizations fine tune the service to its customers.
AND THEN THERE'S "DATA-FRAMING"
In writing about the same trend in marketing, Marketoonist's Tom Fishburne cautions that
data-driven doesn't have to mean "data-blinded."
Just as schools should seek data variety in order to ensure that they pivot when there is a need, they should also put data in context. For example, let's reexamine the scenario mentioned earlier. Some would say that the results suggest failure, on the part of the teacher and/or students, but there is important information to consider. It should be noted that the 8% and 30% scores reported the proficiency level of students in the beginning of the school year. It is here that the differentiation between data as result or as an indicator is important. Viewing these scores as results can lead us to be reactive and premature in assessments of whether programs are working. This is an example of the descriptive analytics, those focused on what happened, that are most often used in education. Using data instead as an indicator to inform decisions gives school leaders the opportunity to examine data contextually, historically, and retrospectively.
Looking solely at a descriptive analysis fails to account for the specific set of needs represented by each group of students. The first class was made up of tenth grade students who had previously struggled in language arts. The second class consisted of academically advanced ninth graders. Although both groups were slated to take the state test in English language arts at the end of the year, given some context the disparity between the scores makes more sense. This is the beginning of diagnostic analysis designed to figure out why the data turned out the way it did.
It is also important to note that this assessment was administered at the beginning of the year. The established goal for language proficiency that year was 60 percent. In this context, both scores were actually on pace to meet or exceed the goal by the end of the year- which we did! Drawing on years of experience, I was not concerned about the immediate results because I had a frame of reference for what a successful school year looked like at every stage. The historical implications of data can not be ignored because the true value of data lies in comparisons between schools or districts and across years. Examining where students scored in previous years at the same time enables a predictive analysis of data to determine what is likely to happen. Overall, it is important not to be so blinded by the end result that we forget to celebrate little victories and set meaningful benchmarks along the way.
The benefits of giving teachers time to individually and collectively reflect on their teaching practices in response to data can not be overstated. Prescriptive analytics focuses on what the organization, as a whole, can do to affect the results. This should not be confused with proscriptive methods which focus on criticism, restrictions, and penalties. It is important for all stakeholders to maintain a growth mindset about data and equally important for those who are in the position to make decisions to practice smarter data analysis (see this week's featured product), so that data does not drive us all crazy.
