While preparing for an upcoming data workshop I began investigating the differences between data-driven and data-informed. Many would find the terminology synonymous or a play with semantics but in reality they are distinctively different. Data-informed decision-making (DIDM) is defined in wikipedia as the collection and analysis of data to guide decisions that improve success. It is pointed out that while data-driven decision-making is a common term; data-informed decision-making is more preferable since decisions should not be based exclusively on quantitative facts (http://en.wikipedia.org/wiki/Data-informed_decision-making). Many organizations like to refer to this as “Big Data” but use of such decision-making can often lead to short-sighted miscalculations which can be perpetually harmful to the overall mission of the school. Educators must be more inclined with improving learning than with solely
increasing test scores.
It is virtually impossible to have data-driven instruction without data from various points. It is further an impossibility to have data from multiple sources if it is narrowly focused on test scores. Data should include test scores, interim and formative assessments, student engagement, student academic behaviors, and student work. Teacher intuition and judgment are powerful influences in the student performance equation as well. Data tools are frequently referenced as ways for school systems to deliver data to teachers. These tools must provide a means for teachers to capture and analyze data that is generated in the classroom daily. Teachers are more than data consumers. They construct valuable information which is applicable “For” student learning.
The following are tips on effectively integrating data tools into classrooms from the Michael and Susan Bell
Foundation:
1. No one tool that solves all problems, so don’t try to find one or sell your teachers on that idea
2. Opt for tools that enable a mash-up of computer-generated data and teacher-generated data (because people
trust their intuition, they will be more likely to embrace information from systems that marry their own insights with those generated by tests and/or software-driven assessments)
3. Consider using a standard data model to enable your various tools to talk to one another
(http://www.msdf.org/blog/2012/06/the-five-building-blocks-of-data-informed-instruction/)
In a 2009 ASCD article Fredrick Hess provides some profound insight. According to Hess, schools have to generate the data they need in addition to the achievement data external stakeholders desire. Hess explains, “Despite quantum leaps in state assessment systems and continuing investment in longitudinal data systems, school and district leaders are a long way from having the data they require...there is a rarely acknowledged tension between collecting data with an eye toward external accountability (measurement of performance) and doing so for internal management (measurement for performance).
Hess recalls a parable of a drunken man searching under a streetlight for his keys. A pedestrian stops to help; after minutes of searching, she finally asks, "Are you sure you dropped your keys here?" The man looks up and gestures toward the other end of the street, saying, "No, I dropped them down there—but the light's better over here." The parable illustrates the caution which must be practiced when using readily accessible data on reading and math scores or high school graduation rates. These yield useful information; however, educators must make sure these
points do not becoming the streetlamps which “distract more than they illuminate.” (http://www.ascd.org/publications/educational-leadership/dec08/vol66/num04/The-New-Stupid.aspx)
In a 2011 blog, Steven Turner writes, “Data is the setting in our story of education. Within this setting, the characters
(students, teachers, parents, etc.) create the plot in their daily interactions. To call for an end to “data-driven decision-making” is sure to raise a few eyebrows. Data is the constant, the solid ground; data takes the guesswork out of what we do. However, focusing on the data to the exclusion of all other elements of our story does not advance the cause of effective education for our students.” Turner believes all data should be considered including student attendance, variables in the life of the teacher and/or students affecting outcomes, problems with the assessment, instructional methods, past performance of teacher and student. Data analysis cannot be quantitative figures alone. “Testing does not happen in controlled settings. Therefore we should not mistakenly treat results as dependent variables.”
High performing students are not always indicative of good instruction. Conversely low performers are not tantamount to poor instruction. Data-informed allows a glimpse at a larger landscape. Turner argues, “Before anxiety leads to poor decisions, all potential reasons for low achievement need to be addressed.” Data-informed decision-making is comprehensive by nature. Students are first and foremost humans and not simply data points. Turner explains, “Unlike the pigeon in a Skinner box, the tools to isolate specific behaviors and tie them to discrete consequences are not at our disposal. Until that becomes possible (and I'm not even sure it would be desirable) we must continue to rely on human judgment, reason, and ultimately decision-making.” In summation, data-informed is a collaborative investigation of the facts and decision about the best course of action to rectify identified deficits. Data-driven
decision-making is often viewed as a “top-down, one-size-fits-all approaches to education reform http://teachingunderground.blogspot.com/2011/03/why-data-informed-trumps-data-driven.html).” More important than data is the story it tells. This information must be transformed from the aggregate to specific students with an targeted weakness. Action is the answer to changing the self-fulfilling prophecy which data may be expressing.
Post by John Schilawski 3/31/14
increasing test scores.
It is virtually impossible to have data-driven instruction without data from various points. It is further an impossibility to have data from multiple sources if it is narrowly focused on test scores. Data should include test scores, interim and formative assessments, student engagement, student academic behaviors, and student work. Teacher intuition and judgment are powerful influences in the student performance equation as well. Data tools are frequently referenced as ways for school systems to deliver data to teachers. These tools must provide a means for teachers to capture and analyze data that is generated in the classroom daily. Teachers are more than data consumers. They construct valuable information which is applicable “For” student learning.
The following are tips on effectively integrating data tools into classrooms from the Michael and Susan Bell
Foundation:
1. No one tool that solves all problems, so don’t try to find one or sell your teachers on that idea
2. Opt for tools that enable a mash-up of computer-generated data and teacher-generated data (because people
trust their intuition, they will be more likely to embrace information from systems that marry their own insights with those generated by tests and/or software-driven assessments)
3. Consider using a standard data model to enable your various tools to talk to one another
(http://www.msdf.org/blog/2012/06/the-five-building-blocks-of-data-informed-instruction/)
In a 2009 ASCD article Fredrick Hess provides some profound insight. According to Hess, schools have to generate the data they need in addition to the achievement data external stakeholders desire. Hess explains, “Despite quantum leaps in state assessment systems and continuing investment in longitudinal data systems, school and district leaders are a long way from having the data they require...there is a rarely acknowledged tension between collecting data with an eye toward external accountability (measurement of performance) and doing so for internal management (measurement for performance).
Hess recalls a parable of a drunken man searching under a streetlight for his keys. A pedestrian stops to help; after minutes of searching, she finally asks, "Are you sure you dropped your keys here?" The man looks up and gestures toward the other end of the street, saying, "No, I dropped them down there—but the light's better over here." The parable illustrates the caution which must be practiced when using readily accessible data on reading and math scores or high school graduation rates. These yield useful information; however, educators must make sure these
points do not becoming the streetlamps which “distract more than they illuminate.” (http://www.ascd.org/publications/educational-leadership/dec08/vol66/num04/The-New-Stupid.aspx)
In a 2011 blog, Steven Turner writes, “Data is the setting in our story of education. Within this setting, the characters
(students, teachers, parents, etc.) create the plot in their daily interactions. To call for an end to “data-driven decision-making” is sure to raise a few eyebrows. Data is the constant, the solid ground; data takes the guesswork out of what we do. However, focusing on the data to the exclusion of all other elements of our story does not advance the cause of effective education for our students.” Turner believes all data should be considered including student attendance, variables in the life of the teacher and/or students affecting outcomes, problems with the assessment, instructional methods, past performance of teacher and student. Data analysis cannot be quantitative figures alone. “Testing does not happen in controlled settings. Therefore we should not mistakenly treat results as dependent variables.”
High performing students are not always indicative of good instruction. Conversely low performers are not tantamount to poor instruction. Data-informed allows a glimpse at a larger landscape. Turner argues, “Before anxiety leads to poor decisions, all potential reasons for low achievement need to be addressed.” Data-informed decision-making is comprehensive by nature. Students are first and foremost humans and not simply data points. Turner explains, “Unlike the pigeon in a Skinner box, the tools to isolate specific behaviors and tie them to discrete consequences are not at our disposal. Until that becomes possible (and I'm not even sure it would be desirable) we must continue to rely on human judgment, reason, and ultimately decision-making.” In summation, data-informed is a collaborative investigation of the facts and decision about the best course of action to rectify identified deficits. Data-driven
decision-making is often viewed as a “top-down, one-size-fits-all approaches to education reform http://teachingunderground.blogspot.com/2011/03/why-data-informed-trumps-data-driven.html).” More important than data is the story it tells. This information must be transformed from the aggregate to specific students with an targeted weakness. Action is the answer to changing the self-fulfilling prophecy which data may be expressing.
Post by John Schilawski 3/31/14