System Failure: Data Overload
The term “data-driven” is en vogue, especially in education circles. While the idea of using data to inform decision-making is not novel, the volume of information available to us is at an all-time high. From a plethora of personalized learning programs to an increasing number of data collection systems, school leaders are rightfully emphasizing data collection from these sources. However, the allure of trying to collect all of the information available has left some school leaders in a trap: schools are inundated with loads of data - some relevant, most excess noise - and don’t know what to do with it.
I recently attended a panel on sports analytics, which very much parallelled the debate in education. Yes, there are key differences between the ed sector and the sports industry, but the main theme is one in the same. In baseball, teams will do anything to gain a competitive advantage and are investing significant resources into data “analytics”. But, since this is happening across all teams and everyone has access to generally the same data, what is really making one team better than another?
Collecting the right data and responding accordingly is the game-changer.
An interesting example from this sports analytics panel was from the 2017 World Series Champion Chicago Cubs. Joe Maddon, the Cubs manager, distilled his data to get the right data by employing 7 full-time data scientists. The lead data scientist refines all insights into a single-page report for Maddon before each game, highlighting the most important and relevant information, while allowing Maddon to avoid significant data noise to boost the Cubs’ chances in each game. Maddon is renowned for his reliance on data-driven insights, but understands the risk of distraction with “too much” - especially in the heat of a game.
Some schools are beginning to utilize a similar approach. In recognizing that analytics often fall outside a teacher’s core skill set and wanting them to focus on their classroom execution, data scientists are joining school staffs to take on the load and deliver digestible and easily actionable insights.
For example, a high-performing elementary school in Chicago has a Director of Data and Technology (DDT) whose main job is to do what Maddon’s data lead does: purify the data and present it into an actionable form to teachers. Essentially, the DDT aggregates district-provided, blended learning, and intervention program data, analyzing it for any correlations linked to student achievement and then provides the teacher with previously undiscovered weekly trends. This team member also creates a student-facing webapp so that they can understand their own data. Like Joe Maddon, there is no need for the teachers at this school to sift through the wealth of information to find the right data; they’ve got what they need delivered to them in an actionable package.
In the age of information, some schools become so focused on collecting ALL the data, teachers never have time to filter and react to the right data. If there is too much information, how can you figure out what to prioritize? There is data from state tests, interim assessments, ed tech programs, teachers’ own data, and so much more. Just because the data may be presented in a way that looks pretty doesn’t mean it’s useful. Even the prettiest data, when there’s too much of it, can actually hurt what you’re trying to accomplish.
There are so many other factors that come into play when schools are trying to sort through the mountains of data to find the right data that they can actually act on. Before you can pick the correct data, school leaders need to ask: What are the instructional priorities for this year? What data needs to be collected in order to assess those priorities at the end of the year? What are the most important indicators along the way that will help you see the story behind the numbers? What are your end of year goals, but more importantly, the meaningful benchmarks that will get you there? Just a name a few. Only then will you be able to prioritize analysis of what you feel is most important and screen out the noise.
Not every school is in a position to hire a DDT to do this work for them though. The example above is how one school is trying to figure out what the “right” data is and get it in the hands of the teachers who can use it, but a lot can be learned from their efforts that can be scaled to fit your school’s data-driven efforts.
How are you thinking about the overload of data coming across your desktop? What’s the “right” data that’s going to get you on the scoreboard and how can you separate those data signals from all of the excess noise? This approach was good enough to break a 108 year World Series title drought for the Cubs after all.