10 Things You Always Wanted To Know About Data-Driven Decision Making
Everyone's talking about D3M. Use this guide to help prevent all that data from driving you nuts.
Way back in the 1980s, a new way of doing business evolved as corporations began collecting, combining, and crunching data from sources throughout the enterprise. Their goal was to improve the bottom line by discerning hidden patterns and thereby improving the decision making process. Two decades later, the No Child Left Behind (NCLB) law is pushing school districts to do the same—this time, of course, the goal is increased student achievement.
While data alone won't provide school leaders with the finesse, experience, and intuition needed to get kids on the right learning path, a growing number of administrators are convinced that the data-driven decision-making (D3M) process can fundamentally change education—from our understanding of what really works with kids to administrative processes and professional development.
To assist you in getting your head around the realities and ramifications of data-driven decision making (and, yes, to make the subject a bit livelier), we've compiled this list of 10 truths. Some of these things will entice you; others will surprise you; a couple are sure to momentarily paralyze you. But, at least, now you know.
1. If you're not using data to make decisions, you're flying blind.
Imagine you're piloting a plane at night, in the middle of a storm. Without data from your navigation instruments, you'd be a goner. "Running a school without a data warehouse is like being a pilot without an instrument rating," says Brian Osborne, supervisor of evaluation, assessment, and research at the Plainfield Public Schools in New Jersey. "This is why there's so much policy churn in schools. Administrators simply don't have enough information to make good decisions." The Plainfield schools began using a data warehouse and analysis solution from eScholar three years ago. Although Osborne is a recent arrival in the district, he says he quickly became a big fan of the technology's power to help school leaders find the right flight path toward student achievement.
2. This is all about a process, not a specific technology.
Traditionally, D3M solutions have centered around a data warehouse—a central repository that collects data from many different sources—in combination with high-end decision-support tools that run queries and reports on the data. But in fact, a wide variety of technologies and solutions can be used to support a D3M process. Harry Hayes, superintendent of the Bloomfield, New Mexico, school district, says the D3M process should always begin with these questions: "How's business? How do you know it? And what can you do to improve it?" Hayes is a big proponent of the management-review process, and uses Northwest Evaluation Association (NWEA) test data to help him analyze teaching strategies and make better-informed decisions about professional development. Gregory S. Decker, principal of Lead Mine Elementary School in Raleigh, North Carolina, analyzes school data using Microsoft Access database software. He also uses Pearson's SuccessMaker management system for student assessments and on-demand reports. Decker and his staff have spent the past four years mapping, benchmarking, and making predictions about where each child—and each teacher—should be at every stage, and he says test scores have soared as a result.
3. Get ready to feel threatened.
"The downside to data-driven decision making is that it makes people much more accountable; there are no corners in which to hide," says Robert Ewy, director of planning at Community Consolidated School District 15 in the suburbs of Chicago. Ewy runs MicroStrategy's decision-support tools on an IBM data warehouse. "Before we began using a data warehouse, we thought we were making decisions based on data," Ewy says, "but now the quality has dramatically improved." Of course, the idea behind D3M is improvement, not punishment—and it's not just about the quality of teaching, but about the quality of leadership. "Data-driven decision making succeeds if the educational administration can model the habit of mind involved in data inquiry," Osborne says. "It's a culture shift, a new learning paradigm." He says that rather than having the answers all the time, educators need to come up with the right questions. Still, asking the right questions and interpreting the data are not the same as coming up with solutions, Hayes warns. "Data are simply indicators, tools to be used," he says. "They are a means, not the ends in themselves." Proponents of D3M in schools say this is where the quality issue really comes in: Once there is evidence of problems, do administrators and teachers have the chops to address the problems effectively?
4. You will be spending more money, not less.
Irene Spero, vice president and project director of the D3M initiative at the Consortium for School Networking (CoSN), says administrators need to prepare for a costly long haul when implementing data-driven decision making. "The first year is all about setting goals in the community and district. Year two is about roll-out and implementation, and it's not until years three or four that you can really see the effects," she explains.That investment in time can translate to a hefty price tag. Regardless of whether you've bought an expensive solution or not, the process itself demands a serious commitment to training, data cleanup, and maintenance that is often overlooked. "Every three to six months there must be a review of the data validation processes and cleansing of the data to ensure quality," says Katie E. Lovett, chief information officer of Fulton County Schools, in Atlanta, Georgia. "End users must be trained on the tool, but more important is training on how to use data for analysis and understanding how the data is structured in the data warehouse. And users also must be trained in data-warehouse security."
5. Data-driven decision making does not save time.
Don't start planning a vacation with all the time you think you're going to save having D3M in your district, warns Timothy L. Schaap, data analyst for Township High School District 214 in Arlington Heights, Illinois. "Data-driven decision making refocuses your time," Schaap says, "but it is definitely not a timesaver." Schaap believes D3M is akin to doing action research—a process of simultaneously pursuing change and understanding, continuously refining methods and interpretation in light of what's been learned. Schaap notes that both administrators and teachers have to take ownership in the action-research process—and that means more work, not less.
6. Your data's cleanliness is next to Godliness.
You've heard the expression, "garbage in, garbage out." The transactional systems that supply data to your warehouse—including student information systems, instructional management systems, testing applications, and financial software—are used and updated daily by staff members throughout your district, making data-entry errors a regular fact of life. You may also discover stagnant "pools" of data that have been forgotten and aren't regularly updated. If you don&