Scott McLeod: Data-driven Decision Making

16 Sep 2005

The No Child Left Behind Act has forced school districts across the U.S. to take a hard look at data about their students’ achievement. Lots of data. But some districts have gone beyond the requirements of NCLB and have embraced data and used the information to identify best practices and improve student achievement.

Dr. Scott McLeod is a professor in the Department of Educational Policy and Administration at the University of Minnesota and a proponent of data-driven decision making. As director of the University’s Center for Advanced Study of Technology Leadership in Education (CASTLE), Scott works with educators around the country, helping them understand how being data-driven doesn’t have to be as scary as it sounds. We sat down for a chat earlier this week about some of the ways district’s are using data, how to overcome barriers to utilizing data, and some examples of using data that go beyond student achievement. And we even take a question from a “caller.”

Scott has agreed to monitor the comments on this post to dialog with any listeners who would like to follow up on something they hear. So don’t hesitate to post another question or ask for clarification on something from our conversation.

Download: STP-ScottMcLeod-1 (15.9 MB, 34:43)

7 responses to Scott McLeod: Data-driven Decision Making

  1. Listening to this discussion lit up my morning! :O) Thanks to both of you for putting it together.

    I’m the Director of Informaiton Technology here in Superior, Wisconsin…just a few hours north of you folks. The process of data collection (a subpart of data-driven decision making) has consumed the better part of the past 3 months for me. I’m fortunate to work in a district that’s close to breaking through with teacher-created common assessments in math as well as DRA (developmental reading assessment), DIBELS (diagnostic language skills assessment, and 6+1 traits writing (writing assessments).

    From the technology perspective, I’m struggling to find the best way to collect and manage this information. We are having conversations right now with TetraData to warehouse the information and Scantron to eventually collect the informaiton. The crux of the issue for us is the difference between the “item bank” style of products (which are plentiful) and the “do it yourself” model of building, collecting, and digitizing the data. On the surface, all of the companies will say they can do anything. I’m a bit skeptical.

    Long term, it will take a product like TetraData to get what we want. Short term, we’re at the same point Oseo is at with using spreadsheets to gather the data. I’m the lucky one developing these spreadsheets…cozying up really close with excel lately. I haven’t made it to the pivot table part of the development…I know enough to know that it would probably help.

    Now that you know where I’m at…questions.

    How do you effectively scale this data collection process with excel? We are about 4850 in student population…a bit smaller than Osseo, but still big enough that this gets really complicated, really fast.

    How is the “data” staffed in Osseo?

    Once the data is collected, how are you working with it? Are you dumping it into a bigger tool in order to make it more friendly to slice, dice, and query?

    Finally, under the question of “asking won’t hurt”, is there any way that you could share the excel template used in Osseo? I realize that all of these things are specialized to the point where it won’t be usable in any other district…I’m just curious to see how the foundation of the template was built.

    Thanks again to both of you.

  2. Hi John, thanks for the kind words re: this podcast and for being a loyal listener to The Savvy Technologist!

    I’d encourage you to read the white paper I wrote for Microsoft, Technology Tools for Data-Driven Teachers, which is available at http://www.scottmcleod.net/dddm_resources. The white paper outlines the landscape of technologies available for summative and formative data collection / analysis and may be helpful to you as you consider your options in this area. You already have captured the essential choice, though, for the formative data collection issue – do you want to subscribe to a commercial “item bank” paradigm (a la Renaissance Learning, Scantron, etc.) or a collaborative, teacher-created assessment paradigm (a la Osseo (MN) data templates project). The former is more sophisticated, convenient, and costly and may (eventually) be able to integrate with your data warehouse. The latter is probably more powerful educationally but also more time-consuming (and your tools won’t be as sophisticated (but still will do nifty things for you)).

    You can see how the templates work by visiting the links in the white paper referenced above. They’re essentially empty pivot tables/charts waiting to be filled. Regarding staffing / hosting the template data, we’re emphasizing that the templates are meant to collect formative data during the school year. As such, they get reset each year for a new round of data collection – we are not making an attempt to connect them with larger-scale data (e.g., in a data warehouse). The templates reside on a teacher’s hard drive (individual template) or in a shared network folder (group template). All analysis tools (pivot charts with disaggregating drop-down buttons) are contained in the template. We are in the process of making common templates that can be used across schools. For example, Osseo’s new elementary reading curriculum has six built-in assessments during the year. We have created a template that can be used by individual teachers / grade-level teams in each elementary school.

    FYI, we submitted an E2T2 proposal to the MN Dept. of Ed. to create a set of generic templates that can be used by anyone nationwide (e.g., generic 3 things twice a month template; generic 4 things thrice a trimester template; generic 1 thing once a week template; etc.). We should hear about that proposal any day now…

    In addition to the examples from the white paper, I’m happy to send you other example templates. Just drop me an e-mail (mcleod [at] umn.edu). All my best.

  3. Tim, howdy! Quick question…do you write out your introductions for your podcasts ahead of time or do you just start talking?

  4. Hey Miguel,

    I usually wing it, but I wrote out the intro for this podcast because the intro was longer than my usual ones. When I wing it I probably do 3-4 takes and edit together one that sounds good.

  5. Chris Shamburg 27 Sep 2005 at 9:12 pm

    Enjoyed the interview very much. You put data driven decisions in perspective.
    I teach Ed Tech in NJ and consult on a several projects outside of my teaching. I’m working on a few projects now–LMSs in high school and examining ways districts are using technology to prep students for our state testing–and your talk gave me food for thought. I’ll read your white paper.

    I’ll keep up with you on your blogs.

    Savvy Tech..great work!

  6. Just wanted to tell you that I found your podcast and listened to the Dr. McLeod’s session on DDDM. I am teaching a class on DDDM and was wondering if I could have permission to post the file on my class website. I am new to podcasting so I am still figuring everything out. I want to direct my students directly to this specific file. Is there a direct URL that I can use or can I post the file itself?

    Thanks,

    Leigh Zeitz

  7. Hi Leigh, you can link to the post

    http://technosavvy.org/?p=303

    or the podcast itself

    http://support.hopkins.k12.mn.us/podcasts/STP-ScottMcLeod-1.mp3

    FYI, I got your e-mail too and will respond to that as well. Thanks for the interest (and kind words)!