Doing statewide assessments online
September 13th, 2005 | by Tim Wilson |I got an interesting letter from the Minnesota Department of Education last week. Here’s a snippet:
During the next legislative session, representatives from the Minnesota Department of Education will present a plan and cost estimates for improving the technology infrastructure in Minnesota schools. This plan will be based on the requirements for delivering all of Minnesota’s statewide assessments via computer, in an online environment. In order to help estimate the extent of the current hardware conditions, improvements and their potential costs, you are asked to participate in a survey of the computer hardware currently in use in your district.
The survey asks how many computers we’ve got (at or above a Pentium III 700 MHz or G3 333 MHz), how much it would cost for us to install 25-30 new computers, how many days it would take to administer a one-hour test to all students, how many days it would take to administer a two-hour test to all students, how many days for one-hour and two-hour tests for our ELL students, and finally, a number of questions about our network bandwidth.
My school district is relatively tech-rich compared to many other districts I’ve seen. Even so, it would take a pretty big chunk of time to get all of our students through the computer labs to complete the tests. This will be a significant burden for less well-equipped districts. I wonder whether students will be advantaged or disadvantaged in an online testing environment depending on their degree of experience with technology. The kids in our one-to-one computing project would probably love doing the tests on their laptops instead of bubbling in ovals by hand. A lack of keyboarding or mousing experience could be a serious disadvantage for students who attend poorly equipped, economically disadvantaged schools.
Tags: mde, nclb, online testing
10 Responses to “Doing statewide assessments online”
By John Pederson on Sep 13, 2005 | Reply
Interesting. I’ve been really struggling with the question of online vs. paper testing of students over here on the Wisconsin side of the border. (Not the big state tests over here yet…)
Here’s the question. By automating this process, who does it benefit?
1) Technology companies will make some money selling boxes.
2) Some very large testing company gets a huge contract.
(Just had to get the easy ones out of the way.)
3) The state will considerably reduce the time it takes to collect, process, and return data.
(This may have some impact on student learning, but it’s a stretch.)
4) Administrators will have more data for decision making.
5) If administrators use the data well, it will get to the teacher level to help improve instruction.
(Districts that know how to use data will translate this into student learning. In my experience, very few districts have this capacity.)
There are now 5 points on the board for adults…a questionable 1 or 2 for students, if things go well. The act of taking a test online vs. paper…it will benefit some and be a serious distraction for others. Let’s tip the score to 5 to 3, adult needs vs. student needs.
Cost? How many millions up front? What happens 5 years down the road? Let’s say the state replaces these labs. Now we’ve doubled the number of computers the tech staff needs to support. A quick glance at TCO research shows that initial purchase cost is only a small % of the 5 year cost of a computer. How much do you trust your state (or any state) government to support you down the road?
Questions. No answers. This stuff is complicated. Keep us informed at how things go down!!!
By Kelly Dumont on Sep 13, 2005 | Reply
Tim,
Our state legislature started investigating online testing a couple years ago. In fact they even passed a law requiring it at one point, once they saw how much they would have to invest to make it happen though they backed off in a hurry. Instead they funded some pilots and the biggest problem was exactly as you stated, the time. Even in our elementaries, which are pretty big (several 1,000+ students) there was no way one computer lab could fill that need. It has now been over 18 months since we heard much about online testing.
By Scott McLeod on Sep 13, 2005 | Reply
As students get more tech-savvy, they may actually benefit from a computerized testing scheme. There is research out of CSTEEP (Boston College) that shows that computer-using students can be disadvantaged by pencil-and-paper tests:
http://www.bc.edu/bc_org/rvp/pubaf/chronicle/v5/F13/csteep.html
Now of course that doesn’t address the question of how to run students through existing tech capacities or how to pay for new testing tech (and what gets done with the tech when it’s not being used for testing?).
By Tim Wilson on Sep 13, 2005 | Reply
Scott, this was a concern of our with respect to our 1-to-1 program. We wondered if our laptop-using students would suffer when they had to take a pencil-and-paper test for the state assessment. We don’t know whether they were disadvantaged or not, but I’m sure most of them would have rather taken the tests online.
I’m reminded of something I used to do when I was teaching physics. I was aware of the research that suggested that some students are willing to revise their writing more (and presumably improve their writing) when working with a word processor compared to writing by hand. So whenever I gave an exam (my physics exams were almost exclusively essays) I would reserve the computer lab so that students could elect to word process their responses instead of write them out. About half the class would use the computer and I’m confident that their responses were better as a result.
By Scott McLeod on Sep 13, 2005 | Reply
See also this recent Education Week article:
(free registration required)
By Wesley Fryer on Sep 15, 2005 | Reply
I have no doubt that this move is going the WRONG direction. This is part of a broader shift that has been documented here in Texas in educational technology spending:
http://www.bloglines.com/blog/wfryer?id=129
This is an issue of sustaining versus disruptive technologies. We should be investing in disruptive tech like one to one technology immersion, the use of collaborative tools like blogs, wikis and podcasts– and shy away from the same old same old technology uses like: “let’s do electronic testing.”
When are our leaders going to catch a clue that it is not the testing or even the curriculum that makes the biggest difference for student learning: it is the teacher in the classroom?! I think this edtech spending trend is very depressing, and will just drive more students and families out of the public schools and into private/home schooling environments.
By Tami Brass on Sep 20, 2005 | Reply
I have to agree with Wesley. We’re moving into our 3rd year of 1:1. Forces outside our building think they’re moving forward while transitioning old, bad practices onto new hardware. Our district has my tech team delivering computer-based testing 4 months out of a school year! Teachers are running out of time to teach. Are kids being served by this? Are kids engaged? Doubt it. Are we jumping through hoops created by people who have no vision for the possibilities created by “disruptive” technologies?
Personally, anything described as disruptive creates an affinity in my heart lately
By Miguel Guhlin on Sep 24, 2005 | Reply
While I agree with the sentiments posted here–pro-disruptive technology–schools just don’t seem to care and/or have the time. Looking on, some teachers will make the change, but my gosh, realize the courage it takes for them to swim against the stream, to go against the flow of high stakes testing. I never imagined writing workshops would go the way of the dodo, but they have. Just as writing workshops proved to be a better approach than explicit teaching of grammar, diagramming sentences, so could have technology enhanced lessons made a difference.
Unfortunately, high stakes testing have driven out writing workshops in teaching of writing…and if that’s been driven out, what hope do disruptive technologies? Now, we don’t teach writing, recognizing the need for students to have time and ownership of their writing. Now, we teach them to write well enough to pass a test. And, teachers, districts and the State are proud of that. Well, I’m not. I’d rather have a writer who wants to write than one that can pass a test. I have faith in the first type of writer than the second…my daughter writes because she loves to write, because she is addicted to writing a story, not because she’s was to be able to write ideal personal narratives.
Teachers must fight to be disruptive or to use those types of technologies in schools. It takes guts to say, “I’m going to do it this way” and then have district teacher specialists come to your class everyday to ask, “Don’t you want to do it this way?”
District Teacher/instructional specialists aren’t practicing innovative curriculum designers, as Ed-Tech folks developed. Over the last few years, those individuals have become “policers” and enforcers of lock-step scope and sequence. The problem is, while curriculum folks might once have embraced disruptive technologies, the high stakes testing craze makes it impossible. Hope lies, not in the district staffs but in classroom teachers that must revolt and take back their classrooms, their curriculum.
The reason we are where we’re at as educational technologists is that we revolutionized the way we did things in our classrooms. We applied new technologies to the work we did, and it made a difference. And, we were promoted to lead these efforts. That is the point where we should have said, “No, I choose to remain in the classroom. I won’t be bought out by increased salary.”
If we fundamentally want to transform teaching and learning, then teachers will have to do it. We must go straight to the campus leadership and make the appeal there. I’m tired of technology fads–and blogs, podcasts, wikis as tools to revolutionize teaching and learning are included in that–that claim they will change everything.
Systematic change will be accomplished by sharing, not pushing, disruptive technology at the classroom level with one teacher and doing so over an extended period of time. The change won’t happen right away, and the EdTech initiative may not survive the time.
http://www.mguhlin.net/blog/archives/2005/09/entry_513.htm
By Tim Lauer on Oct 16, 2005 | Reply
Hey Tim… I posted something on my weblog about online testing… Can’t figure out if you have trackback or some such technorati magic, so here is the link…
http://tim.lauer.name/archives/003482.html
By Tim Wilson on Oct 16, 2005 | Reply
Thanks Tim. Trackbacks should show up automatically if you link to one of the posts on this blog.