Can We Gauge Technology's Impact on Learning Outcomes?

In a Campus Technology article, Ruth Reynard explores the questions of how instructional technology affects students’ mastery of learning outcomes and how the presence or absence of outcomes related to that technology affect the assessment of more traditional outcomes. Here are some excerpts:

The ongoing debate as to the effectiveness of technology use for student learning outcomes still seems to have no clear answers. Recently, some universities have decided to end their laptop programs for students because of the economic challenges facing those institutions. But there is no consistent response as to the effect on students. Some say it has been highly effective for students, and others say that it has not had any significant impact in how students learn.

What is interesting is that there is also no real agreement as to what should be measured or even whether it can be measured in order to quantify success in this regard. Institutions…that have adopted technology for instruction, often have little or no systematic methodology in place for instructional technology use or how its success can or should be measured. Rather, the technology use has typically relied upon individual teachers and faculty who have given up time to learn and use new technology and who are always underfunded and unable, as a result, to expand their use to other programs and other instructors for ongoing research.

What…can be done to truly assess benefits to learning in regards to technology use?

…much of the reason we are not discovering…any meaningful way of measuring whether technology truly improves the learning experience for students or helps them attain the learning outcomes more efficiently is that the knowledge of the technology itself is smattered and that little is consistently taught to those who are currently teaching. This is heightened with the move away from separate instructional or educational technology departments toward all-inclusive IT departments that, owing to budget constraints, house only one or two instructional staff. No self-respecting teaching "expert" will approach an IT help desk with a question like, "Could you please explain to me what a server is and what happens to my documents when I hit the ‘save’ button?"

Most of this, in my opinion, results from education itself being the commodity it now is and teachers being required, as a result, to be business-minded, currently marketable, and technology-savvy. The swing away from any or all of these by teachers, then, is more about defensive practice than retaining rigor.

And now, for the rest of the story...