Questioning Educational Technology Research

I do not know any of these authors personally.  I am making some very accusatory remarks regarding their research.  While I don’t wish the authors ill, I do want to call into question the claims they are making.  We are told to make decisions based on research, but we MUST critically examine the claims of researchers.

I was recently at a conference from for teacher educators.  One presenter/author presented on a review of edtech research in science education.  He noted that the most effective implementations of technology were always coupled with inquiry-based professional development.  I asked if any studies had tried to separate out the role of the PD and the tech to see if it was the tech or the PD that was causing the improvement. He was unaware of any such studies.  I find it fascinating that when the inquiry-based PD was absent the tech made no difference (and then researchers reported on emotional data such as if teachers “liked” the tech).  So if the tech only makes a difference when inquiry-based PD accompanies the tech integration, what reasonable person would think the tech is the important factor?  I think we have to accept the extreme bias of edtech researchers and we should call them on it.

Below are just a few papers that demonstrate how poor much of the edtech research is and why I find little reason to put much stock in their claims.  These are some studies I recently came across, but I assure you, there are many, many more examples.

Lei and Zhao (2008)

This study is typical in that they claim to be studying the effect of technology on student learning, but assessed learning by interviewing and giving surveys  to students and teachers.  Students LIKE technology.  Of course they believe the technology leads to better learning.  Self-report data is problematic.  Even more problematic when the researchers don’t recognize the limitations.  Research questions 1 and 3 are fine, but 2 (the one that is most interesting) cannot be answered from the data they collected.  From the abstract:

This study investigates students’ use of one-to-one laptops for various activities and the impact of one-to-one computing on student learning and school culture. Based on data collected from surveys and interviews of teachers, students, and parents in a Midwestern middle school over one academic year, this study answers the following major questions: 1) How did students use their laptops? 2) What impact did the one-to-one laptop program have on student learning and school culture? 3) What were the perceptions of and concerns over one-to-one computing?

Dunleavy, Dexter, & Heinecke (2007)

This study claims to study the value added by 1:1 computing but collects inappropriate data to support such an investigation.  They claim the study suggests “potentially transformative” value addition.  Great, potential, hardly conclusive. The study goes on to note how varied the implementation of computer use was and how important the professional development piece is.  Here’s a study that needs to be done:  Give the professional development to a group of teachers.  Give the same PD and tech to a second group.  Then give a group tech and no PD and a last group no PD and no tech.  This comparison will appropriately investigate whether technology has the impact we think it does.

Keengwe, Schnellert, & Mills (2011)

This one really gets me.  The researchers claim to examine student learning, but then admit their instrument collects PERCEPTIONS of learning.  Perception of learning and learning are not the same.  They use this crap data to claim 1:1’s increase student learning (no it increases students perception of learning – which again, of course they think they learned more, they like tech).  The authors are even so bold as to include the word “therefore” and go on to claim there is need for teachers to implement computing practices.  This is downright dishonest.  From their abstract:

The purpose of this study was to examine how 1:1 laptop initiative affected student learning at a selected rural Midwestern high school. A total of 105 high school students enrolled in 10th–12th grades during the 2008–2009 school year participated in the study. A survey instrument created by the Mitchell Institute was modified and used to collect data on student perceptions and faculty perceptions of the impact of 1:1 laptop computing on student learning and instructional integration of technology in education. Study findings suggest that integration of 1:1 laptop computing positively impacts student academic engagement and student learning. Therefore, there is need for teachers to implement appropriate computing practices to enhance student learning.

This entry was posted in Critical Examination of Technology and tagged , , , , . Bookmark the permalink.

One Response to Questioning Educational Technology Research

  1. Steve Bolton says:

    Kudos for your apt use of the word ‘crap’


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s