Evaluating the Effectiveness of e-Learning
February 2nd, 2010 by Christine Davies
We generally assume that technology enhances education, and often have anecdotal evidence in support of this. But how much concrete evidence do we have? Given the costs associated with acquiring and using educational technologies, I think this is a question worth asking.
There are quite a few case studies that highlight the benefits of specific technologies in specific contexts – for example, in HE, ‘Exploring Tangible benefits of e-Learning’ (JISC Infonet, 2008), and in HE/FE, ‘Effective Practice in a Digital Age’ (JISC, 2009). However, there are very few studies that attempt to measure effectiveness – one of the few examples I’ve found is the ImpaCT2 project in schools (Becta, 2002) which compared time spent using technology (mainly PC use) with ‘value-added’ scores in a range of subjects. (Becta also discuss the impact of technology in the FE and Skills sector in England in their recent review publication, and note that there is a positive correlation between ‘e-mature’ colleges and Ofsted inspection results). Some literature on this subject can be found in a RSC Wales Moodle area, and I’d be very interested to learn about any other relevant studies that you may have come across.
Perhaps it is no surprise that there have been relatively few quantitative studies in the field of e-learning. For a start, it is difficult enough to study ‘discrete’ aspects of technology in this way – for example, a particular application or an online course – but it’s a real challenge to apply a numerical approach to the more realistic blend of online, mobile and classroom technologies that many learners now encounter. And what exactly should be measured? Cost-savings might be a priority for some, but indices of achievement or retention are usually preferred. However, it is almost impossible to separate the effects of e-learning from the many other factors that have an influence on learning. Learner surveys can generate quantitative data on learners’ preferences and experiences of technology, but interpretation depends on the proportion responding and their profiles. Similarly, teaching staff could be surveyed to look at technology in the context of delivery of learning.
This last approach is currently of particular interest to me, and I am keen to get feedback from educators about their experiences of specific educational technologies. To this end, an evaluation form has been devised which asks respondents to comment on and grade a technology of their choice. This is not a particularly scientific approach, but it may give an indication of relative usefulness from the point of view of teachers/trainers/tutors. If we receive sufficient responses we hope to compile a ‘TechAdvisor’ Moodle area – so please do visit the ‘Evidence of Efectiveness’ area on the RSC Wales Moodle and download and complete in the ‘Evaluating your Technology’ form in section 7 ( and return it to us!)




At the FOTE09 conference we heard a lot of common sense advice about our digital identities from Professor Shirley Williams from the University of Reading. She talked about many of the ways our digital identity can represent us.