“Gathering and Presenting Evidence for Impact and Sustainability” was the theme of an excellent event I attended last week, with tips and lessons for innovators who seek to get the results of their labours embedded in practice. Hosted by UWIC’s JISC project TEL-WFD (Workforce Development) at the new Cardiff School of Management, the event was one in a series of practitioner-led ‘Assemblies’. These are informal clusters of project staff sharing experiences and maximize the benefits of their work. In opening this Assembly, Colleen Connor, UWIC’s Dean of Learning and Teaching, spoke enthusiastically of the value in pooling the learning across the projects on the JISC Institutional Innovation: Lifelong Learning and Workforce Development programme. I certainly got a huge amount from the day, both on the topic of maximizing impact, and also on the latest technology developments in workforce development in three participating institutions. The participating organisations included: UWIC, University of Bradford and the University of Wolverhampton. we also had the benefit of advice from the SSBR team at Oxford Brookes who are providing valuable synthesis and evaluation services for the programme.
Sessions offering general tools and tips to improve a) impact and b) sustainability were interspersed with stories from projects about the things that had helped them achieve both these goals. Below are points I noted from each of the sessions. The Assembly was shared with external participants via the Elluminate webconferencing system, and materials and recordings from the day will be made available at a later date.
Rachel Harris – Inspire Research
Rachel offered advice on Maximising impact, assembling and leveraging the evidence base, linked to her work with the JISC Curriculum Design and Delivery Programme. She highlighted some effective techniques used by projects to capture and present evidence, some of which are mentioned below. You can find more details of ongoing project work in the excellent Design Studio which has a list of all the projects..
* a wide range of data collection techniques have been used including students as researchers and learner blogs (eg Springboard TV);
* different forms of data analysis eg prospective analysis, videos, Google analytics;
* case studies – Cascade, Springboard-TV
* some projects have been able to focus on the bottom line to secure backing for further rollout of their work e.g.
- EbioLabs has led to reductions in breakages in the lab, since students are better prepared for practicals
- Cascade has reduced admin time through online registration and payment for continuing education (based on time and motion study)
During Rachel’s session we had a go at devising 10-word elevator pitches and discussed in groups our ideas on how to “convince the cynic”:
* different stakeholders nay need different messages
* MyStudyBar was mentioned as an example of clever marketing: getting your audience’s attention in the first place is vital
* value of “telling a story” (though note that video case studies can be accused of being unrepresentative of the real world!)
Rachel passed on some excellent advice on how busy people access research (some of it may seem obvious, but it’s still worth reminding ourselves I think):
- they have little time (no surprise there!)
- they are most likely to trust information received face to face or from their peers
- they are reluctant to pay for access eg to academic journals (including paying in terms of time to access articles freely available via their library)
- they access new research mostly via the internet
- Institutional processes are very complex (again no surprise there!)
Then came the slightly depressing bit! We were told that project-based evidence plays a small role in senior staff decision-making! There can sometimes be a tension between the aspirations of funders and those of the institution hosting a project. Given these harsh realities, you need some additional skills in order to ‘sell the benefits’ e.g.
- have a holistic understanding of the operating environment
- marketing and influencing skills
- really want to do it!
I was able to highlight the fact that JISC RSCs are able to, and do, assist projects in maximizing impact, for example through our links with diverse sectors.
Evaluation and Reflection I: Compelling reasons
Jeff Lewis of UWIC talked about the experience of introducing web-conferencing in the teaching of dental laboratory technicians, including a demonstration of a typical teaching scenario where we experienced learning through both online tutorial and practical activities with bits of wire, dentures and pliers. More fun that it sounds!

Lots of tools explored at UWIC
Jeff made a host of serious points to explain the success they had had with the introducting of web conferencing. Just some of the compelling reasons for its continuation were:
- A strong business case; there are few places students can do the courses, and they are scattered from York to Cornwall.
- Students are already in the workplace (dental labs) so UWIC doesn’t now need to invest in such laboratories on campus.
- They can offer more flexibility as students no longer tied to the restrictions of the academic year.
- Students loved the technology, in particular the ease with which they could ask questions more easily than when f2f (they particularly liked private chat, breakout group activities and the ‘virtual bar’ for informal contact)
- However students still appreciated more traditional support methods e.g. email, and UWIC have retained a measure of face to face attendance
- Lecturers could use/adapt existing lecture material rather than author new content
- Web conferencing saves employer time and resources
Less frequent travel means workbased students are not exhausted (saves carbon footprint too!)
- UWIC incorporated support for workplace mentors
- In terms of student performance, UWIC are confident that students are performing at least as well with webconferencing
Following the success of the project, there are further uses planned for the use of the technology, including a Masters programme; management meetings; there is interest from other Schools; and online overseas exam boards offer scope for considerable financial savings.
UWIC’s experience offers a valuable case study in an area (webconferencing) where there are not a great many useful examples. However following a tip off at the event, I came across a useful blog post from JISC CETIS covering web conferencing tools in use on the Transforming Teaching and Learning Using Technology programme, e.g. the Open University’s Atelier-D project:
Features and benefits analysis
Andrew Haldane (UWIC) offered a simple yet effective technique for project staff to rethink the way they present their wares to stakeholders, by shifting perspective. The main points I picked up were:
- when we’re immersed in a project, the features are self evident to us, but when making your case to others, as with a job application, you need to “reorganise your story”
- by all means start by brainstorming your project features (the easy bit!)
- then take an extra step: WHICH MEANS THAT….
- then identify all the benefits, and one or two key overarching benefits
- your features, rather than being the “lead story”, become the evidence to back up your claims. You are offering solutions rather than problems.
So for the UWIC web conferencing project, you might identify “availability of camera close ups’ as a feature WHICH MEANS THAT all students get a much better view than in a face to face session. So there you have your benefit, and you could do the same for all the other features of the project. Key benefits can then be extrapolated such as: providing a valuable service to the profession, or viability of otherwise unsustainable courses.
In discussion, staff with experience of retail and marketing offered additional tips to help ‘sell’ project benefits. Again and again, the point was stressed that it is essential to know the market, understand what your customer is looking for, and be prepared to explore their needs.
Evaluation and Reflection II: WELL Project
We heard from a Bradford-based project called WELL (Workforce Engagement in Lifelong Learning) which offered further tips, which though they sound obvious at the start, are often forgotten once projects get going:
- don’t leave thinking about impact to the end
- you need to be flexible as context changes
Interestingly Bradford had decided to use webconferencing software solely for the purpose of offering prerecorded lectures, since their students were not being able to attend at set time.
Peter Chatterton then spoke about an extension to WELL, known as a Benefits Realisation Project. This is an attempt to transfer the lessons learned from projects to wider problems in other institutions, making sure that project results are “useful, usable and used”. In Bradford’s case, they will distil their learning into a “maturity toolkit” for technology-enhanced learning in work based learning (WBL). The toolkit will be designed for use in conjunction with “CAMEL” groups (communities of institutions sharing experience).
The project are looking for institutions to take part in trials from April 2011, so if you are interested please contact the project, details are available on their website.
Evaluation and Reflection III – E portfolio projects
Alison Felce (University of Wolverhampton) offered some experiences from their ePPSME project, where they’ve looked using E-Portfolios to bring all aspects of a workbased learner’s journey together. Some of the techniques and tips given for enhancing impact and sustainability included:
- carrying out a performance needs analysis of employer needs across sectors and using this to identify common themes
- in the pilot, they offered small chunks of learning (5 credits)
- they made use of retreats to develop staff skills (in house to save money!)
David Lloyd (UWIC) offered some tips from his experience of leading the Eporfolio strand of UWIC’s JISC project. Success points I noted were:
- they improved their customer relationship management system to track work placement students
- they introduced a reflective log for work placements (initially on Blackboard but are now looking at PebblePad). These are short (30 day) non-paid placements, based on a project
- students achieved good log marks
- they found that writing in a reflective way is something that needs to be learned, but feedback is very motivating
- they provide prompt questions at different levels according to the stage students are at i.e. start up of log, maintenance, and drawing conclusions
- the log gives opportunity for linkage of practice to classroom theory
Capturing lessons learned
Andrew Haldane took the floor once again to describe another technique to assist in identifying project benefits, called SMILE (stands for “Significant Milestones Review”). Essentially you ask the following questions:
- what went well?
- why did it succeed?
- if starting again, what would u do differently?
- what did you learn that was specific to this project?
- what could others learn?
It can provide a helpful structure for team discussions, and then converted into a more tailored ‘story’ suitable for external audiences. It has the advantage of being applicable to a whole project, or to individual activities within a project. The SMILE technique, Andrew suggested, is useful when considering project sustainability after the end of funding.
JISC Guide to Sustaining and Embedding innovations
Peter Chatterton flagged up this very useful guide which draws on the experiences of previous JISC projects and programmes to ensure that maximum benefit is reaped from projects. He stressed how important it is to try and identify benefits that can be transferred easily and quickly. One of the programmes that major lessons can be drawn from is the Open Educational Resources programme and Peter observed that there could be great potential to explore OER in the context of Work Based Learning, with FE, HE and employers working together.
One final point on sustainability: are project staff the best people to write the project’s sustainability plan, or would it be better to bring in someone from outside, who can see things more from the point of view of the outside world?
To sum up, the Assembly reminded me of the huge value in sharing across organizations if we are to get best value for money from our investment. At a time when some commentators are keen to question the value of projects, it is good to see the difference they can make when they enable staff to put ideas and tools to the test in real-world settings. The tips on maximizing impact and sustainability are, however, something we need to really focus on more than ever before. So, I hope we can encourage more innovation projects to broadcast their messages effectively, so we don’t find outselves reinventing the wheel for years to come!