BILD – The future of e-assessment

by | Apr 4, 2011 | Articles

These are the findings of City and Guilds following the implementation of Learning Assistant to support work-based assessment of some of their key programmes.  

The e-portfolio plays a vital role in:

  • providing faster and more efficient learning programmes;
  • reducing bureaucracy;
  • simplifying the jobs of assessors enabling them to focus more on feedback and assessment rather than managing huge volumes of evidence;
  • increasing the retention of learners in a learning programme;
  • increasing the ownership of learners in  their assessment programme;
  • providing more transparency for funding organisations; and
  • increasing the transparency of Quality Assurance for work-based assessment.

Learners using e-portfolios to assemble competency based evidence complete their qualifications 40% faster than those with paper based portfolios. Assessors say they are better able to manage their workloads.  Our own research also reports that the use of Learning Technology significantly reduces time to competency by 28% for the most e-learning mature organisations; additionally a case study on work-based assessment demonstrates speed to competency achieved in half the time. (click here for podcast and here for case study article).

However computer driven assessments themselves are having difficulty getting beyond multiple choice questions.

These were the main points to come out of a meeting of the British Institute of Learning and Development at City and Guilds head office in London on the 31st March.

The City and Guilds has been around for a long time being established in 1878 by the City of London and various craft Guilds. Throughout that time they have both remained up to date and maintained quality standards of assessment.

Andrew Boyle, Head of Assessment Research, presented a personal view of e-assessment. There is much still to be done as paper driven systems still dominate assessment procedures; whether work based or not. Isabel Nesbit the outgoing CEO of Ofqual (Office of the Qualifications and Examinations Regulator) is quoted as saying of computer driven assessment; “Let’s get this off the too difficult pile”.

Simulation, mobile media, virtual reality, 2nd Life, augmented reality, social networking, collaborative tools all give us the opportunity to get beyond multiple choice questions in assessment.

City and Guilds themselves have been using computer driven tests since October 2002 since they launched Global On-line Assessment GOLA. Initially usage was approximately 200,000 tests per annum and this has risen to 1 million per annum.  The overall UK market for e-assessment is around £250 million per annum so this is clearly a serious business. One of the most frequently used on-line tests is the Driving Test Theory Test.

However e-assessment is not yet doing enough; Generation Y (Those entering work now) are used to sophisticated, web-based interactive displays and communication.  Again quoting Isabel Nesbit; “Computerised exams should replace pen and paper test for a generation used to digital learning.” The strain of writing for three hours is a novel and tiring experience for Generation Y; although previous generations might also agree.
However many practioners in the field are uncertain about the future of technology in assessment. City&Guilds are experimenting with revolving cameras on PCs to act as invigilators and automatic recognition of candidates to alleviate some of these concerns although this does not take us beyond MCQs. Currently only a few topics are examined by computer based exams.  Some of these totally replace paper based assessment and others are a combination; blended assessment.  Achieving a close link between work performance and exam performance remains elusive for many.
The picture is much more positive for work-based competency assessment where the e-portfolio is proving a valuable tool. Andrew Stone, the Assessment Practice Manager gave the audience a very clear and entertaining presentation of how Learning Advisor works.

Competency based qualifications have a structure broken down into Units with Performance Criteria and Knowledge Statements. The traditional approach for work-based assessment is that candidates assemble a paper portfolio of evidence; witness statements, project work, comments from customers or line managers, perhaps photographs of completed jobs. Each piece of paper will refer to the Unit of competence and the Performance Criteria or Knowledge Statement. Of course one piece of evidence may refer to more than one item of assessment. Learning Adviser brings all this together in one electronic package.

The e-portfolio is a collection of evidence in electronic form suitable tagged and commented.  Evidence can include video, images, audio, emails, witness statements, spreadsheets, presentations, other documents, http links including links to Wikki sites (which may themselves be examples of collaborative exercise and therefore used by several students). Potentially the output from simulation and virtual reality scenarios could be included.

Andrew demonstrated how this might be used in the hypothetical Level 3 qualification in Corporate Skills.  This includes a Unit with the title “Attending and contributing to conferences and seminars”. This included Performance Statements: turn up on time; nod and smile at other participants, remain awake, contribute to discussion etc.  Photographs and videos of participants were added to a mythical student’s portfolio to demonstrate each of these performances.  The evidence was added to the performance statement in the system. The evidence is viewed by an assessor who can pass or fail the evidence and add comments.  An external verifier is then able to access the same record and comment on the judgements made by the assessor.

An advantage of the e-portfolio in this format is that both students and assessors can very easily get an over view of progress as each Unit can be displayed on one page with Assessor comments. At a glance it is easy to see if that Unit has sufficient evidence collected.

Learners can download their e-portfolios making it easier for them to keep a record of their progress and eventual completion.  Students like this approach as it is easier for them to collect evidence with every mobile phone being a video and audio recording device; e-portfolios don’t get lost, are simple to store and easy to save a copy.
Mobile technology also simplifies the process enabling not only evidence gathering on the spot but immediate transmission to the tutor and assessor. This strengthens the links between tutor and a remote work-based learner, a point made by a pilot study into the use of mobile technologies(PDAs) to help work-based assessment in the Health and Care sector starting as early as 2006.  The Centre for Excellence in Teaching and Learning  led a project on  Assessment and Learning in Practice Settings  with the aim of simplifying the collection of evidence for Health and Care students when working in hospitals, clinics and care homes.  Students felt far more supported by their University Tutor with mobile technology than they did without.

The conclusion from the conference is that the e-portfolio is a success, both from a commercial and learning aspect, but that e-testing still needs more development work to get beyond the multiple choice question.

Compare your L&D strategy with the Towards Maturity Learning Health Check

Compare your L&D strategy

Review your L&D strategy to discover your strengths and opportunities for improvement with the Towards Maturity Learning Health Check.

Featured content

The Towards Maturity Journey: River Island

The Towards Maturity Journey: River Island

We always talk about the benefits of undergoing Towards Maturity’s Learning Health Check and using statistics to emphasise the effects of benchmarking, but what does its real impact look like on a business? We interviewed Nebel Crowhurst, Head of People Experience at River Island, to find out what role the Towards Maturity Health Check played in helping her develop her organisation into one that facilitates a high-performing learning culture.

Case Study: Best Learning Technologies Project 2018 (UK commercial Sector)

Case Study: Best Learning Technologies Project 2018 (UK commercial Sector)

In 2018, 93% of learning practitioners wanted to increase employee engagement with learning, but only 27% were successful. With less than a third meeting this goal, it is important to explore how an exemplar organisation turned their learning strategy on its head to achieve just that. Read more about how Mitchells & Butlers teamed up with Kineo to become Gold award winners at Learning Technologies Awards 2018.

Featured

The Towards Maturity Journey: River Island

The Towards Maturity Journey: River Island

We always talk about the benefits of undergoing Towards Maturity’s Learning Health Check and using statistics to emphasise the effects of benchmarking, but what does its real impact look like on a business? We interviewed Nebel Crowhurst, Head of People Experience at River Island, to find out what role the Towards Maturity Health Check played in helping her develop her organisation into one that facilitates a high-performing learning culture.

Our Ambassadors

Our Supporters also influence Towards Maturity's Health Check and research, providing insights on future trends and practices that should be investigated.

Pin It on Pinterest