BILD – The future of e-assessment

by | Apr 4, 2011 | Articles

These are the findings of City and Guilds following the implementation of Learning Assistant to support work-based assessment of some of their key programmes.  

The e-portfolio plays a vital role in:

  • providing faster and more efficient learning programmes;
  • reducing bureaucracy;
  • simplifying the jobs of assessors enabling them to focus more on feedback and assessment rather than managing huge volumes of evidence;
  • increasing the retention of learners in a learning programme;
  • increasing the ownership of learners in  their assessment programme;
  • providing more transparency for funding organisations; and
  • increasing the transparency of Quality Assurance for work-based assessment.

Learners using e-portfolios to assemble competency based evidence complete their qualifications 40% faster than those with paper based portfolios. Assessors say they are better able to manage their workloads.  Our own research also reports that the use of Learning Technology significantly reduces time to competency by 28% for the most e-learning mature organisations; additionally a case study on work-based assessment demonstrates speed to competency achieved in half the time. (click here for podcast and here for case study article).

However computer driven assessments themselves are having difficulty getting beyond multiple choice questions.

These were the main points to come out of a meeting of the British Institute of Learning and Development at City and Guilds head office in London on the 31st March.

The City and Guilds has been around for a long time being established in 1878 by the City of London and various craft Guilds. Throughout that time they have both remained up to date and maintained quality standards of assessment.

Andrew Boyle, Head of Assessment Research, presented a personal view of e-assessment. There is much still to be done as paper driven systems still dominate assessment procedures; whether work based or not. Isabel Nesbit the outgoing CEO of Ofqual (Office of the Qualifications and Examinations Regulator) is quoted as saying of computer driven assessment; “Let’s get this off the too difficult pile”.

Simulation, mobile media, virtual reality, 2nd Life, augmented reality, social networking, collaborative tools all give us the opportunity to get beyond multiple choice questions in assessment.

City and Guilds themselves have been using computer driven tests since October 2002 since they launched Global On-line Assessment GOLA. Initially usage was approximately 200,000 tests per annum and this has risen to 1 million per annum.  The overall UK market for e-assessment is around £250 million per annum so this is clearly a serious business. One of the most frequently used on-line tests is the Driving Test Theory Test.

However e-assessment is not yet doing enough; Generation Y (Those entering work now) are used to sophisticated, web-based interactive displays and communication.  Again quoting Isabel Nesbit; “Computerised exams should replace pen and paper test for a generation used to digital learning.” The strain of writing for three hours is a novel and tiring experience for Generation Y; although previous generations might also agree.
However many practioners in the field are uncertain about the future of technology in assessment. City&Guilds are experimenting with revolving cameras on PCs to act as invigilators and automatic recognition of candidates to alleviate some of these concerns although this does not take us beyond MCQs. Currently only a few topics are examined by computer based exams.  Some of these totally replace paper based assessment and others are a combination; blended assessment.  Achieving a close link between work performance and exam performance remains elusive for many.
The picture is much more positive for work-based competency assessment where the e-portfolio is proving a valuable tool. Andrew Stone, the Assessment Practice Manager gave the audience a very clear and entertaining presentation of how Learning Advisor works.

Competency based qualifications have a structure broken down into Units with Performance Criteria and Knowledge Statements. The traditional approach for work-based assessment is that candidates assemble a paper portfolio of evidence; witness statements, project work, comments from customers or line managers, perhaps photographs of completed jobs. Each piece of paper will refer to the Unit of competence and the Performance Criteria or Knowledge Statement. Of course one piece of evidence may refer to more than one item of assessment. Learning Adviser brings all this together in one electronic package.

The e-portfolio is a collection of evidence in electronic form suitable tagged and commented.  Evidence can include video, images, audio, emails, witness statements, spreadsheets, presentations, other documents, http links including links to Wikki sites (which may themselves be examples of collaborative exercise and therefore used by several students). Potentially the output from simulation and virtual reality scenarios could be included.

Andrew demonstrated how this might be used in the hypothetical Level 3 qualification in Corporate Skills.  This includes a Unit with the title “Attending and contributing to conferences and seminars”. This included Performance Statements: turn up on time; nod and smile at other participants, remain awake, contribute to discussion etc.  Photographs and videos of participants were added to a mythical student’s portfolio to demonstrate each of these performances.  The evidence was added to the performance statement in the system. The evidence is viewed by an assessor who can pass or fail the evidence and add comments.  An external verifier is then able to access the same record and comment on the judgements made by the assessor.

An advantage of the e-portfolio in this format is that both students and assessors can very easily get an over view of progress as each Unit can be displayed on one page with Assessor comments. At a glance it is easy to see if that Unit has sufficient evidence collected.

Learners can download their e-portfolios making it easier for them to keep a record of their progress and eventual completion.  Students like this approach as it is easier for them to collect evidence with every mobile phone being a video and audio recording device; e-portfolios don’t get lost, are simple to store and easy to save a copy.
Mobile technology also simplifies the process enabling not only evidence gathering on the spot but immediate transmission to the tutor and assessor. This strengthens the links between tutor and a remote work-based learner, a point made by a pilot study into the use of mobile technologies(PDAs) to help work-based assessment in the Health and Care sector starting as early as 2006.  The Centre for Excellence in Teaching and Learning  led a project on  Assessment and Learning in Practice Settings  with the aim of simplifying the collection of evidence for Health and Care students when working in hospitals, clinics and care homes.  Students felt far more supported by their University Tutor with mobile technology than they did without.

The conclusion from the conference is that the e-portfolio is a success, both from a commercial and learning aspect, but that e-testing still needs more development work to get beyond the multiple choice question.

Compare your L&D strategy with the Towards Maturity Learning Health Check

Compare your L&D strategy

Review your L&D strategy to discover your strengths and opportunities for improvement with the Towards Maturity Learning Health Check.

Featured content

Towards Maturity Learning Health Check season extended to Monday 16 September. Early 2019 results highlight that only 7% of L&D leaders report that their organisation encourages and provides time for reflection*

Towards Maturity Learning Health Check season extended to Monday 16 September. Early 2019 results highlight that only 7% of L&D leaders report that their organisation encourages and provides time for reflection*

By popular demand, the 2019 Learning Health Check season has now been extended until Monday 16th September. There is still enough time to complete your review and revisiting regularly can also help learning professionals to measure progress year on year, highlight areas where improvement is needed and provide evidence to build a business case for change.

We all have blind spots, why does being aware of yours add value?

We all have blind spots, why does being aware of yours add value?

When it comes to organisational culture, performance and learning, blind spots (or as they are often referred to ‘not seeing the light’) are usually found at the root of most of our problems with work today. These problems shape our day to day conversations, thinking, mood, judgement, decisions-making, or fundamental lack of them. These issues can make or break your workplace experience.

Featured

La Vie en Rose, does seeing ‘life in pink’ matter?

La Vie en Rose, does seeing ‘life in pink’ matter?

Learning can be defined in many ways, but most psychologists would agree that it is a relatively permanent change in behaviour that results from experience. The three major types of learning described by behavioural psychology are; classical conditioning, operant conditioning and observational learning. In order for learning to stick and becomes the new normal, all three types require self-driven will. Therefore, how people view the world through their lens matters.

Towards Maturity Learning Health Check season extended to Monday 16 September. Early 2019 results highlight that only 7% of L&D leaders report that their organisation encourages and provides time for reflection*

Towards Maturity Learning Health Check season extended to Monday 16 September. Early 2019 results highlight that only 7% of L&D leaders report that their organisation encourages and provides time for reflection*

By popular demand, the 2019 Learning Health Check season has now been extended until Monday 16th September. There is still enough time to complete your review and revisiting regularly can also help learning professionals to measure progress year on year, highlight areas where improvement is needed and provide evidence to build a business case for change.

Our Supporters also influence Towards Maturity's Health Check and research, providing insights on future trends and practices that should be investigated.

Pin It on Pinterest