Outcomes from the challenge to change activity: Bristol 3 March 2011

 

Group 1

What is your challenge?

To deliver an engaging, authentic summative assessment online

 

What would success look like?

• Students integrating and improved performance

• Ability to carry forward experiences beyond the assessment

• Ability to apply their understanding to new situations

 

What technology-enhanced practices might help you get there?

• Banks of online questions which can be used across modules; make use of [shared?] archives

• Reflective e-portfolios or online journals

• Role-playing in online problem-based scenarios

• Collaborative tools – can be synchronous in which case record for future reflection

• Web conferencing tools – to enable students to bring in their own experiences

 

Who would you need to involve?

Representatives of IT and learning: learning technologist, lecturers, students, project manager, instructional designer, programme approval board. Communication is vital.

 

What do you need to keep in mind?

• Students need opportunities to familiarise themselves with the tools used BEFORE an assessment

• Time for development and testing

• Support (for staff?)

• A ‘Plan B’ will be needed

• Planning eg on the budget is needed over the short and long term

 

Moderator’s comments

This response reminds us of how important it is – given the current constraints on time and money plus increasing demands on teaching staff – to keep abreast of onscreen or computer-marked assessment. There are two important gains: efficiency (reuse of questions built up and stored in question banks over a period of time) and authenticity (especially where the online assessment is linked to problem-based scenarios or involves immersion in a virtual reproduction of real-world activities.) However, the number of technologies listed here suggests that investing in more than one technology might be desirable ie onscreen assessment could lead to reflection in an e-portfolio or also involve online forums or presentations?

 

There are case studies that might be useful here. Case study 2 in Effective Assessment in a Digital Agecovers the preparation and support needed to deliver summative assessment online. (No suggestion intended that traditional methods demand a less rigorous approach, it’s just that the timing of the workload can be very different ie before rather than afterwards and the support systems must include IT.) It would also be worth looking at Case study 10, where video is used for assessment of students’ skills and for reflection and self-assessment. That’s another way in which technology scores – do it for one purpose, then store and use it for another! Could video be used for assessment in your discipline?

Group 2

What is your challenge?

Engage students better with assessments

 

What would success look like?

• Good feedback to students results in retention and positive final results

• Students enjoy the assessment process

• Students appreciate the relevance of assessment criteria

 

What technology-enhanced practices might help you get there?

• Collaborative group work – wikis and/or blogs

• Choice of approach eg assessed online discussions

• Recording feedback given to students eg email, video conferencing

 

Who would you need to involve?

Technical support

 

What do you need to keep in mind?

Equipment

 

Moderator’s comments

Two JISC-funded projects that have made an impact on the well-known difficulty of getting students to engage with assessment criteria and feedback are Making Assessment Count (University of Westminster) and WebPA (Loughborough University and the University of Hull). These are described in detail online at www.jisc.ac.uk/assessresourceand more briefly in Case studies 8 and 7 in the Effective Assessment in a Digital Age publication. There are also videos available at www.jisc.ac.uk/assessresource. Enjoy!

 

Group 3

What is your challenge?

Making assessment in vocational qualifications more authentic

 

What would success look like?

Assessments and qualifications that would give employers true confidence in the ability (of newly qualified students) to do the job in a true work environment

 

What technology-enhanced practices might help you get there?

Video clips of practice Audio reports

ePortfolio

 

PDP Demonstration of team work

 

Reflective narratives Placement employer blog

 

Who would you need to involve?

Technical teams, professional groups, industry-based bodies, senior management, QA,

 

What do you need to keep in mind?

You will need time and space to implement the practice, possible adjustment of existing assessment policies, guidance, support and resources. You will need buy-in from senior management (to drive the initiative forward). To convince external bodies you will need evidence of potential benefits. Students will need the necessary skills and access to technology.

 

Group 4

What is your challenge?

Students making the same mistakes and being able to locate an answer.

 

What would success look like?

• A one stop shop for help and support with an aggregation of information

• Move towards more of a university approach

• Staff and students aware of online help and resources and a noticeable improvement in the incidence of common errors

 

What technology-enhanced practices might help you get there?

Web / online resource (on the VLE?) with flexible web features eg RSS feeds and communication tools, searchable and refined by user stats.

 

Who would you need to involve?

Senior management support, everyone including the student body.

 

What do you need to keep in mind?

The resource must be flexible and sustainable. There will be investment of time required so manage expectations carefully. Gain engagement with the resource by sharing good practice.

 

Moderator’s comments

It’s a really smart move to upload student-generated resources to an online resource to stop reinventing the wheel each year. With students’ permission, you could upload marked answers from previous years for discussion, for example, or use an unmarked fabricated answer which students mark, then discuss and justify their grade against the published criteria. Both exercises will help students engage better with a task, understand the criteria more fully and recognise the standard required. Multiple-choice questions embedded in support documents such as a pre-assignment tutorial in PowerPoint can also give students the opportunity to see if they have grasped what is required of them. You could use QuestionMark Perception to draw up questions, but there are some simple tools if QMP is unavailable to you eg Hot Potatoes and Quandary or QuiaTo use PowerPoint as an assessment tool, see http://www.wonderhowto.com/how-to-make-powerpoint-multiple-choice-quiz-14869

 

Group 5

What is your challenge?

Providing learners with richer and timelier feedback

 

What would success look like?

• Feedback that is workload-friendly for staff

• Enhanced learner experience

• Feedback that can be personalised

• Improved turnaround time

 

What technology-enhanced practices might help you get there?

Audio feedback via Wimba Voice, a VLE-based solution that is easy to use and scalable, robust, safe and secure.

 

Who would you need to involve?

Learning technologists, students, practitioners, SMT, IT team

 

What do you need to keep in mind?

Organisational barriers

Not making draft versions

Investment

 

Moderator’s comments

Speak to your IT team about Wimba Voice for recording emails. It’s much easier and quicker to speak than it is to write feedback, and more personal too. See www.youtube.com/watch?v=2PbBqpGTEGs.

 

There are several JISC case studies on using Audacity(free software) to record audio files which can be distributed via email or to groups via the VLE. Have a look at Case study 6 inEffective Assessment in a Digital Age;the DUCKLING project team at the University of Leicester have accumulated a lot of useful advice on audio feedback. There is also Case study 3on audio feedback in Effective Practice in a Digital Age

 

Group 6

What is your challenge?

Using online assessment when lacking skills or confidence in using technology for teaching purposes (including having access to technology)

 

What would success look like?

• Widespread digital literacy amongst staff and students

• Use of technology commonplace in assessment

 

What technology-enhanced practices might help you get there?

• Effective staff development (students could ‘develop’ staff)

• Open access to podcasts and other technologies

• Awarding bodies accept digital evidence for assessment purposes eg wikis

• Institutional change: legitimise technology for use in teaching and learning

 

Who would you need to involve?

Senior management, awarding bodies, Government, students and staff

 

What do you need to keep in mind?

Workload, classroom management, clear vision

 

Moderator’s comments

In some FE colleges, schemes have been run where students have been e-mentors to teaching staff: www.excellencegateway.org.uk/page.aspx?o=192263. The idea also surfaces as student to student mentorship as at the University of Staffordshire – see www.staffs.ac.uk/schools/sciences/ementoring

 

Group 7

What is your challenge?

Enabling students to learn from their mistakes

 

What would success look like?

• Students able to give feedback to one another overseen by tutors

• Mistakes become learning resources

 

What technology-enhanced practices might help you get there?

• Turn into anonymised artefacts and reuse

• Simulations and branching resources eg Quandary

• Collaborative tools so that students can make revisions to their work prompted by peers

 

Who would you need to involve?

• Educational designers

• Departments and services

 

What do you need to keep in mind?

Accessibility, technical requirements, IPR, development time, marking workload

 

Moderator’s comments

There is tremendous potential in using technology to support assessment of real-world practices in a safe online ‘fake’ world, usually as tools for self-assessment it has to be said, but that could soon be changing in a range of disciplines. Simulation software and recreations of work-based environments in which assessment can take place are being developed. Examples include the oil rig evacuation recreated in Second Life for occupational psychologists as part of the DUCKLING project at the University of Leicester (www2.le.ac.uk/departments/beyond-distance-research-alliance/About%20Us/videos), Shareville, a townscape recreated at Birmingham City University (see Case study 8in Effective Practice in a Digital Age and virtual patient software as described in Case study 9 in Effective Assessment in a Digital Age). The training and assessment of medics and paramedics is ahead of most disciplines in this respect.

 

Simpler, less expensive options include a non-threatening closed forum in which students can discuss the mistakes they made in an assignment, online reviewing of marked assignments from previous years and peer assessment exercises.

 

Group 8

What is your challenge?

Students knowing what to do with feedback

 

What would success look like?

• Students want their feedback ie they value, understand and reflect on feedback)

• Students act on feedback and share feedback (with tutors and peers)

 

What technology-enhanced practices might help you get there?

• Wiki with peer review

• Reflective blog

 

Who would you need to involve?

Students and those managing the infrastructure eg the VLE

 

What do you need to keep in mind?

 

Moderator’s comments

Have a look at Case studies 4 and8 in Effective Assessment in a Digital Age. The immediate feedback of computer-marked assessments can be very effective. Building in an additional reflective layer as described in Reflecting on feedback (video) can sometimes bear fruit.

 

Timing really matters to the effectiveness of feedback. Case study 1 in Effective Assessment in a Digital Agereminds us that sometimes students can’t act on feedback because their workload has bunched and because they may only receive feedback after they have started the next assignment. Technology can be usefully deployed in rectifying these situations.

 

Group 9

What is your challenge?

Getting students to evaluate their work fully

 

What would success look like?

Parity between tutors’ and students’ understanding of evaluation

 

What technology-enhanced practices might help you get there?

• WebPA

 

Who would you need to involve?

 

What do you need to keep in mind?

Emotional issues. Monitoring feedback students give to one another

 

Moderator’s comments

There is a brilliant video on the peer moderation tool, WebPA, which explains how it can be used at www.jisc.ac.uk/assessresource