Outcomes from the challenge to change activity: Newcastle 24 March 2011
Group 1
What is your challenge?
Help me manage my workload
What would success look like?
What technology-enhanced practices might help you get there?
-
Collaborative tools that help students to ‘talk’ to each other and give feedback to all eg discussion boards
-
Online guidance in the form of podcasts
-
MCQs which give students individual tailored feedback
-
Video resources for revision – could be created by students provided verified by tutors
Who would you need to involve?
-
Educational technologists and IT technicians
-
Awarding bodies (so that some practice can be high-stakes)
-
Students
-
Tutors
-
Senior managers to ensure time allocation
What do you need to keep in mind?
There has to be an initial teacher input to get student-generated practice started. There will be additional skills required to ensure quality. Timeliness and readiness are also prerequisites for success. Champions (student or staff?) could be vital communication strategies to spread awareness of the approach and get it established.
Moderator’s comments
Technologies such as online collaborative tools and audio recording software have proved their worth in reducing the burden of assessing and providing feedback to large groups. The evidence is difficult to ignore. For example, see the trials of podcasting and Wimba voice boards (like discussion boards but voice recorded) that the University of Leicester have conducted as part of a JISC-funded project called DUCKLING (Delivering University Curricula: Knowledge, Learning and Innovations Gains) from the Transforming Curriculum Delivery programme – Case study 6 in the guide with a longer version online. If you want to read the DUCKLING final report, it’s also online. For an example of discussion boards, look at the very powerful REAP case study on delivery of a 1styear psychology programme (see page 14 in the guide).
Many theorists (eg Nicol, Sadler) argue that students are an under-used resource when it comes to improving assessment and feedback practice and that involving students in assessing and giving feedback to one another helps them understand better the criteria and the standards against which they are assessed themselves. A win-win, smiles-all-round scenario - but as this group point out, you need to have a measure of quality control and possibly skills in online moderation.
Is it time that online moderation formed part of every teacher training programme? Don’t be put off if you don’t feel well-trained in this regard. Using tools at a formative level can get everyone up to speed. An interesting discussion of the skills involved (and some leads to online courses) can be found at http://www.alt.ac.uk/docs/SW_eLearning_conference_20031005.pdf
Group 2
What is your challenge?
Effective use of e-portfolios for assessment – eg knowing how to assess and moderate the diverse and personal nature of e-portfolios
What would success look like?
What technology-enhanced practices might help you get there?
Who would you need to involve?
What do you need to keep in mind?
Both staff and students need training or exemplars for effective e-portfolio usage in assessment.
Moderator’s comments
e-Portfolios are often seen as dividing into two distinct types: the reflective e-portfolio (which may be assessed as part of PDP on an HE course) or a vocational e-portfolio containing evidence of competences eg NVQ portfolios in FE).
In practice, issues such as personal choice, depth of reflection and the fair assessment of different formats occur in both types. Questions that frequently arise in both cases include: how do we manage length/complexity of student portfolios and consequent use or misuse of tutor time for assessment? How do we establish equivalence – for example, some students may take the option of video recording their reflections and/or outputs rather than writing about them. Should students who do this be marked on the same scale as students who demonstrate command of complex linguistic skills? How do students acquire the ‘language’ that enables them to be understood by teaching professionals, assessors and employers, thus making the process worthwhile?
We cannot pretend these aren’t challenges or that we have all the answers. Sometimes technology makes possible forms of learning and assessment that we still have to catch up with culturally. This group has identified acceptance of diversity as the first step towards success and I have to agree with them. A degree of personalisation in assessment is something that we should aim for. See also What’s it worth, a JISC TechDis project which highlights the value in inclusivity terms of providing choice in how students are assessed.
This group is clear that any such innovations require a multi-disciplinary approach. What is your view?
A useful discussion of e-portfolio issues can be found in the e-Portfolios infoKit and in the JISC Effective Practice with e-Portfoliosguide. See also more recent case studies from the JISC ESCAPE project.
Group 3
What is your challenge?
Student submission overload
What would success look like?
What technology-enhanced practices might help you get there?
A database of all assignments set across a programme of study involving different modules or units to avoid student and tutor overload.
Who would you need to involve?
What do you need to keep in mind?
Moderator’s comments
We didn’t get any more comments recorded from what was clearly a valuable discussion. But the timing of and communication about assessments is something that technology can clearly help with. Read Case study 1 from the University of Glamorgan (longer version online). The technology is quite ‘old’ (an Access database) but the effect quite far-reaching in terms of enabling students and staff to achieve more. Also take a further look at the outputs from the ESCAPE project which illustrate patterns of assessment and focus attention on the likely consequences of each pattern.
Group 4
What is your challenge?
Effecting cultural change – prompted by the Let’s celebrate what’s good Challenge to Change card
What would success look like?
-
Clarity over what is ‘good’ practice
-
Quality from both a student and a staff perspective universally recognised and understood
-
Everyone on board, not just the enthusiasts
-
Student retention improved
-
Better grades and improved employability for students
What technology-enhanced practices might help you get there?
-
Using video to record case studies
-
Blogs set up to share and reflect on what is meant by good practice
-
Repositories of examples of good practice
Who would you need to involve?
Senior management, existing School and Departmental networks, learning technologists, students
What do you need to keep in mind?
Moderator’s comments
There are numerous ways that technology can help establish the principles underlying proposed changes to assessment and feedback. The most obvious is to share stories via the VLE or VLE-based blog. Staff who keep e-portfolios of reflection on their CPD can also share aspects of their e-portfolios with others using the same platform – see the Flourish project . This approach can be more personal and therefore more persuasive for the late adopters resistant to an institutionally owned process. But we are all agreed – institutional celebrationof change is vital to make everyone feel valued. Do we need to force change when an enjoyable sharing process can bring about the same effect?
Group 5
What is your challenge?
Delivering online summative assessment
What would success look like?
What technology-enhanced practices might help you get there?
-
Turnitin
-
Wikis
-
MCQ tests
-
Assessment 21 (or similar) assessment systems that can cope with short answer free text questions and involve human and computer collaboration ie operating not just as automated marking tools but assisting staff in marking the way they want to mark
-
More prioritised feedback
Who would you need to involve?
What do you need to keep in mind?
Moderator’s comments
Making best use of assessment systems is not widely understood although they have been available for some years now. Note that Assessment 21 is now a commercial enterprise although it originated at Manchester University. Read more about the pedagogic and time-saving gains the tool can offer in Case study 5.
Group 6
What is your challenge?
I want assessment to engage my students and support deeper learning.
What would success look like?
-
Retention of students
-
Increased participation and engagement in feedback
-
Improved performance
-
Engagement with assessment process: design, criteria writing, feedback
What technology-enhanced practices might help you get there?
-
Wiki or discussion forum on assessment and feedback design
-
Online peer assessment tools
-
Online feedback delivered via video and audio as well as written
Who would you need to involve?
Students, programme leaders, technology advisers, staff developers, student induction team
What do you need to keep in mind?
-
Need for effective staff development
-
Availability of hardware and finance
-
Accessibility of the technologies to all students
-
Self-regulation
-
Fun
Moderator’s comments
This group has stressed the need for good induction for staff and students; learning literacy and digital literacy skills are now high on everyone’s agendas. A JISC publicationintroduces this topic in a readily accessible way if it’s not something you have followed before. There will be a new JISC programme focusing on this theme.
There is evidence emerging that technology-enhanced assessment does engage students better and (although it is always difficult to assert a definite cause and effect link) that student performance and retention improve accordingly. The guide Effective Assessment in a Digital Age provides both anecdotal evidence and evaluated work both in the case studies and as part of the supporting discussion. I also like the suggestion that fun is an essential ingredient in engaging students in assessed work. What can you do to make assessment more enjoyable?
Group 7
What is your challenge?
Getting students to support and learn from one another
What would success look like?
A discursive, useful environment where users could develop their own work via interaction with others but also be able to see the value of offering support to one another
What technology-enhanced practices might help you get there?
We have tried to use discussion boards.
Who would you need to involve?
-
IT teams
-
Programme leaders
-
Students
What do you need to keep in mind?
-
Communication strategies are important ie telling students why and how the boards will form part of their programme of work
-
Structuring discussion activities for students ie this week can you post something you have found useful for..
Moderator’s comments
A closed informal forum in which students (minus staff) can discuss the mistakes they made in an assignment, and share reviews of marked assignments from previous years, can be also be an invaluable support for learning, as are peer assessment exercises. Do we always need to be monitoring what students do? Informal student-to-student support networks emerge independently anyway but may exclude some of the most vulnerable students, so it could be useful to set up a ‘semi-official’ student support forum, seed the initial stages, then retire...
Group 8
What is your challenge?
Effecting cultural change (Let’s celebrate what’s good)
What would success look like?
-
Different individuals coming forward, not the same folk every year
-
Teaching fellowships
-
Case studies on a website eg the LSIS Excellence Gateway
-
Regional awards
-
Recognised roles such as Subject Learning Coaches, PDAs (FE)
What technology-enhanced practices might help you get there?
Who would you need to involve?
Teacher educators
What do you need to keep in mind?
Moderator’s comments
A lot of good staff development materials have come forward in recent years, but can lurk unused on websites or on the top shelves of staff rooms. We agree that an essential ingredient for change is managerial support for bottom-up initiatives that are often driven by awareness of local, disciplinary or learner needs.
Give your managers a copy of the Effective Practice guides. They can still be ordered online but are unlikely to be reprinted in the current climate so act quickly! If you are in an FE context, you can argue the importance of preparing students for HE by selecting practices that fit your context – the ideas are usually supportive of work both sectors. If you are working in HE, case studies aim to be generic examples of practice although set in particular disciplines. A starting point could be to discuss what is relevant or not relevant in your context.
Video can be a powerfully persuasive medium. Have a look at the video case studies at www.jisc.ac.uk/assessresourceor at www.jisc.ac.uk/resourceexchange
Comments (0)
You don't have permission to comment on this page.