MAF #3: Inclusive assessment – and a little problem for the holidays

There are many reasons to consider alternative or inclusive assessments – from the push for more engaging and compelling assessment tasks and modes – you know – tasks that are more exciting than producing a recursive report that mainly regurgitates the course; to designing out or preventing plagiarism; to complying with the law on inclusive practice in re students with disability. This post focusses on the Disability Law in the UK – but the advice, guidance and thought processes provoked are designed to get us all thinking more about what assessment we are setting – and why… WHY do we set an essay or the production of a learning and teaching resource? Why do we require a group presentation or an exam? What are we looking for in both the FORM and CONTENT of the assignment? What knowledge and skills ado we want the students to harness and reveal?

Objectives:

to have considered a range of ways of supporting students (with disabilities) in achieving their potential

to have a clearer understanding of the value of supporting students (with disabilities) through alternative assessment approaches.

Activity: Begin thinking about 'inclusive assessment' - what are your first responses to the pharse: inclusive assessment? What is your understanding of that inclusive assessment? 

How would you define 'inclusive - or exclusive - assessment'?  Have you one or two examples from your own experience that you feel demonstrate either inclusive or exclusive assessment practice particularly well?

Some reading and listening:

First, a blog post from our very own Sandra Sinfield, following her experience of the Discourse, Power and Resistance conference in 2013 – which all variously tackled inclusion: 

http://lastrefugelmu.blogspot.co.uk/2013/04/discourse-power-and-resistance-2013.html

Second, chapter 8 of our University Assessment Framework on 'Assessment for Students with Disabilities', pp. 38-42, covering the legislative context, concepts of 'reasonable adjustments' and 'preventing substantial disadvantage', examples of adjustments for different types of assessment, and the role of the DDS – or the relevant chapter in your own institution’s framework.

Third, do listen to this Hybrid Pedagogy podcast on the role of compassion in assessment: http://www.digitalpedagogylab.com/hybridped/assessment/

 

Disability equality duty

All public sector bodies have the following ‘general duties’:

promote equality of opportunity between disabled people and other

people

eliminate discrimination that is unlawful under the Equality Act 2010

eliminate harassment of disabled people that is related to their disability

promote positive attitudes towards disabled people

encourage participation by disabled people in public life

take steps to take account of disabled people's disabilities, even where that involves treating a disabled person more favourably than other people.

Maintaining academic standards

The Disability Rights Commission DDA part 4, Examinations, pp 6-7:

“The purpose of assessment, whether it is formative or summative, is to determine a student’s academic achievement and skills.  To do this, examinations and assessments must be rigorous regarding standards so that all students are genuinely tested against an academic benchmark. 

But, similarly, if they are to fulfil their purpose, they must also be flexible regarding the model of measurement so that each student has an equal opportunity to demonstrate their [sic] achievement.  In some cases this may mean changing the existing examinations or assessment practices within an institution. 

In all cases it will mean being clear about precisely what is being assessed so that modifications may be made without compromising academic standards…The aim, wherever possible, is to change the delivery or mode of assessment, not to change the way the assessment is marked.”

Assessments fair – but still rigorous!

We need to modify assessments without compromising academic standards:

“the aim is wherever possible to change the delivery or mode of assessment, not to change the way the assessment is marked...”:

all students should be tested against an academic benchmark

assessments must be flexible regarding the mode of measurement - this may mean changing existing assessment practices

what is being assessed must be clear.

Amendment to DDA, 2005

Introduced the concept of ‘competence standards’ – defined as: an academic, medical, or other standard applied by or on behalf of an education provider for the purpose of determining whether or not a person has a particular level of competence or ability...

Competence standards are, effectively, entry and assessment criteria. They must be reviewed from a disability discrimination perspective and must describe relevant and genuine competences that are strictly necessary for programme completion. This will ensure that all students can demonstrate their particular competence or ability in a particular area.

What would be considered to be a competence standard?

For example, the personal ability to perform a particular skill or technique e.g. in:

dentistry or other clinical field...

journalism...

physiotherapy...

translation...

counselling psychology...

There can’t be any adjustments made here, either in the form of, say, extra time or permitting the task to be undertaken by a practical assistant at the student’s direction

What would not be considered to be a competence standard?

Under the terms of the DDA, the following examples are unlikely, in most cases, to amount to competence standards:

being able to cope with the demands of a programme (which might be relevant to some students who have anxiety conditions)

having good health and/or fitness (rather than sufficient health)

specific levels of attendance (although attendance is linked to learning, particularly with reference to practical skills; students need to show how they can “catch up” on such skills following non-attendance)

speaking or writing clearly (unless it is essential to the nature of the work).

Setting a competence standard – a guide:

Identify the specific purpose of the standard.   Apply these questions to the standard:

Does the standard apply to all students?

Does the standard have a legitimate aim?

Does the standard meet the specific purpose?

Does the setting of this standard impact negatively on disabled people? If so, is the application of this standard absolutely necessary?

Is this the only way the purpose can be achieved?

Have you taken account of any changes that may affect this standard (such as changes in technology)

If, after considering all of these questions, the standard is still the only method of achieving the aim set, then it is likely to be objectively justifiable and therefore lawful.

Helping students - setting a task providing:

information about how the task contributes towards the overall objectives for the module

transparent success criteria

clear written instructions

where appropriate, the opportunity to discuss the instructions

prioritised reading lists.

Helping students - completion of task

be pro-active in supporting students...

be approachable and ensure that the student knows who they should contact for assistance

where possible, ensure the student has the opportunity to submit a draft for comment

be aware of the issues faced by the student and be patient.

Useful support materials

clearly written course objectives

assessment criteria for tasks

clear and explicit guidelines to help students achieve success

concise and explicit reading lists

glossaries of key terms (simple, straightforward explanations)

provide diagrammatical models/representations whenever possible.

Types of alternative assessment

option to produce another piece of coursework instead of exam

oral presentations and defences

portfolio development

project development

video diaries

viva voce.

Inclusive learning, teaching and assessment website at Plymouth University:

https://www.plymouth.ac.uk/your-university/teaching-and-learning/inclusivity:

Guidance on how to foster inclusive and engaging teaching and learning

environments

Video case studies of academic and professional staff talking about successful inclusivity initiatives

Videos of our students talking about their study experiences and how they might be improved

Guidance on developing inclusive assessment

Links to key debates, research and resources about inclusive practice in Higher Education.

MAF and the alternative assessment

In our case – we have asked for a multimodal or digital artefact as an alternative to the traditional group presentation…

Activity: and our vacation ‘problem’ is – what specific criteria would you generate for grading the artefact – taking into consideration:

The generic MALTHE grading criteria

The MAF Learning Outcomes

The ‘affordances’ of a multimodal or digital presentation.

EXAMPLES:

Examples of multimodal resources

From #edcmooc:

#EDCMOOC – here’s a short artefact that Andy made in GoAnimate of some of my text: http://www.youtube.com/watch?v=dfx1_fVZbyI

Here’s the artefact that I made myself when I did the MOOC again (this time as a CTA):

https://becomingeducational.wordpress.com/2013/12/03/in-the-beginning/

Angela’s thinglink: http://www.thinglink.com/scene/360982057624535042#tlsite

Fran Monaghan’s VoiceThread – beautiful, gentle and a sort of low-tech, high-tech: http://voicethread.com/?#q.b4186028.i21377601

And Ess Garland’s timeline – and now for something completely different!

http://www.tiki-toki.com/timeline/entry/101035/Edmooc-Digital-Artifact/#vars!date=2013-02-28_06:07:26!

Terry Elliot’s Zeega: http://zeega.com/162387 and our

Collaborative poem: https://titanpad.com/sXgaTJMniP

Writing ones – scroll down:

http://learning.londonmet.ac.uk/studyhub/writing.html

First year Becoming an Educationalist students - 2015/16:

Creativity and me: https://www.powtoon.com/online-presentation/dEIoVM0pw6W/

Finding creativity: https://www.youtube.com/watch?v=KOW3wq57Q5s&feature=youtu.be

Team work: https://www.powtoon.com/online-presentation/dbBTDtPjk81/ (broken)

Planning an essay: https://www.youtube.com/watch?v=jMlvbODnjmE&feature=youtu.be

Handing in your first assignment: https://www.youtube.com/watch?list=LLxula4mg1zVEuvQdeGR2VIQ&v=SuteEXGBHXo&app=desktop

Transition to uni – comic strip: https://www.pixton.com/uk/comic-strip/rtimsfe9

Studying feels like: https://www.powtoon.com/online-presentation/fa71WeC43mQ/caroline-digi-me/#/

Digital ME (studying and creating in a digital age): https://www.powtoon.com/online-presentation/fs3MBKCr7o1/

2014/2015: Welcome to LondonMet (livescribe): https://www.youtube.com/watch?v=VMzY4pB_Vq4&feature=youtu.be

 

MAF Preparation for Session #2 (Spring 2016): Assessment of- versus assessment for learning

It is argued that many students do not read or even access their feedback – many do not perceive the formative value of summative feedback, especially when a particular course of study is over. Typical practice sees feedback as correction, or justification, as opposed to dialogue – and we lecturers are offered little guidance on how to offer constructive and accessible feedback. Written feedback especially places the student in the role of passive recipient of judgement – no wonder that feedback is emotional.

It started with the prep

For our second MAF workshop, we explored the use of assessment of vs assessment for learning.  In preparation, we asked participants to:

READ sections 6 and 7 of our University Assessment Framework on 'Marking' (pp.28-32) and 'Feedback' (pp. 33-38) ( https://metranet.londonmet.ac.uk/fms/MRSite/psd/hr/capd/Assessment%20Framework/University%20Assessment%20Framework%20Oct%202010.pdf)

WATCH a short video on 'Assessment and feedback - dialogical and relational’ -  specifically at the one on the feedback process http://www.brookes.ac.uk/aske/MultimediaResources/

ASK yourself: What do I feel I am already doing well in the way I mark and provide feedback to students? What are my biggest challenges when it comes to implementing effective marking and feedback practice? How confident do I feel that my students are able to make sense of and use feedback beneficially for their learning?  

Assessment of versus assessment for learning.  Discuss.

We opened with a short writing activity: Drawing directly on the pre-reading and viewing you have done for this class, write for 20 minutes on the above topic. You may wish to freewrite or brainstorm ideas first; Resources can be consulted directly, as in an open-book exam setting; Your writing will be peer-reviewed according to the MALTHE grading criteria.

A bit of a shock at first – these sorts of short, very focussed writing activities can be an excellent way of opening or closing a seminar – or even a lecture. They help students to learn through writing – and to learn that writing in this way is part of the learning process.

Peer Review

After the writing – we went into a peer review process – asking people to grade and give useful feedback on the writing – according to our module criteria. At first people only received the written feedback – and then we moved on to discuss the feedback.

Again we attempting to model the student experience – so that we could experience it ourselves in a very embodied and perhaps emotional way.

Unsurprisingly, in our session, everybody appreciated the opportunity to discuss the feedback – feeling typically that it was only in the discussion phase that real benefit was gained from the feedback process.

These days large class sizes make it extremely difficult to make class time available for this sort of dialogic encounter. It was suggested that audio-feedback could simulate this process to some extent – typically spoken feedback is experienced less negatively (http://jolt.merlot.org/vol10no1/cavanaugh_0314.pdf) or at least may be engaged with differently (especially if the class itself is mainly taught online).

Another alternative used by several members of the group is to scaffold meaningful peer review – and to use this opportunity for timely, formative feedback. With peer review, the receiver of feedback a participant in process, not just a passive recipient. The process itself requires the receiver to engage with and act upon feedback – and the focus of feedback is on supporting learning, rather than justifying a mark. Moreover – this practice changes the feedback dynamic from one of correction – to the dialogic co-production of knowledge: The dialogic co-construction of knowledge is a particularly pertinent, though sometimes underrated element in academic knowledge production Olga Dysthe (2003).

Image Mediated Dialogue

Our next step was to use IMD to seed discussion on the experience of feedback: the giving and the receiving of it. IMD is dialogue mediated by an image – where participants choose their own image in response to prompt questions – here on the feedback experience – they then briefly WRITE a literal description of the picture – followed by a brief analysis of why it answers the questions; this is followed by discussion. At the end of the discussion we asked participants to write their conclusions and the implications for their practice of what they had covered.

Application to practice

Based on an exhaustive literature review, Gibbs & Simpson (2004) identified 11 conditions under which assessment best supports learning, 7 of which pertain to feedback:

1. Sufficient feedback is provided, both often enough and in enough detail
2. The feedback is provided quickly enough to be useful to students
3. Feedback focuses on learning rather than on marks or students themselves
4. Feedback is linked to the purpose of the assignment and to criteria
5. Feedback is understandable to students, given their sophistication
6. Feedback is received by students and attended to
7. Feedback is acted upon by students to improve their work or their learning

Tip: take Principles 3, 5 and 7 – and after reflecting on the session as a whole WRITE what you might do in one of your modules so that feedback practices embody these principles.