MAF Preparation for Session #2 (Spring 2016): Assessment of- versus assessment for learning

It is argued that many students do not read or even access their feedback – many do not perceive the formative value of summative feedback, especially when a particular course of study is over. Typical practice sees feedback as correction, or justification, as opposed to dialogue – and we lecturers are offered little guidance on how to offer constructive and accessible feedback. Written feedback especially places the student in the role of passive recipient of judgement – no wonder that feedback is emotional.

It started with the prep

For our second MAF workshop, we explored the use of assessment of vs assessment for learning.  In preparation, we asked participants to:

READ sections 6 and 7 of our University Assessment Framework on 'Marking' (pp.28-32) and 'Feedback' (pp. 33-38) ( https://metranet.londonmet.ac.uk/fms/MRSite/psd/hr/capd/Assessment%20Framework/University%20Assessment%20Framework%20Oct%202010.pdf)

WATCH a short video on 'Assessment and feedback - dialogical and relational’ -  specifically at the one on the feedback process http://www.brookes.ac.uk/aske/MultimediaResources/

ASK yourself: What do I feel I am already doing well in the way I mark and provide feedback to students? What are my biggest challenges when it comes to implementing effective marking and feedback practice? How confident do I feel that my students are able to make sense of and use feedback beneficially for their learning?  

Assessment of versus assessment for learning.  Discuss.

We opened with a short writing activity: Drawing directly on the pre-reading and viewing you have done for this class, write for 20 minutes on the above topic. You may wish to freewrite or brainstorm ideas first; Resources can be consulted directly, as in an open-book exam setting; Your writing will be peer-reviewed according to the MALTHE grading criteria.

A bit of a shock at first – these sorts of short, very focussed writing activities can be an excellent way of opening or closing a seminar – or even a lecture. They help students to learn through writing – and to learn that writing in this way is part of the learning process.

Peer Review

After the writing – we went into a peer review process – asking people to grade and give useful feedback on the writing – according to our module criteria. At first people only received the written feedback – and then we moved on to discuss the feedback.

Again we attempting to model the student experience – so that we could experience it ourselves in a very embodied and perhaps emotional way.

Unsurprisingly, in our session, everybody appreciated the opportunity to discuss the feedback – feeling typically that it was only in the discussion phase that real benefit was gained from the feedback process.

These days large class sizes make it extremely difficult to make class time available for this sort of dialogic encounter. It was suggested that audio-feedback could simulate this process to some extent – typically spoken feedback is experienced less negatively (http://jolt.merlot.org/vol10no1/cavanaugh_0314.pdf) or at least may be engaged with differently (especially if the class itself is mainly taught online).

Another alternative used by several members of the group is to scaffold meaningful peer review – and to use this opportunity for timely, formative feedback. With peer review, the receiver of feedback a participant in process, not just a passive recipient. The process itself requires the receiver to engage with and act upon feedback – and the focus of feedback is on supporting learning, rather than justifying a mark. Moreover – this practice changes the feedback dynamic from one of correction – to the dialogic co-production of knowledge: The dialogic co-construction of knowledge is a particularly pertinent, though sometimes underrated element in academic knowledge production Olga Dysthe (2003).

Image Mediated Dialogue

Our next step was to use IMD to seed discussion on the experience of feedback: the giving and the receiving of it. IMD is dialogue mediated by an image – where participants choose their own image in response to prompt questions – here on the feedback experience – they then briefly WRITE a literal description of the picture – followed by a brief analysis of why it answers the questions; this is followed by discussion. At the end of the discussion we asked participants to write their conclusions and the implications for their practice of what they had covered.

Application to practice

Based on an exhaustive literature review, Gibbs & Simpson (2004) identified 11 conditions under which assessment best supports learning, 7 of which pertain to feedback:

1. Sufficient feedback is provided, both often enough and in enough detail
2. The feedback is provided quickly enough to be useful to students
3. Feedback focuses on learning rather than on marks or students themselves
4. Feedback is linked to the purpose of the assignment and to criteria
5. Feedback is understandable to students, given their sophistication
6. Feedback is received by students and attended to
7. Feedback is acted upon by students to improve their work or their learning

Tip: take Principles 3, 5 and 7 – and after reflecting on the session as a whole WRITE what you might do in one of your modules so that feedback practices embody these principles.

MAF#1: Introduction to Managing the Assessment and Feedback process

Welcome to the MAF blog – and to MAF itself. We look forward to exploring assessment and feedback with you…

This module asks you to investigate the theory and practice of Assessment and Feedback – and to relate that to your own practice. It is intended to help staff develop their current practice in ways that are helpful to them and to their students and to make assessment part of the learning dialogue: wrestling with assessment of, for and as learning. Each week there will be some preliminary reading and/or viewing to do before the class – which will be explored practically and dialogically in the session itself. This opportunity to work with others across the disciplines is what people value most about the course itself – and we can see already that this is a lively group with much to offer (see also arguments for a ‘flipped’ approach: https://net.educause.edu/ir/library/pdf/eli7081.pdf).

Assignments

Group presentation (40%): explore an aspect of assessment that intrigues you and produce a multimodal presentation or resource (5mins) that presents an issue, argument and/or ‘case’ to the class – with short critical commentary.

Individual project (60%): develop a sustained argument about a specific aspect of assessment related to your own context and perhaps extending an issue that arose in exploring the group project. Typically presented as a formal essay, we are open to creative interpretations of that genre.

Tip: Re-visit the module Learning Outcomes – and the notes you made when we discussed these in class. List the ways that YOU might be able to demonstrate that you have met each of the LOs.

Bring your own context

We started with a brief free write and longer discussion on what we like about our current assessment and feedback processes, what areas of curiosity and concern we have – and what we hope to gain from the module itself.

What we like

Thoughtful and transparent assessment design was valued: where the assessment is clearly relevant to the LOs; where it is weighted and balanced well across criteria; and where students can clearly see, ‘This is what they are looking for.’

Some liked tried and tested ‘safe’ or known methods of assessment – the essay and the exam – others argued that the ‘not safe is best’ – where the outcomes of the assessment are not fixed in advance – but the participants can argue or make a good case. The analysis of a Case Study was cited here: where the students are not led to a given answer – but must interrogate a case study and come up with their own diagnosis or draw their own conclusions.

There was an argument offered for a diversity of assessment instruments across a course: presentations, short tests (including online) and course work – so that all participants would have an opportunity to showcase their learning or harness their own preferred learning strategies.

A few offered arguments for what might be called the emergent graduate identity approach (Len Holmes: http://www.graduate-employability.org.uk/) – where thought is given to the ‘skills’, capabilities and capacities required of the graduate – and the course is worked backwards from there. Examples cited were the Montessori course where participants have to produce lessons plans, group presentations and give peer feedback – and the prospective Science teacher set many small stakes assignments – that could lead, say, to the planning and leading of a trip to the science museum.

This connected with arguments around developing self-assessment and the students’ ability to evaluate their own work; possibly enhanced where the learning and teaching clearly feeds forward into the assessment – and where the students make their learning conscious: in blogs and by illustrating the session and/or their learning in some way (this paper explores blogging to learn – in the context of two LondonMet modules – available from Investigations – or this link to Academia Edu: https://www.academia.edu/9117059/Disrupting_learning_landscapes_Mentoring_engaging_becoming_in_Investigations_in_University_Teaching_and_Learning_Vol_9_Spring_2014_pp15-21).

Finally – it was argued that re-designing the assessment could provoke positive change in the learning and teaching processes.

Concerns

How do we know that a particular assessment instrument is valid and fair? And – how on earth do we know what students are ‘making’ of the feedback? Here again there was discussion of the use of assignment change to provoke positive change in pedagogic practice – and the invitation to re-investigate assessments alternative to the essay – that involve the equivalent rigour, critical engagement and endeavour.

What we want MAF to do

Overall it seems that most people want to use the module to investigate what constitutes successful assessment – defined as tasks that that help students both to learn what they need to – and that help them transition to the next level of study. A bonus would be the opportunity to explore the assessment tools in WebLearn – and to work out how to make a good case for assessment change that would convince a line manager.

TMD: Cheating: friend, foe or scapegoat

Our getting to know you activity was a Topic Mediated Dialogue (TMD) session – where we investigated teaching, learning and assessment through the lens of ‘cheating’. After paired discussions, participants had to draw a representation of their partner – and use that to introduce them to the class.

NB: TMDs can be structured around any topic with which you want to engage - we also use the same TMD prompts to get students to explore their own thoughts on ‘cheating’: https://becomingeducational.wordpress.com/2016/02/08/becomingeducational-w16-analyse-that-now-this/.

Thank you!

Thank you for an engaged and engaging first session – we have already touched on key aspects of assessment and feedback that the rest of the course will explore in ever greater depth… We are looking forward to taking this MAF journey with you.

#artmooc week 3 Correspondence with memory – suggestions for practice

This week we studied ‘Mail Art’ or Correspondence with Memory – and connected with Fluxus Art of the sixties. There are elements of the personal and the surreal in this movement – from Van Gogh’s illustrated letters to his brother to Eleanor Antin and her series of 100-boot postcards capturing images of 100 boots as they journeyed from the Pacific to New York – and Ray Johnson’s provocative abstracts and collages – connected by cartoon-like bunny faces that then became a signature.

Our task was to create a memory of our own and capture it upon an envelope – front, back, sides and inside. This was to be extended by the insert(s) that we made and placed within the envelope; designed to intrigue the audience.

I chose to illustrate memories of Leysdown – a seaside town where we had a caravan by the sea. We used to go there from when I was about eight – and it was freedom.

I used a variety of media to make this artefact:

• Medium quality deckle paper for the envelope
• A4 paper for the insert
• Watercolours – for envelope and the ‘swimming’ shoes part of the insert
• Pencil and small brushes
• Small colour prints of Leysdown - pictures downloaded from the internet – cut to look like miniature postcards and photographs
• Shells and stones gathered from Leysdown itself
• Tissue paper sea and shore
• Small boat – in blue and white. 

I chose a square envelope shape so that it looked like an explosion of images and ideas when it burst open. I painted the front of the envelope to look like a postcard from the seaside. It is a Surreal representation (after Miro) of my niece sunbathing in black outside the caravan, by the sea. I chose Surreal to juxtapose the ordinary with the extraordinary – and thus illustrate the potentiality of the seaside. The external side flaps are drawn and painted with images of pill boxes – a common sight in and around Leysdown – from the war – but I’ve turned them into representations of the face of Ray Johnson’s rabbit – a nod to Mail art and its history and connecting it to Leysdown and its history. The top and bottom flaps show the flat sea and the sea forts (more memories of the War) in the distance – with seagulls. The images mirror each other: the top is more grey showing the more regular reality of the North Sea – cold and desolate – but the bottom image has more yellow and a lighter blue, it is slightly sunnier than the top. I’ve painted very sparsely, a few black lines of paint on white, on the inside of the envelope to create a simple, calm Buddha like face – representing my inner calm and peace when I go to Leysdown. The insert is a cut out of a ‘blind drawing’ of a pair of the ‘swimming’ shoes that I use, there are sharp cockles (shell fish) in the silt-like mud under your feet – I always try to swim there and it is so much easier when your feet are not bleeding. I also included many small pictures of the flat and desolate coast itself - and the shoes and the images are designed to fall out of the exploded envelope amongst small rocks and shells from Leysdown itself. I have placed this on layers of tissue paper – sheets of different blues and green mirroring the complexity of the seascape – and a layer of yellow for the shore. I added a small boat in blue and white to tie the whole together.

Here are a few pictures of all of the outside of the envelope – and the inserts – and the setting that I placed them in:

Possibilities for practice:  of course one possibility is just to do this Mail Art with your students as an alternative form of reflection – and perhaps as a precursor to writing. Other lessons: don’t wait for inspiration - because assessment can wound… and peer review is excellent. Let me unpack that a bit:

#5: Assessment can wound!

So – received my peer assessment and my grade – which incidentally was MUCH WORSE than the grade that my YOUNGER SISTER received! (And yes I do know that I am shouting.)

Three people reviewed my collage and none of them actually liked it. Nearly all of them gave me friendly and encouraging comments – but my grade was just a little over the average. Now I am not young and I’ve been knocking about Education for some very long while so I did not take to my bed and weep, but … Feedback can be dispiriting no matter *how* helpful. I wanted to discuss those comments with my assessors – and put my case – and change their minds… and yes I felt a lot less enthusiastic about this week’s task than I did before.  What can we do about this? We do mark and grade and we do give feedback and feed forward and we hope that it helps. Perhaps we need to actively develop resilience (not Gove’s form of resilience!) in our students so that they do not bend and break under our feedback? Perhaps we need to make more time to give feedback in person and in conversation – and wouldn’t it be great if that time was valued by our institutions.

#6: Let’s build in more work – and lots of peer review

MOOCs set a lot of work each week (and still we usually do *more*) – and peer review is considered to be part of the learning and part of the assessment process. I do know of postgraduate courses that set peer review, but everyone tells me that undergraduate students won’t do it. However, I hope that if we start in the first year and from the first week to set rigorous weekly tasks that are peer-reviewed as the norm, no one will even think to object. 

** If you have any suggestions and strategies for developing peer review in undergraduate courses – please do let me know. ***

#7: Don’t wait to be inspired – just do it!

As said, my feedback dispirited me somewhat, and I started to feel under-whelmed by this week’s task. However, I have studied often before – and I wanted to get the assignment done… so I just did it anyway. I always advise students not to wait for inspiration: have a set time and place to work – and just do it. Just start; just brainstorm; free write; put BLAH BLAH BLAH and move on … but get something down and you are on your way. I did exactly that for this task – and it is done.

Of course when working with ‘non-traditional’ students, we may need to help them build this work habit rather than just expect it to emerge. Hence the suggestion above that in at least one first year module we do set weekly, rigorous and hard tasks for them to do – and to peer assess as part of their learning process. If we help to build positive habits they too will be sustained when tired or uninspired or lovelorn or any of the things that get in the way of work.

NEXT WEEK – portrait with collage – ooo errrr.