[English] Meet the Assessors

Wed Feb 18 22:40:26 EST 2009

Malcolm I  didn't mark "Whose Reality" (we only marked two contexts each and I did Identity and Belonging and Encountering Conflict) but I think that it is true that the Context prompts suited some texts better than others and that some prompts were more accessible than others. There was quite some (heated) discussion about this point during one of the plenary sessions I was involved in on the training day for markers and a number of the exam setting panel were there, so they have heard about it! However, we are in a dream world if we think the questions for each text had equal difficulty  in the halcyon days of the past when everything was text based.  

Without wishing to act as an apologist for the VCAA exam setters, I think we have to acknowledge that the writing of a paper is very complex and that the way the exam setters think about a text or Context does not always match the way those teaching it perceive it or, on a lesser level, the way your school/you (I!) have taught it.  Exam setters are not allowed to teach Year 12 that year and will be dealing with texts and Contexts that they have not necessarily taught. 

As to the discrepancy rates I don't keep figures in my head very well, but in the past I know that English has had the lowest discrepancy rates of any study (suggesting that English markers as a group know what they are doing) I was under the impression that this year the discrepancy rate was still low (I was told  the figure but I can't drag it from the recesses of my mind) Discrepant scripts are, of course, re-marked to gain consensus. My allotment of discrepancy marking was not huge, but then I am not sure if there were extra people appointed to do this task or just the normal number -- it didn't seem bigger.

I know that a number of people raised most of the central concerns from this discussion in their Assessor's reports and I am sure that the setting panel will reflect on the issues, and I know that a number of the VCAA people will be watching this discussion with intense interest. It is great to have a forum where we can be heard!

Gail

________________________________

From: english-bounces at edulists.com.au on behalf of Malcolm Martin
Sent: Wed 18/02/2009 3:07 PM
To: english at edulists.com.au
Subject: RE: [English] Meet the Assessors

Hello Gail,

Thanks for your observations and I'm glad that examiners were sensitive
to the possibility for mis-reading.  I also take on board your point
that these concerns have little impact on student outcomes, but teachers
and students don't know that that is the case.

Could I ask you about how you felt about the point about the
variability in the Context Questions?  The question for Whose Reality?
seemed to be more complex than the other three and didn't allow equal
access for all the listed texts. 

Your point about discrepancy is a good one.  Do you have any idea as to
the extent of scripts that were discrepant?

Yours sincerely,
Malcolm

Yours sincerely,
>>> reynolds.gail.g at edumail.vic.gov.au 18/02/2009 12:17:57 pm >>>
I wasn't at the meet the assessor's meeting but I would like to put my
two cent's worth into this discussion.

First, it seems to me that the discussion is centred around some
specifics which do not form key strategies in the marking of the actual
paper, but which have been highlighted out of proportion with their
importance.  Yes, there were a number of students who did not understand
the donkey reference, but in the scheme of things this did not impact
disasterously on their overall marks.  The mantra for the marking is
"reward what is there, do not penalise what is not"  Students will have
gained fewer marks for ignoring the cartoon completely, but those who
made a sincere attempt were rewarded for what they did do, regardless of
their degree of recognition of the donkey. I thought the students
generally handled this graphic well and most were able to make
meaningful connections with a range of aspects of the visual.

Secondly from my observation, students handled that Look Both Ways
question well, without showing signs of asking the questions which have
been raised in this discussion. (Surely the important thing is the
questions the students ask?)  The question actually directed students'
thoughts toward the obvious connection between the cartoon
representations and Meryl's thoughts and also the connections between
Nick's flashbacks and his experiences and the question appears to have
empowered the students to focus on the film techniques and the way they
added to our understanding. I would further add that the Assessors are
able to make the connections to the question without the cumbersome "and
so the visual imagery is very helpful in communicating her state of
mind" being a useful inclusion -- in fact only the plodding students
would find this kind of statement necessary.

We are lucky in English as each paper is, at a minimum, double marked
with a question possibly marked up to 5 times to avoid mishaps.
Discrepancy is low.  There are lots of things to dislike about the
process (the statistical fiddling for example) but this angst is not
something I would be sharing at this point.

Gail R

________________________________

From: english-bounces at edulists.com.au on behalf of Malcolm Martin
Sent: Wed 18/02/2009 10:25 AM
To: english at edulists.com.au
Subject: Re: [English] Meet the Assessors

Hello Peter,

Thank you for sharing your observations.  I agree that the exam was
being vigorously defended and the reason for this was because it was
rather flawed.

My concern was the attitude taken towards the 'brilliant' Section
Three.  The Chief Examiner opened by poking fun at students who did
not
recognise the cartoon animal - implying that they were stupid.  I
remember clearly thinking on the day of the exam that there was a
culturally specific set of iconography operating that many students
would miss - the Chief Examiner's observation would seem to bear this
out.  If a student thinks it is warthog then the implication from the
Chief Examiner is that they are badly taught.  I would suggest that
the
scale of 'misreading' of the image shows that the stimulus material
was
flawed.

I (along with many others) also thought that the Context Questions
were
unbalanced and were clealry more suitable to some texts than others.
The presentation from Bob Hillman was worrying from the point of view
that he was keen to identify what characterised a bad response but
failed to articulate what characterised a good response except in the
vaguest terms.  In a way he failed to heed his own advice and work
with
the prompt to explore broad concepts and instead was lost in working
at
a broadly conceptual level.

I hope you have a good year and I appreciate your willingness to share
your thoughts.

Regards,

Malcolm Martin

Head of Senior English
Wesley College
620 High Street Road
Glen Waverley VIC 3150
Ph: (03) 8102 6605

Email: Malcolm.Martin at wesleycollege.net

Head of Senior English
Wesley College
620 High Street Road
Glen Waverley VIC 3150
Ph: (03) 8102 6605

Email: Malcolm.Martin at wesleycollege.net

>>> PeterPidduck at caulfieldgs.vic.edu.au 17/02/2009 3:47:15 pm >>>
Dear All
I am glad we have the opportunity to get feedback from the assessors
in
English, especially as they often raise as many questions as they
answer, which is healthy for the profession. I am particularly curious
about what people made of the advice given by the Meet the Assessor
panel last Thursday for VCE English with regard to some of the text
questions asked by VCAA. When dealing with student responses to the
question 'Does the film-maker's use of visual imagery and setting help
or hinder the viewer's understanding of the concerns of the
characters?', we were told repeatedly how excellent the question was
for
its ability to differentiate between students (actually, I think the
more telling phrase 'discriminate between students' was used), but
surely, even if there seemed to be a categorical determination to
assert
the excellence of the exam, people are right to feel that this
question
uncovers many problems with the expectation of the new course. We were
lectured about the need for students to link the author's language
choices to interpretation as though this was a new rationale for
English, but I don't believe that this is the main thrust of the
criticism to a question like this (the repeated defence of the
question
at least implies that there was criticism).

To ask whether the form helps or hinders with the content is very
difficult, because it assumes problematically that the form can exist
independent of the content (thus the very understanding that
apparently
underpins the question is contradicted in the question). The sample
response flashed all too briefly on the PowerPoint (it seemed from my
too brief perusal) clearly examined the director's use of imagery and
linked it to a defined understanding of the characters' concerns, but
when looking at whether it helped or hindered, it had to be rather
superficial because what can you say? For example, if a student were
to
write something like: 'By using animation to depict Meryl's anxiety
about death, we get an insight into the way she visualises the world
through her Art whilst emphasising the worries are taking place inside
her head, detached from the external world, and so the visual imagery
is
very helpful in communicating her state of mind' , surely the last
clause answers the question, but to what end? The meat is in linking
the
visual imagery to the character's concerns, not in evaluating whether
they help or hinder, although a student would be marked down if he/she
didn't make some sort of comment in this direction. And as such, any
student who had looked at the link between what was in the film with
why
it was there could answer the question, if indeed they were not so
confused by what they had to do with the question in the first place.
And widespread confusion there was (at least, that seems to be many of
the whispers I have heard in relation to many of the same styled
questions). In a situation of widespread confusion, the fault either
lies with a) the student, b) the teacher, c) the exam setter, and
where
I might be wrong to assume c), I felt the speaker at the VCAA Meet the
Assessors strongly, and erroneously, implied a).

My main point is that where it is desirable to have students forge
their
interpretations from an awareness of the language choices an author
has
made, asking questions in this way has the effect of closing down such
an enquiry; indeed, as I formerly suggested, it treats form as though
itr can be considered independently of the interpretation of content.
And regardless whether or not I will find agreement with my
observations, I am a little concerned that any honest criticism of the
exam or exam marking process is construed as griping that is to be met
with a blanket statement of the paper's excellence (I seem to remember
the term 'brilliant' - or a similarly inflated superlative - being
used
to refer to the writing of section 3. There are a lot of words to
describe this composition, but 'brilliant' - I'd need more convincing.
Why did the examination need such active talking-up?).

What do other people think?

Regards
Peter

_______________________________________________
http://www.edulists.com.au <http://www.edulists.com.au/>  <http://www.edulists.com.au/>  - FAQ,
resources, subscribe, unsubscribe
VCE English Teachers' Mailing List kindly supported by
http://www.vate.org.au <http://www.vate.org.au/>  <http://www.vate.org.au/>  - Victorian
Association for the Teaching of
English VATE and
http://www.vcaa.vic.edu.au/vce/studies/english/index.html - Victorian
Curriculum and Assessment Authority
____________________________________________________________________________

Sapere Aude - Dare To Be Wise

Wesley College Melbourne is a world class coeducational independent
school
developing the whole person through timeless principles of learning:
- to know
- to do
- to live with
- to be
with innovation and wisdom

ABN 38 994 068 473  CRICOS 00354G
____________________________________________________________________________

This email is intended only for the use of the individual or entity
named
above and may contain information that is confidential and privileged.
If
you are not the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this email is strictly
prohibited.
If you have received this email in error, please email a reply to
Wesley
College and destroy the original message.

_______________________________________________
http://www.edulists.com.au <http://www.edulists.com.au/>  <http://www.edulists.com.au/>  - FAQ,
resources, subscribe, unsubscribe
VCE English Teachers' Mailing List kindly supported by
http://www.vate.org.au <http://www.vate.org.au/>  <http://www.vate.org.au/>  - Victorian
Association for the Teaching of English VATE and
http://www.vcaa.vic.edu.au/vce/studies/english/index.html - Victorian
Curriculum and Assessment Authority

Important - This email and any attachments may be confidential. If
received in error, please contact us and delete all copies. Before
opening or using attachments check them for viruses and defects.
Regardless of any loss, damage or consequence, whether caused by the
negligence of the sender or not, resulting directly or indirectly from
the use of any attached files our liability is limited to resupplying
any affected attachments. Any representations or opinions expressed are
those of the individual sender, and not necessarily those of the
Department of Education and Early Childhood Development.
_______________________________________________
http://www.edulists.com.au <http://www.edulists.com.au/>  - FAQ, resources, subscribe, unsubscribe
VCE English Teachers' Mailing List kindly supported by
http://www.vate.org.au <http://www.vate.org.au/>  - Victorian Association for the Teaching of
English VATE and
http://www.vcaa.vic.edu.au/vce/studies/english/index.html - Victorian
Curriculum and Assessment Authority
____________________________________________________________________________

Sapere Aude - Dare To Be Wise

Wesley College Melbourne is a world class coeducational independent school
developing the whole person through timeless principles of learning:
- to know
- to do
- to live with
- to be
with innovation and wisdom

ABN 38 994 068 473  CRICOS 00354G
____________________________________________________________________________

This email is intended only for the use of the individual or entity named
above and may contain information that is confidential and privileged. If
you are not the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this email is strictly prohibited.
If you have received this email in error, please email a reply to Wesley
College and destroy the original message.

_______________________________________________
http://www.edulists.com.au <http://www.edulists.com.au/>  - FAQ, resources, subscribe, unsubscribe
VCE English Teachers' Mailing List kindly supported by
http://www.vate.org.au <http://www.vate.org.au/>  - Victorian Association for the Teaching of English VATE and
http://www.vcaa.vic.edu.au/vce/studies/english/index.html - Victorian Curriculum and Assessment Authority

Important - This email and any attachments may be confidential. If received in error, please contact us and delete all copies. Before opening or using attachments check them for viruses and defects. Regardless of any loss, damage or consequence, whether caused by the negligence of the sender or not, resulting directly or indirectly from the use of any attached files our liability is limited to resupplying any affected attachments. Any representations or opinions expressed are those of the individual sender, and not necessarily those of the Department of Education and Early Childhood Development.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/ms-tnef
Size: 14577 bytes
Desc: not available
Url : http://www.edulists.com.au/pipermail/english/attachments/20090218/5fd73fdb/attachment-0001.bin