|
In the National Investigators' Examination
(NIE), candidates will have to answer 80 multiple-choice questions.
However, only 70 of those questions will count towards a final
mark. This is because 10 of the questions in the examination are
validation questions.
Validation questions are 'test' questions
inserted at random points within the examination paper. Effectively,
they are included in an examination as a test of the question
itself (and not of the candidates' law knowledge) to establish
if the actual question is good enough to be included in a future
examination. Another way of thinking about these questions is
that they represent a form of quality assurance for questions.
If, after validation, the question matches set statistical criteria
then it will be introduced into a future examination as a 'real'
question and will count as one of the 70 questions that make up
a candidates' final score.
Many candidates criticise the inclusion
of validation questions and, on the face of it, it is easy to
understand why as these 'test' questions can have a detrimental
effect on candidate performance. For example:
|
Let us
imagine that question 24 in one of the 2010 NIEs is a verification
question. Of course, this fact is not known to a candidate.
The question is problematic and somewhat lengthy and because
of these factors a candidate gets into difficulty with the
question and it takes them 5 minutes to formulate an answer.
The drawback is that with 80 questions in an examination,
spending five minutes on one question is going to have a detrimental
effect on time management. It is also highly likely to dent
the candidates' confidence after having spent so long on a
question. And all of that effort and time has been wasted
as the question is a verification question and will not count
towards the candidates' eventual mark.
|
Considering this scenario, it is reasonable
that candidates may strongly feel that verifications questions
should not be placed in an examination. However, there is a far
weightier counter-argument in favour of the inclusion of verification
questions in the examination.
How would a candidate feel if they revised
over a period of three months, spent a considerable amount of
time, effort and money in the process and then took a 80 question
examination where the questions had just been thrown together
because they were relevant to the syllabus but had never been
strictly analysed to make sure they were fair and then, as a result,
that candidate failed the examination? Can you imagine the amount
of thoroughly justified and warranted complaints this would create?
This would mean that a candidate fails the examination because
of the large number of poor quality questions and not a lack of
knowledge and/or understanding of the law. This danger far outweighs
the problems that answering verification questions in the examination
creates for an individual candidate.
One suggestion I have heard is that as
the NPIA examination question bank must contain hundreds if not
thousands of multiple-choice questions, the need to verify new
questions is no longer required. Such an approach could be taken
but it would mean that the examination question bank would remain
static and be unable to ask questions on new law as it was introduced.
In addition, as 'old' law fell away, so would the multiple-choice
questions on that 'old' law in the examination and the NPIA question
bank would offer less and less questions for examiners to choose
from. Consequently, within a short period of time the examination
would become stale and verification questions would have to be
re-introduced.
A further point that a candidate may
make is to ask why verification questions cannot be verified somewhere
else and not within the NIE? This is because the only place that
accurate feedback on multiple-choice questions can be obtained
is in an actual examination. Although NPIA run such large scale
'mock' exams this will not stop the continuing verification process
in an 80 question examination. Placing verification questions
in the examination enables realistic and large-scale feedback
to be obtained to ensure that the question can legitimately be
introduced at a later stage having undergone a strict testing
regime.
In conclusion, candidates would be far
worse off if verification questions were not included as part
of the examination process. They are an absolute necessity to
ensure that questions are not only current but also fair (please
see the 'Myth, Rumour and Barrack-room Advice'
section for further information on verification questions).
|
