Marking system

A closed marking system is used from 4 to 8 and this equates to the following:

 4 – Poor fail.

 5 – Fail.

 6 – Pass.

 7 – Good pass.

 8 – Exceptional pass.

Examiners assess nine trainee characteristics during the standardized oral examination.

1.     Personal qualities.

2.     Communication skills.

3.     Professionalism.

4.     Surgical experience.

5.     Organizational and logical, step-wise sequencing of thought processes, ability to focus on the answers quickly.

6.     Clinical reasoning and decision making.

7.     Ability to handle stress.

8.     Ability to deal with grey areas in practice and complex issues.

9.     Ability to justify an answer with evidence from the literature.

This list has been simplified into three domains.

• Overall professional capability/patient care  Personal qualities, professionalism and ethics, surgical experience, ability to deal with grey areas.

• Knowledge and judgement

•  Knowledge, ability to justify, clinical reasoning.

• Quality of response

•  Communication skills, organization and logical thought process. Assess questions, answers and prompting (QAP).

Marking in detail

4            – Unsafe and potentially dangerous. A very poor answer. Gross basic mistakes and poor knowledge. Should not be sitting the exam. The examiners have severe reservations about the candidate’s performance and are essentially flagging this up. Too ignorant of the fundamentals of orthopaedic practice to pass. Difficult to salvage even if other marks are 7 or 8, which is probably unlikely if the candidate is scoring a 4 in the first instance.

Did not get beyond the default questions, fails in all/most competencies. Poor basic knowledge/judgement/understanding to a level of concern.

5            – Some hesitancy and indecisiveness. The answer is really not good enough with too many deficiencies. Too many basic errors and not getting to the nub of the issue. Wandering off at tangents and not staying focused on the question. Misinterpreting the question, wrong examination advice for tactics.

The same ATLS and/or radiograph talk with each oral viva question.

Difficulty in prioritising, large gaps in knowledge, poor deductive skills, patchy performance, struggled to apply knowledge and judgement. Confused or disorganized answer. Poor higher-order thinking.

6            – Satisfactory performance. Covered the basics well, safe and would be a sound consultant. No concerns. Performance acceptable but not anything special or outstanding.

Good knowledge and judgement of common problems. Important points mentioned, no major errors and requires only minor prompting.

7            – Good performance. Would make a good consultant. Articulate and to the point. Able to quote papers, knows various guidelines and publications.

Coped well with difficult topics/problems. Goes beyond the competency questions. Logical answers. Strong interpretation/judgement but wasn’t able to quote or use the literature effectively. Good supporting reasons for answers.

No prompting needed for answers but prompting required for the literature.

8            – Potential gold medal or prize-winning performance. Smooth, articulate and polished. Able to succinctly discuss controversial orthopaedic issues in a sensible way. Excellent command of the literature. Switched on and makes the examiners feel very reassured. Looks and talks the part.

Stretches the examiners, no prompting necessary. Confident, clear, logical and focused answers.

The marking system should allow exceptional candidates to be identified and should in theory allow feedback to be given to unsuccessful candidates.

The two examiners give separate independent marks without conferring with each other.

The marking system may be reviewed in the future and one suggestion is to reduce the choice to poor fail, fail and pass in an attempt to reduce potential bias and variability. Any change to the marking system will throw up a number of conflicting issues and opinions and may not necessarily improve on the current method.

Within a 2-hour period (120 minutes) eight examiners can independently assess each trainee on a total of 24 topics. This generates 48 test scores, which should provide a reliable and valid measure of a candidate’s ability in terms of the educational domains being assessed, namely professionalism/patient care, knowledge and judgement and quality of response.