Topics for the structured oral exam

Ideally topics should be asked that cannot be assessed in a hands-on setting at the clinical exams, e.g. trauma emergencies, critical condition and acute illnesses. Some topics such as avascular necrosis (AVN) find their way into both parts of the exam so this distinction isn’t particularly clear-cut at times. The clinical scenario should be realistic and be able to generate enough questions. The scenario should be neither too long with too much information nor too short with insufficient data.

Lines of questions should easily be developed for the ‘introduction’, ‘competence’ and ‘advanced’ question categories.

Examiners have identified appropriate acceptable and unacceptable answers to the questions.

OralvivacoursesfortheFRCS(Tr&Orth)

There are many viva teaching courses to choose from. Some are well established with excellent candidate feedback whilst others are less well known and illustrious. Oral practice courses are less difficult to organize than clinical courses and this explains the wider choice available.

The sensible option is to ask the advice of several local trainees who have recently passed the exam as to which one they would recommend. Another useful source is the regional training programme websites that usually have an area in which candidates are encouraged to provide feedback from the various courses they have attended. This should give you some idea of which courses are worth going to or avoiding. This material may be restricted unless you are a local trainee.

What is the evidence? Do I need to know papers?

Yes, you do. We are not convinced when we hear people say ‘you do not need to quote the literature’. Looking good by quoting the latest journal articles is impressive but not to your examiner if you are quoting papers inappropriately within an answer.

You would need to know the seminal papers on different subject areas within the last few years. There is a subtle difference between quoting journal articles to support four different ways to manage a tibial plateau fracture or saying ‘This is what I would do ...’ If the examiners ask ‘Why?’ you can then quote the literature.

If you want to score an 8 the examiners would expect that not only should you have an excellent command of the literature but be able to use the literature to justify and support your management decisions.

To score a 7 a candidate needs to be familiar with the literature and be able to quote papers but perhaps is not quite as expert using it to support arguments or justify management decisions. The examiners may need to occasionally prompt and help out the candidate with the current literature.

Scoring 6 suggests a candidate would probably know the seminal papers but struggle to get further beyond this. With good knowledge and judgement and the important points mentioned a candidate may score a safe 6 without knowing any significant literature to back up the evidence.

A score of 5 suggests the literature doesn’t really matter as you are struggling to keep your head above the water and are trying to get past the default questions onto the competency questions.

Scoring a 4 means you will have time to read the various key orthopaedic papers before you re-sit the examination.

Educational value of the structured, standardized oral exam

The oral examination questions are ideally sourced on patient care (i.e. clinical scenarios), designed to promote higher-order thinking (i.e. use of knowledge for decision making, interpretation, clinical judgement) and centred on a trainee’s quality of answer (quality, focus, confidence displayed when answering and amount of prompting required).

Advantages of the oral exam include:  It is a face-to-face exam. It can therefore be used to assess aspects of trainees that other exams may fail to assess such as quality of responses.

 It is a flexible exam. The examiner can choose from a pool of predetermined questions to ask an easier or more difficult question, depending on the candidate’s response to the earlier question.

 Oral exams can be used to assess the candidate’s

cognitive abilities related to clinical practice, such as problem solving and decision making.

 Oral exams may capture certain important examinee traits which other exams may fail to assess; e.g. fitness-to-practise, worthiness for recognition as senior clinicians, professionalism.

	Disadvantages of the oral exam include:
	Meticulous planning is required to ensure the exam is structured according to the examination blueprint.
	Oral exams require a large number of examiners to maintain reliability.
	The examiners should be pre-trained to apply the same standards to each candidate using prevalidated rating scale descriptors.
	The organization and administration of an oral exam is costly and time consuming.
	It has been shown that oral exams may bias against some candidates, e.g. certain ethnic groups.
	Oral exams tend to assess certain candidate attributes which are not intended to be assessed, e.g. examinee style of speaking.
	Oral exams can feel threatening and stressful to the candidate.

Although it is outside of the scope of the book to discuss in detail educational principles behind assessment several theories do warrant a brief mention.

Miller’s pyramid of assessment stresses that four levels of assessment need to be tested to obtain a comprehensive understanding of a trainee’s ability (Table 1.1).

Bloom’s taxonomy categorizes knowledge into six levels:

1.     Knowledge recall.

2.     Comprehension or understanding.

3.     Application.

4.     Analysis.

5.     Synthesis.

6.     Evaluation.

This is a hierachical classification with the lowest cognitive level being ‘knowledge recall’ and the highest ‘evaluation of knowledge’ (Table 1.2). The lower levels can be attained with a superficial approach to learning with memorizing lists and rote learning of facts but the upper levels involve higherorder thinking and can only be attained by deep learning.

The assessment of recall and comprehension of knowledge is essential, but if only recall and comprehension are tested, lower-order thinking will be promoted. In contrast, higher-order thinking is encouraged by assessing the knowledge at application,

Table 1.1 Miller’s pyramid of assessment. Knows Knowledge or information that the candidate has learned SBAs Knows how Application of knowledge to medically relevant situations EMIs Shows how Simulated demonstration of skills in an examination situation Intermediate case, short case Does Behaviour in real-life situations   Table 1.2 Bloom’s taxonomy. Level Learner action Question cues Knowledge Recall content in the exact form that it was presented. Memorization of definitions, formulas, or procedures gives examples of knowledge-level functioning List, define, label, identify, name Comprehension Restate material in their own words, or can recognize previously unseen examples of a concept Describe, associate, categorize, summarize Application Apply rules to a problem, without being given the rule or formula for solving the problem Apply, calculate, illustrate, solve Analysis Break complex concepts or situations down into their component parts, and analyse how the parts are related to one another Analyse, compare, separate, order, explain Synthesis Rearrange component parts to form a new whole Combine, modify, rearrange, ‘what-if’ Evaluation Evaluate or make judgements on the worth of a concept, object, etc. for a purpose Assess, decide, grade, recommend, explain, judge

analysis, synthesis and evaluation levels. Context-free questions, i.e. questions that are not based on practical/clinical scenarios, encourage the consideration of simple answers, e.g. yes/no.

The promotion and assessment of higher-order thinking can be achieved by introducing context-rich questions, i.e. questions based on patient, practical or clinical scenarios, for knowledge assessments. The six levels have been telescoped to:

A.   Recall–comprehension of knowledge; i.e.

reproducing and understanding.

B.    Application–analysis; i.e. making use of knowledge.

C.    Synthesis–evaluation; i.e. doing different things with knowledge and making use of judgement.

The Van der Vleuten utility index formula is used to analyse the usefulness of different assessment methods. It is defined as the product

Unity ¼ R  V  E  C  A

Where

R = Reliability

For high reliability guidelines need to be rigidly followed by the examiners.

V = Validity

Clinical scenarios should be drawn from the assessment blueprint and biased on higher-order thinking.

E = Educational impact

If pre-validated questions probing higher-order thinking skills are used, the oral exam can focus the trainees on sound clinical practice.

C = Cost moderate

Running an oral examination is expensive to organize, requiring examiner time and infrastructure, but is less expensive than clinical exams involving patients.

A = Acceptability

Examiners may dislike the loss of autonomy that resultsfromtheimposedstructureintheoralexam.High stress levels have been reported amongst candidates.

Additionally, P = Practicability

Some difficulties but generally much easier to organize than the clinical exams.

Higher-order thinking involves the learning of complex judgemental skills such as critical thinking and problem solving and is more difficult to learn but is also more valuable because such skills are more likely to be useable in everyday clinical practice. The simplest thinking skills are learning facts and recall, while higher-order skills include critical thinking, analysis and problem solving.

With the increasingly complex nature of orthopaedic practice, it is becoming even more important that orthopaedic surgeons are capable of thinking divergently and creatively.

Endnote

1. We believe this to be the case but the only thing consistent about the exam regulations is that they change from year to year.