Recruitment into specialty training in the UK
Authors: Bill Irish, Alison Carr, David Sowden, Neil Douglas, Fiona Patterson
Publication date: 12 Jan 2011
Bill Irish and colleagues outline how national and specialty specific recruitment to specialty training has developed since the introduction of the Medical Training Application Service
The recruitment process for specialty training in the United Kingdom continues to attract considerable attention and is an emotive issue. There has been substantial investment in developing processes that are robust, demonstrably efficient, transparent, and fair. Now is an opportune moment to review progress since the introduction of the Medical Training Application Service in 2007. Here we compare the process against international literature on selection in other professions and emerging evidence in the United Kingdom.
From the “good old days” to “specialty tailored assessments”
Many doctors still fondly reminisce about the “good old days” of local recruitment. Typically such processes involved shortlisting numerous unstructured CVs and undertaking panel interviews, often with large numbers of interviewers. Such methods were discernibly unreliable, with the huge variation in person specifications confusing candidates and encouraging patronage.  The inevitable variation in standards had the potential for unfair discrimination and legal challenge. Multiple applications were usual, with candidates assessed many times in various locations in the UK. Numerous applications were received for each post, often over 2000, making rational appointments impossible. Consequently, in 2006 a national approach to selection was developed, which used person specifications agreed with the royal colleges.
In 2007, Tooke recommended the move to “specialty tailored assessments at selection centres,” and each specialty began incremental development of its recruitment processes. However, although UK medical school output and trainee requirements are generally matched, a great excess of doctors wish to train in some specialties and many strong applications come from doctors who trained elsewhere. There is high competition for training posts in most specialties, and distinguishing reliably between highly qualified candidates is a major challenge.
Person specifications and selection centres
The aim of the new process is to select doctors who will succeed in training and excel in subsequent practice, through aptitude in the relevant specialty. Specialty specific person specifications are crucial to this process. Although commonality exists between specialties, job analysis studies show different priorities between specialties in the personal qualities sought. Medical school performance alone does not reliably predict success in postgraduate medical education.  It cannot be assumed that those with high academic ability alone will become good specialists and generalists, because other qualities are needed.
Large scale meta-analyses in non-medical fields consistently show strong predictive validity for selection methods such as situational judgment tests and selection centres for job performance, training success, and career progression.   For example, situational judgment tests show good predictive validity among managers, civil service employees, and supervisors in the armed forces, and these tests have incremental validity over cognitive ability tests, experience, and personality.  
Selection centres use exercises to sample across a breadth of criteria from person specifications, using a multi-trait multi-method approach. Selection centres originated before the second world war to assess military officers, and they are still used by the British civil service selection board. Selection centres show validity over other selection methods and have been used in selecting police officers, lawyers, and pilots.
Specialty specific approaches
General practice led the way to developing national selection for entry into training. Recruitment processes were developed over several years to provide a fair and transparent system with equal opportunities for all applicants. This incremental progression, underpinned by an analysis of the general practitioner role, was a key factor in its successful implementation. Selection focuses on candidates demonstrating aptitude through machine marked tests and a selection centre, rather than reporting their competences at interview.
Other specialties have piloted machine marked tests for selection; tests for core medical training, surgery, and acute specialties show promising predictive validity. Histopathology was the first specialty to implement national selection. A selection centre for anaesthesia has been validated, which shows good prediction of performance in the first three years of appointment. Obstetrics and gynaecology, paediatrics, and psychiatry are moving in this direction, and most specialties will have nationally coordinated recruitment within 12 months.
Since recruitment for specialties became nationally coordinated, the overall number of applications has fallen from 128 000 in 2007 to around 40 000 in 2010. This has vastly reduced the time spent processing repeated applications by clinicians and administrative staff, thus reducing applications to numbers appropriate for careful evaluation. Costing the time of senior clinicians who are involved in selecting trainees is difficult. Cancelling clinical work has a clear opportunity cost, and coercing consultants to shortlist at evenings and weekends is not ideal. Using machine marked tests, instead of hand scored applications, reduces costs considerably. Robust selection also reduces expenditure on trainee remediation, resulting in substantial cost saving. In general practice such costs are overt, at £40 000 per doctor for a typical six month extension to training.
Towards a nationally coordinated appointments’ system
Candidates’ perspectives in selection are crucial, especially regarding perceptions of fairness and personal treatment in selection.      Candidates prefer selection methods that are clearly related to the person specification and provide multiple opportunities to demonstrate aptitude, as in selection centres. Candidate feedback on nationally coordinated specialty recruitment is good, and improving year on year.
Specialty selection has made great strides since 2006 through careful, incremental development of nationally agreed selection methods, international best practice and evaluation, and a strong professional contribution to the processes. Emerging data show good predictive validity, cost effectiveness, and candidate acceptability of the process, with scope for further improvements. Continued convergence of methodology around evidence based best practice is essential, and most specialties are adopting common approaches to selecting applicants to specialty training. Both the royal colleges and the postgraduate deans seek a nationally coordinated appointments’ system that incorporates such individual national specialty recruitment processes.
Competing interests: BI, AC, DS, ND, and FP have received support for the submitted work from the GP National Recruitment Office, Department of Health, Conference of Post Graduate Medical Deaneries, Academy of Medical Royal Colleges, and Cambridge University; and FP is founding director of the work psychology group that advises the Department of Health and a number of other medical specialties on issues relating to recruitment and workplace psychology.
- Plint S, Patterson F. Identifying critical success factors for designing selection processes into postgraduate specialty training: the case of UK general practice. Postgrad Med J 2010;86:323-7.
- Field S. Vocational training; the dawn of a new era? Education for General Practice 2000;11:38.
- Highhouse S. Stubborn reliance on intuition and subjectivity in employee selection. Industrial and Organizational Psychology: Perspectives on Science and Practice 2008;1:333-42.
- Tooke J. Aspiring to excellence. Findings and recommendations of the independent inquiry into Modernising Medical Careers. MMC Inquiry, 2007.
- Patterson F, Ferguson E, Thomas S. Using job analysis to identify core and specific selection criteria for three secondary care specialties. Med Educ 2008;42:1195-204.
- Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ 2002;324:952-7.
- McManus IC, Partridge P, Keeling A, Fleming PR. A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study. BMJ 2003;327:139-42.
- Patterson F, Ferguson E. Selection into medical education and training. Edinburgh: Association for the Study of Medical Education, 2007.
- Schmidt FL, Hunter JE. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychol Bull 1998;124:262-74.
- Robertson IT, Smith M. Personnel selection. Journal of Occupational and Organizational Psychology 2001;74:441-72.
- McDaniel MA, Morgeson FP, Finnegan EB, Campion MA, Braverman EP. Predicting job performance using situational judgment tests: A clarification of the literature. J Appl Psychol 2001;86:730–40.
- Campbell JP, Dunnette MD, Lawler EE III. Weick Research results: Actuarial studies of managerial effectiveness, managerial behavior, performance, and effectiveness. McGraw-Hill, 1970.
- Chan D, Schmitt N. Situational judgment and job performance. Human Performance 2002;15:233–54.
- Borman WC, Hanson MA, Oppler SH, Pulakos ED, White LA. Role of early supervisory experience in supervisor performance. J Appl Psychol 1993;78:443-9.
- Clevenger J, Jockin T, Morris S, Anselmi T. A situational judgment test for engineers: Construct and criterion related validity of a less adverse alternative. Paper presented at the 14th annual conference of the SIOP, Atlanta, 2001.
- O’Connell MS, Hartman NS, McDaniel MA, Grubb WL III, Lawrence A. Incremental validity of situational judgment tests for task and contextual job performance. International Journal of Selection and Assessment 2007;15:19-29.
- Lievens F, Harris MM, Van Keer E, Bisqueret C. Predicting cross-cultural training performance: The validity of personality, cognitive ability, and dimensions measured by an assessment center and a behavior description interview. J Appl Psychol 2003;88:476-89.
- Vernon PE, Parry JB. Personnel selection in the British forces . London: University of London, 1949.
- Spychalski AC, Quinones MA, Gaugler BB, Pohley K. A survey of assessment center practices in organizations in the United States. Personnel Psychology 1997;50:71-90.
- Dayan K, Kasten R, Fox S. Entry-level police candidate assessment center, an efficient tool or a hammer to kill a fly? Personnel Psychology 2002;55:827-49.
- Sackett PR. Performance assessment in education and professional certification: Lessons for personnel selection? In: Hakel MD, ed. Beyond multiple choice tests. Mahwah: Lawrence Erlbaum, 1998: 113-29.
- Damitz M, Manzey D, Kleinmann M, Severin K. Assessment center for pilot selection: Construct and criterion validity and the impact of assessor type. Appl Psych, An Int Rev 2003;52:193-212.
- Patterson F, Ferguson E, Norfolk T, Lane P. A new selection system to recruit general practice registrars: preliminary findings from a validation study. BMJ 2005;330:711-4.
- Patterson F, Carr V, Zibarras L, Burr B, Berkin L, Plint S, et al. New machine-marked tests for selection into core medical training: evidence from two validation studies. Clin Med 2009;9:417-20.
- Gale T, Roberts M, Sice P, Langton J, Patterson F, Carr A, et al. Predictive validity of a new selection centre testing non-technical skills for recruitment to training in anaesthesia. Br J Anaesth 2010;105(5):603-9.
- Crawford ME. Reassuring evidence on competency based selection. BMJ 2005;330:711-4.
- Kumar K, Roberts C, Rothnie I, du Fresne C, Walton M. Experiences of the multiple mini-interview: A qualitative analysis. Med Educ 2009;43:360-7.
- Humphrey S, Dowson S, Wall D, Diwakar V, Goodyear HM. Multiple mini-interviews: Opinions of candidates and interviewers. Med Educ 2008;42:207-13.
- Patterson F, Zibarras L, Carr V, Irish B, Gregory S. Evaluating procedural justice in postgraduate medical selection. Med Educ [forthcoming].
- Gilliland SW. The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review 1993;18:694-734.
- Gilliland SW. Effects of procedural and distributive justice on reactions to a selection system. J Appl Psychol 1994;79:691-701.
- Gilliland SW. Fairness from the applicant’s perspective: Reactions to employee selection procedures. International Journal of Selection and Assessment 1995;3:11-18.
Bill Irish chairman, General Practice National Recruitment Office
Severn Deanery, Bristol, UK
Alison Carr senior clinical adviser, Medical Education and Training Programme (England) Department of Health, London, UK
David Sowden chairman, Conference of Postgraduate Medical Deans East Midlands Deanery, University of Nottingham, Nottingham, UK
Neil Douglas president, Academy of Medical Royal Colleges London
Fiona Patterson professor Department of Social and Developmental Psychology, Cambridge University, Cambridge, UK