Recruitment into specialty training in the UK

Authors: Bill Irish, Alison Carr, David Sowden, Neil Douglas, Fiona Patterson 

Publication date:  12 Jan 2011


Bill Irish and colleagues outline how national and specialty specific recruitment to specialty training has developed since the introduction of the Medical Training Application Service

The recruitment process for specialty training in the United Kingdom continues to attract considerable attention and is an emotive issue. There has been substantial investment in developing processes that are robust, demonstrably efficient, transparent, and fair. Now is an opportune moment to review progress since the introduction of the Medical Training Application Service in 2007. Here we compare the process against international literature on selection in other professions and emerging evidence in the United Kingdom.

From the “good old days” to “specialty tailored assessments”

Many doctors still fondly reminisce about the “good old days” of local recruitment. Typically such processes involved shortlisting numerous unstructured CVs and undertaking panel interviews, often with large numbers of interviewers. Such methods were discernibly unreliable, with the huge variation in person specifications confusing candidates and encouraging patronage.[1] [2] The inevitable variation in standards had the potential for unfair discrimination and legal challenge.[3] Multiple applications were usual, with candidates assessed many times in various locations in the UK.[1] Numerous applications were received for each post, often over 2000, making rational appointments impossible. Consequently, in 2006 a national approach to selection was developed, which used person specifications agreed with the royal colleges.

In 2007, Tooke recommended the move to “specialty tailored assessments at selection centres,”[4] and each specialty began incremental development of its recruitment processes. However, although UK medical school output and trainee requirements are generally matched, a great excess of doctors wish to train in some specialties and many strong applications come from doctors who trained elsewhere. There is high competition for training posts in most specialties, and distinguishing reliably between highly qualified candidates is a major challenge.

Person specifications and selection centres

The aim of the new process is to select doctors who will succeed in training and excel in subsequent practice, through aptitude in the relevant specialty. Specialty specific person specifications are crucial to this process. Although commonality exists between specialties, job analysis studies show different priorities between specialties in the personal qualities sought.[5] Medical school performance alone does not reliably predict success in postgraduate medical education.[6] [7] It cannot be assumed that those with high academic ability alone will become good specialists and generalists, because other qualities are needed.[8]

Large scale meta-analyses in non-medical fields consistently show strong predictive validity for selection methods such as situational judgment tests and selection centres for job performance, training success, and career progression.[9] [10] [11] For example, situational judgment tests show good predictive validity among managers,[12] civil service employees,[13] and supervisors in the armed forces,[14] and these tests have incremental validity over cognitive ability tests, experience, and personality.[13] [15] [16]

Selection centres use exercises to sample across a breadth of criteria from person specifications, using a multi-trait multi-method approach.[17] Selection centres originated before the second world war to assess military officers,[18] and they are still used by the British civil service selection board. Selection centres show validity over other selection methods[19] and have been used in selecting police officers,[20] lawyers,[21] and pilots.[22]

Specialty specific approaches

General practice led the way to developing national selection for entry into training. Recruitment processes were developed over several years to provide a fair and transparent system with equal opportunities for all applicants. This incremental progression, underpinned by an analysis of the general practitioner role, was a key factor in its successful implementation.[1] Selection focuses on candidates demonstrating aptitude through machine marked tests and a selection centre, rather than reporting their competences at interview.[23]

Other specialties have piloted machine marked tests for selection; tests for core medical training, surgery, and acute specialties show promising predictive validity.[24] Histopathology was the first specialty to implement national selection. A selection centre for anaesthesia has been validated, which shows good prediction of performance in the first three years of appointment.[25] Obstetrics and gynaecology, paediatrics, and psychiatry are moving in this direction, and most specialties will have nationally coordinated recruitment within 12 months.

Saving costs

Since recruitment for specialties became nationally coordinated, the overall number of applications has fallen from 128 000 in 2007 to around 40 000 in 2010. This has vastly reduced the time spent processing repeated applications by clinicians and administrative staff, thus reducing applications to numbers appropriate for careful evaluation. Costing the time of senior clinicians who are involved in selecting trainees is difficult. Cancelling clinical work has a clear opportunity cost, and coercing consultants to shortlist at evenings and weekends is not ideal. Using machine marked tests, instead of hand scored applications, reduces costs considerably. Robust selection also reduces expenditure on trainee remediation, resulting in substantial cost saving. In general practice such costs are overt, at £40 000 per doctor for a typical six month extension to training.[26]

Towards a nationally coordinated appointments’ system

Candidates’ perspectives in selection are crucial, especially regarding perceptions of fairness and personal treatment in selection.[27] [28] [29] [30] [31] [32] Candidates prefer selection methods that are clearly related to the person specification and provide multiple opportunities to demonstrate aptitude, as in selection centres.[30] Candidate feedback on nationally coordinated specialty recruitment is good, and improving year on year.

Specialty selection has made great strides since 2006 through careful, incremental development of nationally agreed selection methods, international best practice and evaluation, and a strong professional contribution to the processes. Emerging data show good predictive validity, cost effectiveness, and candidate acceptability of the process, with scope for further improvements. Continued convergence of methodology around evidence based best practice is essential, and most specialties are adopting common approaches to selecting applicants to specialty training. Both the royal colleges and the postgraduate deans seek a nationally coordinated appointments’ system that incorporates such individual national specialty recruitment processes.

Competing interests: BI, AC, DS, ND, and FP have received support for the submitted work from the GP National Recruitment Office, Department of Health, Conference of Post Graduate Medical Deaneries, Academy of Medical Royal Colleges, and Cambridge University; and FP is founding director of the work psychology group that advises the Department of Health and a number of other medical specialties on issues relating to recruitment and workplace psychology.

References

  1. Plint S, Patterson F. Identifying critical success factors for designing selection processes into postgraduate specialty training: the case of UK general practice. Postgrad Med J   2010;86:323-7.
  2. Field S. Vocational training; the dawn of a new era? Education for General Practice   2000;11:38.
  3. Highhouse S. Stubborn reliance on intuition and subjectivity in employee selection. Industrial and Organizational Psychology: Perspectives on Science and Practice  2008;1:333-42.
  4. Tooke J. Aspiring to excellence. Findings and recommendations of the independent inquiry into Modernising Medical Careers. MMC Inquiry, 2007.
  5. Patterson F, Ferguson E, Thomas S. Using job analysis to identify core and specific selection criteria for three secondary care specialties. Med Educ   2008;42:1195-204.
  6. Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ   2002;324:952-7.
  7. McManus IC, Partridge P, Keeling A, Fleming PR. A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study. BMJ   2003;327:139-42.
  8. Patterson F, Ferguson E. Selection into medical education and training. Edinburgh: Association for the Study of Medical Education, 2007.
  9. Schmidt FL, Hunter JE. The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychol Bull   1998;124:262-74.
  10. Robertson IT, Smith M. Personnel selection. Journal of Occupational and Organizational Psychology  2001;74:441-72.
  11. McDaniel MA, Morgeson FP, Finnegan EB, Campion MA, Braverman EP. Predicting job performance using situational judgment tests: A clarification of the literature. J Appl Psychol  2001;86:730–40.
  12. Campbell JP, Dunnette MD, Lawler EE III. Weick Research results: Actuarial studies of managerial effectiveness, managerial behavior, performance, and effectiveness. McGraw-Hill, 1970.
  13. Chan D, Schmitt N. Situational judgment and job performance. Human Performance  2002;15:233–54.
  14. Borman WC, Hanson MA, Oppler SH, Pulakos ED, White LA. Role of early supervisory experience in supervisor performance. J Appl Psychol   1993;78:443-9.
  15. Clevenger J, Jockin T, Morris S, Anselmi T. A situational judgment test for engineers: Construct and criterion related validity of a less adverse alternative. Paper presented at the 14th annual conference of the SIOP, Atlanta, 2001.
  16. O’Connell MS, Hartman NS, McDaniel MA, Grubb WL III, Lawrence A. Incremental validity of situational judgment tests for task and contextual job performance. International Journal of Selection and Assessment  2007;15:19-29.
  17. Lievens F, Harris MM, Van Keer E, Bisqueret C. Predicting cross-cultural training performance: The validity of personality, cognitive ability, and dimensions measured by an assessment center and a behavior description interview. J Appl Psychol   2003;88:476-89.
  18. Vernon PE, Parry JB. Personnel selection in the British forces  . London: University of London, 1949.
  19. Spychalski AC, Quinones MA, Gaugler BB, Pohley K. A survey of assessment center practices in organizations in the United States. Personnel Psychology  1997;50:71-90.
  20. Dayan K, Kasten R, Fox S. Entry-level police candidate assessment center, an efficient tool or a hammer to kill a fly? Personnel Psychology  2002;55:827-49.
  21. Sackett PR. Performance assessment in education and professional certification: Lessons for personnel selection? In: Hakel MD, ed. Beyond multiple choice tests.  Mahwah: Lawrence Erlbaum, 1998: 113-29.
  22. Damitz M, Manzey D, Kleinmann M, Severin K. Assessment center for pilot selection: Construct and criterion validity and the impact of assessor type. Appl Psych, An Int Rev  2003;52:193-212.
  23. Patterson F, Ferguson E, Norfolk T, Lane P. A new selection system to recruit general practice registrars: preliminary findings from a validation study. BMJ  2005;330:711-4.
  24. Patterson F, Carr V, Zibarras L, Burr B, Berkin L, Plint S, et al. New machine-marked tests for selection into core medical training: evidence from two validation studies. Clin Med   2009;9:417-20.
  25. Gale T, Roberts M, Sice P, Langton J, Patterson F, Carr A, et al. Predictive validity of a new selection centre testing non-technical skills for recruitment to training in anaesthesia. Br J Anaesth  2010;105(5):603-9.
  26. Crawford ME. Reassuring evidence on competency based selection. BMJ  2005;330:711-4.
  27. Kumar K, Roberts C, Rothnie I, du Fresne C, Walton M. Experiences of the multiple mini-interview: A qualitative analysis. Med Educ  2009;43:360-7.
  28. Humphrey S, Dowson S, Wall D, Diwakar V, Goodyear HM. Multiple mini-interviews: Opinions of candidates and interviewers. Med Educ  2008;42:207-13.
  29. Patterson F, Zibarras L, Carr V, Irish B, Gregory S. Evaluating procedural justice in postgraduate medical selection. Med Educ  [forthcoming].
  30. Gilliland SW. The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review  1993;18:694-734.
  31. Gilliland SW. Effects of procedural and distributive justice on reactions to a selection system. J Appl Psychol  1994;79:691-701.
  32. Gilliland SW. Fairness from the applicant’s perspective: Reactions to employee selection procedures. International Journal of Selection and Assessment  1995;3:11-18.

Bill Irish chairman, General Practice National Recruitment Office  Severn Deanery, Bristol, UK
Alison Carr senior clinical adviser, Medical Education and Training Programme (England)  Department of Health, London, UK
David Sowden chairman, Conference of Postgraduate Medical Deans  East Midlands Deanery, University of Nottingham, Nottingham, UK
Neil Douglas president, Academy of Medical Royal Colleges  London
Fiona Patterson professor  Department of Social and Developmental Psychology, Cambridge University, Cambridge, UK

 bill.irish@southwest.nhs.uk

Cite this as BMJ Careers ; doi: