Could local initiatives shape the future of revalidation?

Authors: Abi Rimmer 

Publication date:  06 Jul 2015


Abi Rimmer talks to Julian Archer about his team’s review of the revalidation process and how local innovation could alter the national system

It has been nearly three years since the General Medical Council launched revalidation in December 2012, and so far over 100 000 doctors have successfully gone through the process.[1] But reactions to the system have been mixed. While most revalidated doctors have reported having a positive experience, other members of the profession have questioned revalidation’s ability to detect incidents of poor practice.[2] [3] Some doctors have even gone as far as to blame their colleagues’ early retirement on the workload entailed in taking part in the process.[4]

To discover how well revalidation is working on the ground, the General Medical Council has commissioned a review of the process. It will be carried out by a UK-wide collaboration of researchers led by the Collaboration for the Advancement of Medical Education, Research and Assessment (CAMERA) at Plymouth University Peninsula Schools of Medicine and Dentistry.

Julian Archer, director of CAMERA, says that despite the existence of national guidelines for revalidation there are still questions around how the process is working in practice, and it is these that CAMERA’s review hopes to explore. “Basically we’re looking at what’s happening out there,” he says. “Is revalidation really happening as people think it should be happening? Are people really doing what’s expected of them? What goes well? What doesn’t go well? What are people really doing? What is best practice? We want to draw out those things to then hopefully shape policy going forward to say, ‘There’s really good practice here, the idea that one size fits all may not be the long term.’”

Survey about appraisals

As part of the review, CAMERA plans to survey around 153 000 doctors who have completed their training about their appraisals. “We are trying to understand what people really bring to appraisal, what they really use, what they really find helpful, whether they really reflect on things, and whether they think it’s made any difference,” Archer says.

“There will also be some questions that will help us do some minimal work around costs,” he adds. “How much time are people putting in [to their appraisal], how much time is allocated, and how much time do they really spend? We will also try to get some information about what happened before revalidation came in and what has changed.”

In addition to the survey, CAMERA hopes to audio record around 90 appraisals of doctors working in primary and secondary care. It will also gather appraisal data from Wales and Scotland which, unlike in England, are held centrally.

Although it is impossible to predict the outcome of CAMERA’s review, Archer says it may find that there is an argument for moving away from a universal, “one size fits all” system of revalidation. “This is all conjecture but it might be that we find that certain groups of doctors want to do something specific, such as the Royal College of Surgeons, for example, being interested in engaging more surgeons in national audit. Or that they are already doing something different and they are finding it really helpful and they’re taking it to appraisal and it’s really shaping their practice,” Archer says.

National variation

There are standard rules for how revalidation appraisals should be carried out, but Archer says that the review is likely to find that in practice there is some variation across the United Kingdom. “I suspect that interpretation of the rules is really very different across the country,” he says. “In some places that will be for the better and in other places it might not be. And that will be one of the things that comes out of the review.”

It is these sorts of differences in how the system is carried out that Archer thinks should be shared more widely. “The system is already quite different across different parts of the UK,” he says. “It is quite surprising how little good practice does get spread and disseminated and learnt from.”

Once the review is completed, CAMERA’s report will outline how it thinks revalidation could be changed in the future. “We will say, ‘Here’s what is happening, this is why we think things have changed, this is what the literature tells us about it, and therefore this is what we think should happen going forward,’” Archer says.

It is, in principle, possible that parts of the revalidation process could become automated in future, Archer says, but this is not something that he would support. “In theory you could end up with a whole load of metrics. You could remove appraisal and ultimately all those metrics could come together. You would be told, ‘Congratulations you are a 1.2 this year up from 1.15 last year,’ and unless you drop two standard deviations below whatever, no lights go off. The system could be set up so that, if something happens, an amber light goes off and, if something else happens, a red light goes off.”

This type of automated system is not, however, compatible with medicine, Archer points out. “Medicine is a holistic, complicated profession and it isn’t just about numbers,” he says. “The whole patient journey is a genuine team thing. Certainly in secondary care very few doctors work in isolation and increasingly in primary care it’s becoming the same.”

In both primary and secondary care, teams have an impact on the care of a patient, Archer says, so it is difficult to attribute outcomes to individual doctors. “So then to say, ‘We’ve worked out a clever way of allocating this amount of credit or accountability for the outcomes of that patient to this doctor’ would be a very hard thing to do.”

Revalidation as exam

With the introduction of the UK medical licensing assessment for all doctors wishing to work in the UK, there is also the question of whether revalidation will become an exam.[5] Archer says, “If you have to go through a process to get your licence which involves an exam you could imagine that the next discussion would be, then why not have an exam to maintain your licence?”

But Archer says that looking at how doctors work in practice, which is what the current system of revalidation aims to do, is a far better method of assessment. “The Holy Grail is to really understand what people do in practice, and that is what revalidation is loosely trying to do,” he says “At least it is at people’s place of work. There is the idea that it covers the full scope of their practice and that as a professional you have everything that you do brought to one place and discussed.”

Ultimately, Archer thinks the CAMERA review will highlight areas of good practice in revalidation, and places where the system can be improved. “There clearly will be a group of doctors who, when our conclusion is not ‘It’s all a load of rubbish and a waste of time and it needs to be stopped tomorrow’ are not going to be happy. I don’t think that is what the main findings are going to be.”

Competing interests: I have read and understood BMJ’s policy on declaration of interests and have no relevant interests to declare.

References

  1. General Medical Council. Revalidation operational data reports. [Link] .
  2. Rimmer A. Three quarters of doctors are confident in GMC regulation, researchers say. BMJ Careers  2015. [Link] .
  3. Rimmer A. Just 23 GPs and 36 hospital specialists have had revalidation deferred owing to concerns, GMC figures show. BMJ Careers  2014. [Link] .
  4. Rimmer A. Bureaucracy is forcing GPs to quit under “euphemism of early retirement”. BMJ Careers  2015. [Link] .
  5. Gulland A. GMC proposes single test for all doctors wishing to work in UK. BMJ Careers  2015. [Link] .

Abi Rimmer BMJ Careers

 arimmer@bmj.com

Cite this as BMJ Careers ; doi: