The purpose of audit: to improve care or boost trainees’ CVs?

Authors: Lawrence Kidd 

Publication date:  29 Jun 2015


Audit is seen as an important tool for career progression rather than improving patient care, Lawrence Kidd’s survey shows

Clinical audit can be a powerful tool for change, providing data and the impetus to improve.[1] There has been some disquiet about its limitations, however,[2] [3] and anecdotally some reports of trainee apathy towards it. Given that audit is a key tool in maintaining patient safety we did a survey investigating trainees’ attitudes and motivations towards it.

An anonymous online survey was sent to all specialty trainees across the Peninsula Deanery. Questions focused on motivations and expectations of the audit process and were designed to reflect the subjective nature of opinions. A modified Likert subjective scale was used, with answers selected from a range of opinion statements. Multiple ranked answers underwent sequence randomisation to reduce order bias.

In total, we received 321 responses (response rate 26%), of which 43% were from core trainees. Around half or 56% of trainees responding thought that the expectation to perform audit was “mandatory.”

The four most strongly supported reasons for performing audit were scoring “points” for job shortlisting, expectation to perform audit as part of training, need to satisfy criteria for job applications, and annual review of competency progression requirement.

The four least important reasons given were understanding the audit process, adherence to new guidelines, improving patient safety, and improving patient care.

When asked about their most recent audit, 16% of respondents reported no improvement in quality of patient care and 21% reported no improvement in patient safety. Around half (53%) of trainees thought improvements from their last audit project would remain long term, and 44% of respondents thought audit improved training.

This survey was designed to identify themes and opinions. The response rate (26%) is disappointedly low and raises the risk of considerable bias as only those who felt strongly enough replied. Nonetheless, a number of themes can be identified.

Cornerstone of clinical governance

Although just over half of those responding recognised it, clinical audit is mandatory. It is a cornerstone of clinical governance, a key part of postgraduate training, and an important domain in the shortlisting process for job applications.[4] Audit also has a greater purpose: to measure and drive improvement in what doctors do. It provides transparency and is ultimately directed towards improving safety and quality of care. It is therefore regrettable that these two motivations were two of the four least ranked motivations. Instead, the major motivations identified by trainees related to the “need” to perform audit with regard to “scoring points,” getting jobs, and succeeding in training.

Likewise, it is perhaps disappointing that despite being mandated by training and regulatory bodies, 56% of respondents thought they gained nothing from audit. Some themes that have arisen include the fact that audit is a time consuming and frustrating burden where information quality is often poor and incomplete. Despite this, there is pressure to complete the audit cycle, no matter how inadequate the results. Accepting poor quality and focusing on quantity undermine the whole audit process.

Only 53% of respondents expected to achieve long term change with their most recent audit. Interpretation of this depends on whether you see the glass half full or half empty, but it means that half of projects are done with low expectations of achieving meaningful improvement. While showing futility of interventions is important, it seems that some trainees are disillusioned.

One of the challenges for those involved in training and steering the move towards quality improvement is how to prevent the development of apathy. It would be a considerable waste if the enthusiasm and energy for quality improvement were lost and over time became tarnished as “audit by another name.” It would be a loss not just for the profession, but more importantly for patients also.

Trying to find solutions to such a complex problem is fraught with difficulty. Recently, Tantrige argued for the inclusion of protected time for audit with a focus on local guideline development.[5]

Collaboratives

Another solution may be the development of trainee led collaboratives. These have shown success in terms of recruitment[6] and doing research[7] and audit[8] and have become increasingly recognised in surgery[9] and anaesthetics.[10] They encourage trainees to initiate projects and therefore might make audit seem more worthwhile. They also use the potential disadvantage of trainees rotating through different hospitals and turn this into an asset, enabling better quality projects with more meaningful results.

Cynicism and resentment towards audit will remain as long as the focus is on minimum standards, which simply encourage the minimum to be done. Becoming involved in a larger ongoing project may be highly worthwhile for a trainee, but if the loop is not completed then it may count for nothing when it comes to shortlisting and appraisal processes.

A trainee led “ground-up” model with support from senior clinicians and management allowing improvements based on observations and areas of perceived improvement may well improve trainee buy-in.

The opinions of today’s trainees matter because they are the consultants and medical leaders of tomorrow. The evidence of apathy and cynicism towards something so fundamental to our work is regrettable, but it could act as a catalyst for change. Achieving meaningful change requires trainees who understand the rationale for their efforts and want to improve practice in response to problems they see. Hurdles need to be overcome to change the mindset and thus enable trainees to “not just do the job, but also to improve the job.”[11] Changing the culture requires re-energising and re-engaging trainees so that they do audits not because they must, but because they can.

Many thanks to Rob Price and Quentin Milner for their proofreading and advice.

Competing interests: I have read and understood BMJ’s policy on declaration of interests and declare the following interests: I am a South West Anaesthesia Research Matrix committee member 2014-5.

References

  1. Peden CJ, Rooney KD. Making improvement happen. Raising the standard: a compendium of audit recipes. 3rd ed. Royal College of Anaesthetists, 2012.
  2. Fulton RA. Goals and methods should be reappraised. BMJ  1996;312:1103.
  3. Berger A. Why doesn’t audit work? BMJ  1998;316:87.
  4. NHS. 2014 Person specification. Anaesthetics—ST3. Entry criteria. [Link] .
  5. Tantrige PM. Clinical audits must improve to benefit patients, providers, and doctors. BMJ Careers  2014. [Link] .
  6. Utility of an anaesthetic trainee led research network to contribute to multicentre studies. South West Anaesthesia Research Matrix (“SWARM”). Anaesthesia  2013;68(S3):44.
  7. Bhangu A, Kolias AG, Pinkney T, Hall NJ, Fitzgerald JE. Surgical research collaboratives in the UK. Lancet  2013;382:1091-2, doi: [Link] .
  8. South West Anaesthesia Research Matrix. [Link] .
  9. Association of Surgeons in Training. Trainee research collaborative. [Link] .
  10. National Institute of Academic Anaesthesia. A “how-to” guide to setting up a trainee-led research network. [Link] .
  11. Winthrop C, Wilkinson I, George J. The Francis and Keogh reviews have made junior doctors powerful agents for change. BMJ Careers  2013. [Link] .

Lawrence Kidd anaesthetics trainee, Royal Devon and Exeter Hospital, Exeter

 lawrence.kidd@nhs.net

Cite this as BMJ Careers ; doi: