ACPS

Auditing foot & ankle surgery to improve outcomes: The ACPS experience

As a Podiatric surgeon, Dr Rob Hermann is a registered specialist. He is endorsed by the Podiatry Board of Australia (PBA) to prescribe from a national list of scheduled medicines when treating podiatric conditions. He is a Commonwealth Accredited Podiatric surgeon for the purposes of private health insurance. Dr Hermann is passionately committed to the practice and advancement of podiatric surgery. He has incorporated this commitment in the provision of reconstructive foot and ankle surgery to provide the best patient care.

Dr Rob Hermann reflects on the development and implementation of an online audit tool to measure and report foot and ankle surgical outcomes, and in doing so, he advocates the importance of surgical audits.

Measuring to assist in managing outcomes has become an important framework in all healthcare disciplines. The use of clinical audit to measure and improve outcomes is crucial to providing the best quality healthcare.

The concept of ‘what gets measured gets managed’ has been attributed to management consultant Peter Drucker. However, the notion of ‘what gets measured gets managed’, may have origins from the Scottish physicist Lord Kelvin who developed the absolute temperature scale. So, the concept of ‘what gets measured gets managed’ has been with us for nearly 200 years.

Measuring to assist in managing outcomes has become an important framework in all healthcare disciplines. ^{1, 2, 3} The use of clinical audit to measure and improve outcomes is crucial to providing the best quality healthcare.

The ACPS National Audit Reports

In 2011, the Australasian College of Podiatric Surgeons (ACPS) developed and implemented an online audit tool to measure and report foot and ankle surgical outcomes. The ACPS National Audit Reports are used to improve the outcomes of foot and ankle surgery through national peer review as a quality improvement activity.

The ACPS National Audit Reports remain the only such publications available to the public for any specialist providers of foot and ankle surgery. This means the ACPS National Audit has set the benchmark for best practice in delivery of foot and ankle surgical care.

In the development of this tool a standardised surgical audit cycle was utilised. Auditing surgery requires that practice standards are identified. Where no standards exist, the literature is used to establish them. If minimal literature can be found expert consensus is used to establish standards. The surgical outcomes of interest are then measured. A comparison of the measured outcomes is then made with the previously selected standards. Analysis of the outcomes as compared to standards informs reflective practice. Then if indicated planning and executing practice change can happen. The audit cycle is an iterative process as shown in Figure 1.

Figure 1: The Audit Cycle.

Why audit surgical outcomes

Audit provides an accepted method for surgical providers to evaluate their performance and outcomes to improve quality of service provision. In essence, surgical audit is really the only way that surgeons can provide evidence that they do in practice what they say they do.

The importance of surgical audit rapidly increased in Australia from the late 1990s with the growing attention to clinical governance in healthcare. The Australian Commission on Safety and Quality in Health Care (ACSQHC) has set a standard for clinical governance. The ACSQHC standard requires health care organisations to take responsibility for continuous improvement of safety and quality in services ensuring patient centred care. Clearly auditing outcomes is an integral component of meeting the ACSQHC standard.

How did the ACPS develop its audit?

Under the supervision of Queensland University of Technology, audit activity in general, orthopaedic, plastic, and podiatric surgery was examined using a case study design. Enablers and barriers to audit participation were then identified and explored.

Results of the case study provided guidance to develop a Delphi survey. An international expert panel was formed; comprised of individuals from the above-mentioned surgical specialties or who were associated with foot and ankle surgery.

Using the Delphi technique, expert consensus was used to determine what and how to audit foot and ankle surgery. The Delphi-derived consensus informed modification of the Royal Australasian College of Surgeons generic dataset for surgical audit.

The ACPS online audit tool was then developed based on the Delphi survey findings. Piloting of the ACPS online audit tool was then completed. Usability of the ACPS online audit tool and data entry validity and reliability was assessed using an online survey. The ACPS online audit tool has been through two additional I.T. upgrades and is currently in a third phase of development and improvement.

The ACPS audit captures data such as all postoperative infection and thromboembolism data including cases managed as outpatients. In this way the ACPS Audit provides postoperative surveillance that extends far beyond any similar audit for other providers of foot and ankle surgery.

Improving outcomes using the ACPS audit

The ACPS audit exists in an online web-based environment. Real-time data capture and reporting is used for all cases of office and hospital-based foot and ankle surgery. Data including patient comorbidities, preoperative diagnosis and expected parameters of outcome such as length of time in recovery are entered prior to surgery.

All outcome data within 28 days of discharge is then captured in the postoperative period as recommended by the Australian Council on Healthcare Standards (ACHS). The ACHS only requires postoperative infection and thromboembolism data to be captured if readmission occurs. However, the ACPS audit captures data such as all postoperative infection and thromboembolism data including cases managed as outpatients. In this way the ACPS Audit provides postoperative surveillance that extends far beyond any similar audit for other providers of foot and ankle surgery.

Real-time data reporting is available to ACPS surgeons. This allows each surgeon to see their outcomes for a defined period and compare it to the national average in real-time. Surgeons can only see their own data and the deidentified national outcomes.

The ACPS online audit tool therefore provides an individualised and private environment where surgeons can compare themselves to the national outcomes. This means that ACPS surgeons can be aware of any outcomes that are outliers compared to the national group at any time. In essence surgeons can complete their audit cycles in real-time allowing immediate quality improvement at an individual level.

National peer review of ACPS audit

All ACPS member surgeons must participate in the College surgical audit to meet their annual accreditation requirements. Likewise, participation in six monthly national peer review is a requirement for College accreditation. Outcome audit data is monitored by the ACPS Audit Data Manager.

Cases identified as outliers such as readmission for infection, transfer to another facility or medical readmission are identified by the audit data manager. Deidentified cases records are then reviewed by the audit data manager and reported to the ACPS Clinical Audit Committee. The committee then analyses any outliers that require further assessment. Comparison with standards is performed and if needed, the surgeon is asked to discuss the case with the Audit Committee. If a deficiency in practice is found the surgeon is advised on how their practice should be modified. Should further review or mentoring be required a program is developed and put in place by the Audit Committee and the ACPS Council.

All cases of readmission are discussed at the national peer review meeting to drive quality improvement of the entire College. The ACPS has developed a collegiate and safe environment for its national peer review meetings. Surgeons are free to discuss their outcomes to drive discussion without the fear of intimidation or professional attack. By providing such an environment peer review of outcomes drives quality improvement. These activities fall under the auspices of the ACPS Clinical Governance Framework which supports ACPS member surgeons in providing quality care to all patients at all times.

For the past 11 years ACPS Audit Reports demonstrate a 97% chance of no complications in the first 28 days following discharge.

What are the ACPS audit outcomes?

The ACPS Audit Reports have consistently provided good evidence that podiatric foot and ankle surgery is safe and effective. For the past 11 years ACPS Audit Reports demonstrate a 97% chance of no complications in the first 28 days following discharge. The most common complication is outpatient managed infection. Readmissions have never been above 0.4 % since 2011. The ACPS Audit findings (on page 6) regarding complications are either within or below rates reported in the peer reviewed literature.⁶

A summary of the 2020 audit outcomes is provided in the below table:

Outcome	2020
Admissions	2296
Total principal procedures	2463
Forefoot/RF, Ankle. Amp	94% / 5.9%
Most frequent pathology	Ingrown toenail (33%)
Readmissions	0.34% (8 cases)
Infection readmissions	0.21% (5 cases)
Infection outpatient	1.8 % (42 cases)
VTE readmissions	0%
VTE outpatient	0.04% (1 case)
Wound break down readmissions	0.00% (0 case)
Wound break down outpatient	0.26% (6 case)

Table 1: 2020 ACPS Audit Outcomes

ACPS audit data has been compared to health department data for orthopaedic surgeons. This comparison occurred in the College’s resubmission to the Medical Services Advisory Committee (MSAC) in 2019. The purpose of this resubmission was to seek MBS funding for podiatric surgery. Of interest is the finding in April 2020 by the MSAC that it could not support the submission as, “there remains a lack of directly comparable evidence for safety and effectiveness, albeit primarily because of the paucity of high-quality contemporary evidence for the safety and effectiveness of the same services delivered by orthopaedic surgeons.”

The findings by MSAC were disappointing. This is particularly the case when no publicly available equivalent data for orthopaedic foot and ankle surgery exists. That said the ACPS will continue to publish its audit outcomes because supporting and implementing the College audit has set a benchmark in best practice.

The College remains hopeful that what gets measured does in the end get managed.

References

1. Connors, E.H., Douglas, S., Jensen-Doss, A. et al. What Gets Measured Gets Done: How Mental Health Agencies can Leverage Measurement-Based Care for Better Patient Care, Clinician Supports, and Organizational Goals. Adm Policy Ment Health 48, 250–265 (2021). https://doi.org/10.1007/s10488-020-01063-w

2. Managing antimicrobial resistance. Susanna J Dunachie, Nicholas PJ Day, Christiane Dolecek, The challenges of estimating the human global burden of disease of antimicrobial resistant bacteria, Current Opinion in Microbiology, Volume 57, 2020, Pages 95-101,

3. Management of Covid 19 vaccination priorities. Persad G, Emanuel EJ, Sangenito S, Glickman A, Phillips S, Largent EA. Public Perspectives on COVID-19 Vaccine Prioritization. JAMA Netw Open. 2021;4(4):e217943. doi:10.1001/jamanetworkopen.2021.7943