Development and validation of an automated assessment tool of echocardiography skills acquired on a neonatal simulator

Abstract

Introduction Simulators are increasingly used for training in echocardiography. However, there is no objective method to assess the skills acquired. Our objective was to develop and test an automated method to assess echocardiography skills. Methods To automate the image quality evaluation, we expanded our previously developed neonatal echocardiography simulator to enable recording of images of the 26 standard cuts and process the image quality. We then compared the automated and visual methods in scoring image quality of the echocardiograms obtained by 22 trainees. Results Each echocardiographic image representing a slice of a three-dimensional volume possesses 3 axes (X, Y, and Z) that correspond to the roll, pitch, and yaw angles of the transducer, respectively. Therefore, if the placement and orientation of the transducer are correct, the acquired image represents the appropriate cardiac window with the desired orientation in all 3 axes. The automated system gives a score of 0 if the transducer is not in the appropriate cardiac window. A score of 1, 2, or 3 is given if the image falls within the range of one, two, or three angles, respectively. There was no difference in the image quality score between automated and visual assessment methods (46.0 +/- 13.0 vs 45.1 +/- 14.4, P = .19). The two methods had excellent correlation (r = .95). The bias and precision were 0.9 and 8.8, respectively. Conclusions The automated method is comparable to visual method for assessment of image quality. The automated process allows for instantaneous feedback and has the potential to standardize assessment of echocardiography skills of trainees.