Performance of Pediatric Neuroradiologists Working from Home during a Pandemic at a Quaternary Pediatric Academic Hospital

A.C. Sher; R. Salman; V.J. Seghers; N.K. Desai; M.B.K. Sammer

doi:10.3174/ajnr.A7411

Abstract

BACKGROUND AND PURPOSE: As a result of the coronavirus disease 2019 (COVID-19) pandemic, many radiology departments shifted to working a portion of clinical assignments from home. To determine the effect of working from home on performance, productivity, quality, and safety, we evaluated turnaround time, volume of studies, and error rates on rotations worked from home compared with in the hospital.

MATERIALS AND METHODS: The number of studies interpreted per day for each neuroradiologist, turnaround times, and error rates reported to peer learning was identified from April 1, 2020, through September 30, 2020. For each neuroradiologist, mean turnaround times and volumes per day at home versus in the hospital were compared. Similar comparison was performed for STAT studies.

RESULTS: During the time period, 2597 CTs (1897 at home, 700 in the hospital) and 3685 MRIs (2601 at home, 1084 in the hospital) were read. By individual neuroradiologists, 57% (4/7) had shorter turnaround time at home and 57% (4/7) demonstrated an increase in the mean number of studies per day read at home. No statistically significant difference was noted in the neuroradiologists’ performance while reading STAT studies. Reported error rates were not found to be higher at home, with statistically significantly lower rates when working at home (P = .018).

CONCLUSIONS: Variable productivity and performance of neuroradiologists when working from home versus in the hospital were found, being 57% faster and/or more productive while working at home without an increase in error rates. The decision to work at home versus in the hospital may best be based on local factors, balancing the variability among individual neuroradiologist’s and the institution’s needs, recognizing that working from home is not a one-size-fits-all phenomenon but requires adaptability for successful implementation.

ABBREVIATIONS:

COVID-19: coronavirus disease 2019
SARS-CoV-2: Severe Acute Respiratory Syndrome coronavirus 2
STAT: statim
TAT: turnaround time

The coronavirus disease 2019 (COVID-19) pandemic caused by Severe Acute Respiratory Syndrome coronavirus 2 (SARS-CoV-2) has impacted medical institution workflows, including radiology departments, which reorganized and redeployed resources. Multiple institutions have reported their experience with off-site reading or internal teleradiology through the use of home PACS workstations.^1⇓⇓⇓-5 These measures allowed radiology departments to continue their operations while ensuring the required social distancing and limiting staff exposure to SARS-CoV-2.^2,3

A survey distributed to radiologists from different US radiology residency programs reflected improved faculty satisfaction with off-site reading in terms of lower stress levels, improved performance, and decreased interruptions.⁵ However, academic institutions have also reported negative feedback from faculty and trainees concerning the lack of direct contact with each other and ordering providers.^5⇓-7

Facing similar challenges as a result of the COVID-19 pandemic, neuroradiologists in our department were all able to participate in working assignments from at-home workstations beginning April 1, 2020. Neuroradiologists shared in-hospital versus at-home reading locations to ensure continuity of hospital presence; rotational assignments and responsibilities remained unchanged regardless of reading location. Our aim was to describe the results of these modified workflows regarding performance and productivity metrics from our neuroradiologists as well as the quality and safety of their off-site reading by evaluating the turnaround time (TAT), volume of studies, and error rates from at home compared with in hospital.

MATERIALS AND METHODS

Study Design

This retrospective study is Health Insurance Portability and Accountability Act–compliant and received institutional review board approval via protocol H-49723. It was conducted in a quaternary care pediatric academic health system that includes a primary teaching hospital and 2 community pediatric hospitals.

We previously published our experience and technical specifications regarding use of off-the-shelf home PACS workstations in response to the COVID-19 pandemic.² Briefly, the memory of the home PACS workstations, central processing unit, and video card specifications allows similar processing power compared with the hospital workstations, and the home PACS workstations were installed with identical software. Monitors were chosen that, together with calibration software, are FDA-approved for nonmammography diagnostic imaging.

In our imaging department, neuroimaging work rotations between 7:00 am and 10:00 pm are staffed exclusively by neuroradiologists with additional training in pediatric neuroimaging. While largely similar, there are some differences among these clinical rotations because some may have additional clinical conference assignments or educational responsibilities. The rotations ranged from 8 to 10 hours. Imaging examinations were not assigned to any particular rotation or radiologist but were listed on a common worklist accessible to all. Our neuroradiologists do not perform interventional procedures such as lumbar puncture.

Data Collection

The number of studies (CT or MR imaging) read by members of the pediatric neuroradiology division per day (7:00 am to 10:00 pm), TATs, and error rates were evaluated from April 1 through September 30, 2020. TATs were defined as the time elapsed between when the study was available for interpretation (images in PACS and listed on the reading worklist) until the final report timestamps. Studies spanning multiple days, those read by an attending on a trainee rotation, head and spinal ultrasounds, and studies performed overnight were excluded. When we compared at-home versus in-hospital intrareader variability, 3 of 10 pediatric neuroradiologists were excluded because they did not work >5 rotations both at home and in the hospital. The number of studies was defined by the number of accession numbers. For example, MR imaging of the cervical spine and thoracic spine were considered 2 separate studies if there were 2 accession numbers. Reading errors were classified into cognition, perception, and reporting errors.

Statistical Analysis

Statistical analysis was performed using SAS (Version 9.4, SAS Institute), R statistical and computing software (Version 4.04; http://www.r-project.org/), and the epiR package (Version 2.0.19; https://rdrr.io/cran/epiR/).

For each neuroradiologist who performed >5 rotations both at home and in the hospital, the mean TATs and volumes from at home compared with in the hospital were compared on a per-neuroradiologist and per-technique basis. Similar comparison was performed for STAT studies.

The 2-tailed t test was used to compare the statistical significance of differences in means, and the Fisher exact, for error rates. Statistical significance was defined as P ≤ .05.

RESULTS

The inclusion criteria were met by 2597 CTs (1897 at home, 700 in the hospital) and 3685 MRIs (2601 at home, 1084 in the hospital), and the scans were read by 7 pediatric neuroradiologists. For these readers, experience after residency ranged between 9 and 32 years (mean, 15.7 years) (Table). Studies spanning multiple days (n = 110, eg, functional studies requiring delayed image lab postprocessing), those read by an attending on a trainee rotation (n = 2710), head and spinal ultrasounds (n = 1027), and studies performed overnight (n = 271, ie, 10:00 pm to 7:00 am) were excluded, for a total of 4118 excluded studies.

View this table:

Number of rotations worked by each neuroradiologist at home versus in the hospital and their years of experience after residency

The Online Supplemental Data summarize the mean TAT and the number of all studies (including STAT studies) read by the neuroradiologists at home versus in the hospital for each technique (CT and MR imaging). By individual radiologist, 57% (4/7) had shorter TATs at home for CT and/or MR imaging, though only 43% (3/7) were statistically significant. Conversely, 43% (3/7) had longer TATs at home, though only 14% (1/7) were statistically significant. The mean number of studies read by neuroradiologists was statistically higher at home than in the hospital in 57% (4/7), though statistically significantly higher in 43% (3/7). In 43% (3/7), the mean number of studies read at home was fewer, though in only 14% (1/7) was this result statistically significant.

The Online Supplemental Data summarize the mean TAT and number of STAT studies performed by the neuroradiologists at home versus in the hospital for each technique (CT and MR imaging). There was no statistically significant difference in the TAT, with 71% (5/7) of neuroradiologists demonstrating shorter TATs at home with either or both imaging modalities. The mean number of studies read by neuroradiologists was statistically significantly higher at home than in the hospital in 29% (2/7). Only 1 attending (1/7, 14%) read fewer CT and MR imaging studies at home compared with in the hospital.

Reported error rates for peer learning were not higher when working from home, with a statistically significantly lower rate at home (P = .018). This finding was derived from 11 peer reviews submitted on 4498 studies read at home (0.2%) compared with 12 submitted on 1784 read in the hospital (0.7%). At home, reported errors included perception (n = 9), cognition (n = 1), and reporting (n = 1) versus in the hospital where errors included perception (n = 9) and cognition (n = 3).

DISCUSSION

At our institution, the impact of working at home compared with in the hospital on productivity was not found to be consistent among individual radiologists. Rather, there was variability in individual neuroradiologists’ productivity and performance with >57% of neuroradiologists having shorter TATs and reading more studies at home compared with in the hospital. In addition, the neuroradiologists showed comparable performance while reading STAT studies from home versus in the hospital. There was also a statistically significant lower rate of reported errors for the studies interpreted at home. Our findings are concordant with the overall experience of several radiology departments with internal teleradiology⁵ but provide additional insight into how working from home may affect operational and quality measures. In the existing literature, 96% of radiologists (119/124), responding to a survey sent to US radiology residency program directors in March 2020, subjectively reported improved or no substantial change in TAT.⁵ Decreased interruptions were reported by most radiologists (64%) in this survey⁵ and can be considered one of the factors that contributed to improved TAT, reporting performance, and accuracy. It is known that interruption during imaging interpretation might lead to a significant increase in time spent on the report and can be associated with a higher number of reading discrepancies.^8,9 Our findings of improved TATs for a subset of radiologists is concordant with these reports.

On the other hand, TATs were faster in the hospital for a few neuroradiologists. Although the main goal of applying internal teleradiology in radiology departments during COVID-19 was to safely handle the departmental workflow, many challenges can be encountered while working at home that could affect the reader’s performance, such as sharing the home with a partner who is also working remotely, virtual school for children, or childcare. Technical problems related to home workstations and Internet connectivity may also take longer to solve remotely despite technical support from information technology.^2,4 Open communication with faculty regarding their home environment, technical needs, and desired balance between in-hospital and at-home work rotations can help ensure professional satisfaction and maximize productivity. Annual or biannual review of productivity metrics could enable discovery of undisclosed challenges, whether of a technical, communication, diagnostic, or social nature. For example, some radiologists may simply prefer working in the hospital rather than at home. Other radiologists may be more or less productive in the hospital versus at home, depending on whether they have additional teaching or administrative responsibilities that day or on the basis of the unique nature of a particular rotational assignment. Awareness of these issues and recognizing that working from home is not a one-size-fits-all phenomenon but requires flexibility and adaptability will be instrumental in successful implementation.

We also noticed fewer quality and safety submissions from at home. This issue was beyond the scope of our study but could be attributed to the decreased volume of studies ordered during the pandemic. In addition, there was not a substantial change in the quality of reports and interpretations when working from home. Given the overall relatively few reported errors, the impact of our findings is uncertain. Our system of daily peer review only assigns a small subset of cases per day to each radiologist, for example, so the bulk of peer review submissions is voluntary. There is an additional voluntary system of peer-review submissions on the part of referring providers to enable awareness of missed opportunities. These voluntary systems may underestimate the total number of errors from imaging reports. However, the voluntary nature of error reporting is the same for both in-hospital and at-home rotations, and our findings do not indicate that working from home is associated with increased errors.

The improved ability for radiologists to work from home necessitated by the COVID-19 pandemic and its continued use despite easing of community lockdowns and Stay Home–Work Safe orders has raised concern for the decreased emotional connection between faculty members and their respective departments and institutions, as well as erosion of relationships between radiologists and referring providers and hospital administrators. There is apprehension that increased isolation from work due to home arrangements will create a more transactional relationship with the hospital, in which the focus is more on productivity metrics of imaging interpretation and less on its impact on patient care, leading to radiology being perceived as a fungible commodity.^5,10

Some authors have suggested that radiologists should avoid functioning as production line workers but should be consultants, with an emphasis on collaborative effort with referring providers to pool their collective knowledge and experience to arrive at a diagnosis or diagnostic plan. As described by Gunderman and Chou,¹¹ radiologists can perform as “Isolated Radiologists,” in which their reading room is distant and the ability for providers to communicate with the radiologist is cumbersome; “Available Radiologists,” in which the reading room is more easily available but the radiologist is reactive, only responding to requests for help on initiation by the referring provider; an “Eager Radiologist,” in which the reading room remains convenient and the radiologist actively builds consultative relationships with referrers by interacting with them on a regular basis, often on their own initiative; or as an “Embedded Radiologist,” in which the radiologist functions as an integrated member of the patient care team, spending a substantial portion of the day in direct contact with referring providers and patients. Each of these concepts has advantages and disadvantages, but an obvious concern is that working from home may create a dominant Isolated Radiologist model.

However, the use of video cameras, online collaboration software such as Teams (Microsoft) or Zoom (Zoom Video Communications), and a radiology operator service to quickly route requests for consultation to the relevant radiologist can help ensure that radiologists maintain their pivotal role in the patient’s health care team, regardless of actual physical location. Creation of a “virtual” radiology reading room has the potential to make the radiologist even more accessible and more involved in patient care and need not imply in-hospital or at-home coverage by radiologists. Reconsideration of the strategic goals in radiology and value propositions such as building consultative relationships with referrers can create accessible radiology teams and remove geographic constraints, thereby allowing radiologists to operate more efficiently.¹²

The study has limitations, including its retrospective design. It stands to reason other unmeasured factors contributed to TATs, volumes, and error rates such as variability in the number of neuroradiologists working each day, the experience postresidency of available providers, or referring clinic operating hours. There was heterogeneity in the number of at-home versus in-hospital rotations among the radiologists, with 3 radiologists working significantly fewer in-hospital rotations relative to at-home rotations, a factor that could have impacted comparative assessment. In addition, the complexity of cases was not taken into consideration, which may have impacted performance because complex cases often require more time for investigation of prior cases and the medical record, interpretation, and reporting. However, because cases were randomly read between at home and in the hospital, we would expect a relatively similar distribution of complex cases between the 2 locations. Nevertheless, because the radiology community predominately uses these metrics for operational decisions and assessment of quality and safety, they are used here.

This inquiry into the effects of neuroradiologists working from home during the COVID-19 pandemic is focused on the metrics of performance, productivity, quality, and safety but does not address the entirety of the experience. For example, we did not investigate the individual radiologist’s feelings of professional accomplishment when working from home versus in the hospital nor his or her greatest challenges and benefits when working from home. In any academic institution, there are many groups with whom the radiologists communicate on a daily basis, including referring providers, radiology technologists, nursing staff, and trainees, and successful implementation of a work-from-home arrangement needs to take their perspective into account. Future inquiry into research productivity and education should be made to ensure that any work-from-home arrangements continue to enable growth and investment in our medical specialty, as well as teaching of the next generation of radiologists.

Also of note, we focused on the intravariability of each neuroradiologist when working from home versus in the hospital. This study did not evaluate intervariability within the group, though it stands to reason that some individuals simply prefer working at home versus in the hospital or vice versa, which could account for their performance metrics. Finally, this study was conducted in a single pediatric academic institution with a small cohort of pediatric neuroradiologists. As demonstrated by the variability among the radiologists, if other sites conducted similar studies, they may have different results. However, this study does provide a framework for comparing working from home with in-hospital metrics and also a plausible representation of how the metrics may change more generally at other sites.

CONCLUSIONS

At our hospital, there was not a consistent operational impact of working from home versus in the hospital on TATs, volume of studies interpreted per day, or error rates. This finding suggests that postpandemic hospitals can pivot more toward a hybrid model and allow a greater number of at-home rotations to address burnout and retention of faculty. The decision to work at home versus in the hospital likely should be based on local factors, balancing the variability among individual radiologists and the institution’s needs and preferences.

Acknowledgments

We acknowledge members of the Texas Children’s Hospital COVID-19 Imaging Taskforce, of which these authors were members, but also included Ananth V. Annapragada, R. Paul Guillerman, Thierry A.G.M. Huisman, Prakash M. Masand, Gunes Orman, and Amir H. Pezekhmehr.

Footnotes

Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.

References

1.↵
1. Shi J,
2. Giess CS,
3. Martin T, et al
. Radiology workload changes during the COVID-19 pandemic: implications for staff redeployment. Acad Radiol 2021;28:1–7 doi:10.1016/j.acra.2020.09.008 pmid:33036897
CrossRef PubMed
2.↵
1. Sammer MBK,
2. Sher AC,
3. Huisman TA
et al. Response to the COVID-19 pandemic: practical guide to rapidly deploying home workstations to guarantee radiology services during quarantine, social distancing, and stay home orders. AJR Am J Roentgenol 2020;215:1417–20 doi:10.2214/AJR.20.23297 pmid:32602344
CrossRef PubMed
3.↵
1. Tridandapani S,
2. Holl G,
3. Canon CL
. Rapid deployment of home PACS workstations to enable social distancing in the coronavirus disease (COVID-19) Era. AJR Am J Roentgenol 2020;215:1351–53 doi:10.2214/AJR.20.23495 pmid:32432912
CrossRef PubMed
4.↵
1. Lowery B,
2. Sandhu S,
3. Cook TS, et al
. The role of imaging informatics in disaster preparedness during the COVID-19 pandemic. J Digit Imaging 2021;34:330–37 doi:10.1007/s10278-021-00469-8 pmid:34160715
CrossRef PubMed
5.↵
1. Quraishi MI,
2. Rizvi AA,
3. Heidel RE
. Off-site radiology workflow changes due to the coronavirus disease 2019 (COVID-19) pandemic. J Am Coll Radiol 2020;17:878–81 doi:10.1016/j.jacr.2020.05.008 pmid:32425711
CrossRef PubMed
6.↵
1. Matalon SA,
2. Souza DAT,
3. Gaviola GC, et al
. Trainee and attending perspectives on remote radiology readouts in the era of the COVID-19 pandemic. Acad Radiol 2020;27:1147–53 doi:10.1016/j.acra.2020.05.019 pmid:32507612
CrossRef PubMed
7.↵
1. Le VT,
2. Akbari YS,
3. El-Ali AM
. An unexpected upheaval: pediatric radiology fellows’ experience during COVID-19. Pediatr Radiol 2021;51:216–19 doi:10.1007/s00247-020-04899-4 pmid:33416923
CrossRef PubMed
8.↵
1. Drew T,
2. Williams LH,
3. Aldred B, et al
. Quantifying the costs of interruption during diagnostic radiology interpretation using mobile eye-tracking glasses. J Med Imaging (Bellingham) 2018;5:031406 doi:10.1117/1.JMI.5.3.031406 pmid:29531970
CrossRef PubMed
9.↵
1. Bell LT,
2. James R,
3. Rosa JA, et al
. Reducing interruptions during duty radiology shifts, assessment of its benefits and review of factors affecting the radiology working environment. Clin Radiol 2018;73:759.e19–59.e25 doi:10.1016/j.crad.2018.04.007 pmid:29853302
CrossRef PubMed
10.↵
1. Hayes E
. Will the shift to remote reading during COVID-19 be permanent? https://www.auntminnie.com/index.aspx?sec=ser&sub=def&pag=dis&ItemID=133708. Accessed November 7, 2021
11.↵
1. Gunderman RB,
2. Chou HY
. The future of radiology consultation. Radiology 2016;281:6–9 doi:10.1148/radiol.2016152781 pmid:27643764
CrossRef PubMed
12.↵
1. Yacoub JH,
2. Swanson CE,
3. Jay AK, et al
. The radiology virtual reading room: during and beyond the COVID-19 pandemic. J Digit Imaging 2021;34:308–19 doi:10.1007/s10278-021-00427-4 pmid:33620622
CrossRef PubMed

Received October 8, 2021.
Accepted after revision November 20, 2021.

In this issue

Download PDF

Email Article

Citation Tools

Purchase

Cited By...

Working from Home: Brave New World, or Best Forgotten?

More in this TOC Section

Show more PEDIATRICS

[1] 1.↵
Shi J,
Giess CS,
Martin T, et al
. Radiology workload changes during the COVID-19 pandemic: implications for staff redeployment. Acad Radiol 2021;28:1–7 doi:10.1016/j.acra.2020.09.008 pmid:33036897
CrossRef PubMed

[2] Shi J,

[3] Giess CS,

[4] Martin T, et al

[5] 2.↵
Sammer MBK,
Sher AC,
Huisman TA
et al. Response to the COVID-19 pandemic: practical guide to rapidly deploying home workstations to guarantee radiology services during quarantine, social distancing, and stay home orders. AJR Am J Roentgenol 2020;215:1417–20 doi:10.2214/AJR.20.23297 pmid:32602344
CrossRef PubMed

[6] Sammer MBK,

[7] Sher AC,

[8] Huisman TA

[9] 3.↵
Tridandapani S,
Holl G,
Canon CL
. Rapid deployment of home PACS workstations to enable social distancing in the coronavirus disease (COVID-19) Era. AJR Am J Roentgenol 2020;215:1351–53 doi:10.2214/AJR.20.23495 pmid:32432912
CrossRef PubMed

[10] Tridandapani S,

[11] Holl G,

[12] Canon CL

[13] 4.↵
Lowery B,
Sandhu S,
Cook TS, et al
. The role of imaging informatics in disaster preparedness during the COVID-19 pandemic. J Digit Imaging 2021;34:330–37 doi:10.1007/s10278-021-00469-8 pmid:34160715
CrossRef PubMed

[14] Lowery B,

[15] Sandhu S,

[16] Cook TS, et al

[17] 5.↵
Quraishi MI,
Rizvi AA,
Heidel RE
. Off-site radiology workflow changes due to the coronavirus disease 2019 (COVID-19) pandemic. J Am Coll Radiol 2020;17:878–81 doi:10.1016/j.jacr.2020.05.008 pmid:32425711
CrossRef PubMed

[18] Quraishi MI,

[19] Rizvi AA,

[20] Heidel RE

[21] 6.↵
Matalon SA,
Souza DAT,
Gaviola GC, et al
. Trainee and attending perspectives on remote radiology readouts in the era of the COVID-19 pandemic. Acad Radiol 2020;27:1147–53 doi:10.1016/j.acra.2020.05.019 pmid:32507612
CrossRef PubMed

[22] Matalon SA,

[23] Souza DAT,

[24] Gaviola GC, et al

[25] 7.↵
Le VT,
Akbari YS,
El-Ali AM
. An unexpected upheaval: pediatric radiology fellows’ experience during COVID-19. Pediatr Radiol 2021;51:216–19 doi:10.1007/s00247-020-04899-4 pmid:33416923
CrossRef PubMed

[26] Le VT,

[27] Akbari YS,

[28] El-Ali AM

[29] 8.↵
Drew T,
Williams LH,
Aldred B, et al
. Quantifying the costs of interruption during diagnostic radiology interpretation using mobile eye-tracking glasses. J Med Imaging (Bellingham) 2018;5:031406 doi:10.1117/1.JMI.5.3.031406 pmid:29531970
CrossRef PubMed

[30] Drew T,

[31] Williams LH,

[32] Aldred B, et al

[33] 9.↵
Bell LT,
James R,
Rosa JA, et al
. Reducing interruptions during duty radiology shifts, assessment of its benefits and review of factors affecting the radiology working environment. Clin Radiol 2018;73:759.e19–59.e25 doi:10.1016/j.crad.2018.04.007 pmid:29853302
CrossRef PubMed

[34] Bell LT,

[35] James R,

[36] Rosa JA, et al

[37] 10.↵
Hayes E
. Will the shift to remote reading during COVID-19 be permanent? https://www.auntminnie.com/index.aspx?sec=ser&sub=def&pag=dis&ItemID=133708. Accessed November 7, 2021

[38] Hayes E

[39] 11.↵
Gunderman RB,
Chou HY
. The future of radiology consultation. Radiology 2016;281:6–9 doi:10.1148/radiol.2016152781 pmid:27643764
CrossRef PubMed

[40] Gunderman RB,

[41] Chou HY

[42] 12.↵
Yacoub JH,
Swanson CE,
Jay AK, et al
. The radiology virtual reading room: during and beyond the COVID-19 pandemic. J Digit Imaging 2021;34:308–19 doi:10.1007/s10278-021-00427-4 pmid:33620622
CrossRef PubMed

[43] Yacoub JH,

[44] Swanson CE,

[45] Jay AK, et al

Main menu

User menu

Search

American Journal of Neuroradiology

Performance of Pediatric Neuroradiologists Working from Home during a Pandemic at a Quaternary Pediatric Academic Hospital

Abstract

ABBREVIATIONS: