© 2022 MJH Life Sciences and AJMC – Managed Care News, Research, and Expert Insights. All rights reserved.
© 2022 MJH Life Sciences™ and Clinical Care Targeted Communications, LLC. All rights reserved.
The authors describe a primary care–based diabetic retinopathy screening program incorporating telemedicine, strong health information technology engagement, and development of clinical informatics tools.
ABSTRACT
Objectives: To improve diabetic retinopathy (DR) screening rates through a primary care–based “teleretina” screening program incorporating clinical informatics tools.
Study Design: Quality improvement study at an academic institution.
Methods: Existing DR screening workflows using in-person eye examinations were analyzed via a needs assessment. We identified gaps, which clarified the need for expanding DR screening to primary care settings. We developed informatics tools and described associated challenges and solutions. We also longitudinally monitored imaging volume and quality.
Results: The needs assessment identified several gaps in baseline DR screening workflows. Health information technology (IT) considerations for the new primary care–based teleretina screening program included integrating the new program with existing information systems, facilitating care coordination, and decreasing barriers to adoption by incorporating automation and other features aimed at decreasing end-user burden. We successfully developed several tools fulfilling these goals, including integration with the ophthalmology picture and archiving communication system, a customized aggregated report in the electronic health record to monitor screenings, automation of billing and health maintenance documentation, and automated results notification to primary care physicians. Of 316 primary care patients screened between October 2020 and July 2021, 73 (23%) were found to have ocular pathology, including DR, glaucoma, age-related macular degeneration, and a range of other eye conditions that were previously undiagnosed.
Conclusions: New models of health care delivery, including telemedicine workflows, have become increasingly important for complex diabetic care coordination and require substantial health IT engagement. This program illustrates how clinical informatics tools can make substantial contributions to improving diabetes care.
Am J Manag Care. 2022;28(10):In Press
Takeaway Points
Diabetic retinopathy (DR) is a leading cause of blindness and visual impairment in the United States. Screening is critical for preventing vision loss and is expanding to primary care and other settings. We describe the design and implementation of a primary care–based DR screening program, with a focus on health information technology (IT) considerations.
Diabetes is a global epidemic and is associated with multiple complications.1-4 One devastating complication is diabetic retinopathy (DR), which in the United States is the leading cause of blindness and visual impairment among working-age adults.5,6 Because vision loss can be prevented with early detection and treatment, DR screening is a public health priority and has been incorporated into quality measures such as the Merit-based Incentive Payment System (MIPS).7-9
DR screening programs in clinical and community-based settings have demonstrated feasibility, accuracy, and cost-effectiveness.10-14 The existing literature on telemedicine approaches for DR screening largely focuses on referral patterns to establish the ongoing need for screening.15-17 Recent advances in artificial intelligence (AI) have also enabled automated diagnosis of DR, an area of active research investigation.18-23
However, the literature on applied informatics related to DR screening is scarce. Scanlon et al24 described an automated data extraction process from general practitioners’ electronic health records (EHRs) to automatically update DR screening lists, which found patients who were previously unidentified. Another study by Melles et al25 demonstrated that utilizing a centralized virtual reading center (ie, a centralized group of ophthalmologists and optometrists interpreting retinal images for a wide range of clinical sites) was associated with improved screening accuracy. However, prior studies have not delved into the specific details of implementing DR screening programs from a health information technology (IT) perspective nor examined challenges related to integrating these programs with existing clinical information systems and workflows in detail.
Here, we describe our experience developing and implementing clinical informatics tools supporting a new primary care–based “teleretina” DR screening program. Our objectives were (1) to design a workflow for improving DR screening rates, (2) to develop tools facilitating integration of this workflow with existing clinical information systems, and (3) to engage relevant stakeholders and reflect on lessons learned to inform future improvements.
METHODS
Study Setting
This study was conducted at the University of California San Diego (UCSD), an academic medical center in San Diego County. Retinal cameras were installed at 3 primary care (internal medicine and family medicine) clinics. The study adhered to the Declaration of Helsinki and was approved by the UCSD Institutional Review Board as a quality improvement protocol.
Needs Assessment
Prior to program implementation, only about half (47%) of patients with diabetes at our institution had documented eye exams. Because in-person eye clinic examinations were not meeting quality metric goals, the need to expand screening to primary care sites (teleretina DR screening) was identified.
First, we conducted a detailed examination of the existing workflows for in-person DR screening by interviewing primary care physicians, eye clinic leadership, ophthalmologists, optometrists, and scheduling personnel. We identified care gaps to formulate a needs assessment, which informed the design of the new primary care–based screening program.
The framework of the new program entailed retinal imaging at primary care clinics, asynchronous image interpretation by ophthalmologists or optometrists (“eye care providers”), automated notification of primary care physicians (PCPs) regarding screening results, and subsequent care coordination. Primary care–based imaging would decrease the need for in-person eye examinations. Eligible patients consisted of adults (≥ 18 years) with a diagnosis of diabetes (any type) who lacked a documented eye exam within the prior year, without preexisting ophthalmic diagnoses or active vision complaints.
Design Goals for Informatics Tools
First, we aimed to integrate screening images with the existing eye clinic picture and archiving communication system (PACS) (ZEISS Forum; Carl Zeiss Meditec AG). Therefore, retinal images acquired in primary care would be uploaded to the same PACS, allowing eye care providers to view these images alongside patients’ prior ophthalmic imaging/testing.
Based on prior literature regarding the time and cognitive burden of health IT on health care providers,26-29 another design goal was to integrate the new workflow with the existing institutional EHR system (Epic Systems) and incorporate automation to reduce the time and effort required by end users. Specifically, we aimed to develop automated completion of billing and quality metric reporting via the “health maintenance topic” in Epic, a functionality to document and monitor completion of preventive health and quality measures.
Another design goal centered on facilitating the ease of finding patients who required imaging interpretations, entering imaging interpretations, and notifying PCPs of results. At UCSD, primary care staff utilized Epic’s Ambulatory module, whereas eye care providers utilized Epic’s Kaleidoscope module. Each module has different tools for ordering, exam interpretation, and result notification. Our goal was to integrate both modules while ensuring that users from both areas were still working in familiar EHR screens. We aimed to develop an EHR-integrated report to easily monitor the status of imaging exams and to facilitate interpretation completion while minimizing clicks and navigational steps.
Implementation Processes
Preimplementation stages included workflow design, equipment installation, and training activities. Primary care staff were trained regarding the rationale of the new DR screening program, patient eligibility criteria, camera use and image acquisition, and details of the new workflow within the PACS and EHR. This training was conducted via live/synchronous training sessions with the program leadership team, including a PCP lead and a nurse manager, and supplemented with written materials that were distributed to each screening site and posted on internal institutional websites. Training on camera use and image acquisition was provided by the camera vendor (Optos Inc) through in-person, hands-on training sessions prior to implementation. A camera vendor representative was also available via email and videoconference meetings to address additional questions.
Testing was performed to ensure successful integration of primary care images with the eye PACS as well as successful automated completion of billing/charge capture and the health maintenance topic upon imaging interpretation entry. Training of eye care providers was conducted to orient them to the teleretina imaging report and to the custom image interpretation entry tool. Three test patient charts, which were available in a “play” environment provided by the institutional EHR vendor, were used to ensure successful execution of the workflow prior to implementation in the production environment.
At go-live, on-site support was provided to the primary care clinics that were acquiring retinal images by informatics analysts, the PCP lead, and the nurse manager, who were physically present on the day of go-live and available to answer any questions from primary care staff as they entered orders, imaged patients, and uploaded images. Informatics support from the analysts was also available to eye care providers reading images. This included synchronous videoconference meetings to address questions regarding the image interpretation process and asynchronous support via email. Eye care providers were expected to read images each weekday and provide interpretations within 72 hours of image acquisition. Night/weekend coverage was not necessary because this was a screening program for asymptomatic individuals without known ocular diagnoses and not a venue for evaluating ocular emergencies.
Weekly stakeholder meetings were conducted preimplementation (for 6 months prior to go-live) and for 2 weeks after go-live. Thereafter, meetings were conducted monthly to review outcome metrics (see below) and highlight opportunities for ongoing improvement. Stakeholder groups represented at these meetings included primary care, endocrinology, ophthalmology, optometry, clinical informatics, EHR analysts/information services, imaging IT specialists, the camera vendor, the UCSD Quality Department, and the UCSD Population Health Services Organization (PHSO).
Outcome Evaluations
A predefined goal was specified as 4% improvement in the MIPS “diabetic eye exam” quality measure by June 30, 2021. Each month after go-live (October 12, 2020), reporting was completed regarding the number of images acquired, percentage of poor-quality images, and performance on the institutional diabetic eye exam metric. For this study we analyzed data from October 2020 to July 2021. We generated descriptive statistics of longitudinal trends using Microsoft Excel (Microsoft Corporation). We also identified health IT challenges that arose and qualitatively described solutions that were developed.
RESULTS
Needs Assessment
We examined existing DR screening workflows involving in-person eye examinations. Patients were identified via multiple methods: a red “health maintenance” alert in the EHR, review of encounters by primary care clinic staff during a “daily huddle,” and manual chart review. For these patients, PCPs placed referral orders to eye care providers. Additional patients needing screening were identified from the EHR and given bulk referral orders by the UCSD PHSO, the entity overseeing UCSD’s managed care population. Any patients with referral orders would receive an automated phone call with a prerecorded message asking the patient to schedule an eye clinic appointment. Once scheduled, the patient would go in person to the eye clinic and undergo imaging and examination by an eye care provider, who would write a progress note.
This workflow revealed several gaps (eAppendix Figure [available at ajmc.com]). First, patients could be lost to follow-up in the scheduling process. The automated message asking the patient to call and schedule their eye appointment required patient proactivity, and additionally they may not have been aware of the rationale for an eye exam. Furthermore, the eye clinic’s standard phone triage algorithm included the question, “Do you need glasses?” If the patient responded yes, they could be inappropriately scheduled for a refraction (ie, measured for glasses) rather than a dilated DR screening examination. An additional barrier was that at the time of this program implementation, not all eye care providers at our institution had adopted EHRs—some were still documenting on paper. Thus, PCPs were unable to easily discern the screening results. Even for eye care providers using the EHR, not all sent referral letters communicating results back to PCPs. Finally, there was no existing linkage to complete the EHR’s health maintenance topic related to diabetic eye exams. This required manual completion, of which most eye care providers were unaware.
We designed the new teleretina DR screening program (Figure 1) to fill some of these gaps. Acquiring retinal imaging in primary care clinics would reduce loss to follow-up and eliminate barriers related to time, transportation, and costs associated with an in-person eye examination. In addition, a workflow integrated with the EHR would allow screening results to be directly visible to PCPs and automatically complete health maintenance documentation for institutional quality metric reporting.
Informatics Tools for the Primary Care–Based DR Screening Program
We achieved our design goals with the following:
Integration with existing ophthalmology viewing systems. Images acquired in primary care were successfully integrated into the existing eye PACS. This required acquisition of additional licenses and a detailed security review for new device integration. Eye care providers were able to view all eye images in a single PACS, whether acquired in eye clinics or in primary care clinics.
Automated billing and charge capture. Charges were automatically entered upon signing the image interpretation and were automatically distributed between primary care (technical fees) and ophthalmology (professional fees) without requiring end-user participation.
Automated completion of health maintenance topic. Signing the image interpretation automatically completed the EHR health maintenance topic on diabetic eye exams without requiring manual completion. This feature was used not only for identifying patients requiring DR screening but also for institutional quality metrics reporting.
Report for tracking screened patients. In this primary care–based screening program, the ordering providers were PCPs from multiple clinics. To avoid eye care providers having to manually search across imaging orders from a wide range of PCPs, a custom report was created to aggregate all teleretina DR screening orders (Figure 2).
Macros for standardizing imaging interpretation. We connected the screening report to a customized image interpretation window (Figure 3 [A]), which enabled (1) data entry without needing to open the patient’s chart, (2) subsequent reporting of structured/discrete data elements, and (3) the building of macros (a saved set of features) to facilitate interpretation completion. This window still generated a natural language note (Figure 3 [B]) easily readable by PCPs.
Outcomes
A total of 316 patients underwent primary care–based DR screening between October 12, 2020, and July 31, 2021. Monthly imaging volume ranged from 17 (July 2021) to 46 (March 2021) (Figure 4 [A]). Of 316 patients screened, 58 (18.4%) had at least 1 eye with insufficient image quality for interpretation. Monthly proportions of low-quality images ranged from 6.9% to 42.9% (Figure 4 [B]). Of screened patients, 73 (23.1%) had ocular pathology. Diagnoses included DR, glaucoma/glaucoma suspect, age-related macular degeneration, epiretinal membranes, optic nerve drusen, choroidal nevi, and chorioretinal scarring. Images of a patient with previously undiagnosed DR identified through this program are depicted in Figure 5.
Our institutional performance was 47% for the diabetic eye exam MIPS metric before implementation of the primary care–based DR screening program. As of June 30, 2021, the program achieved 65% for the diabetic eye exam metric, exceeding the predefined goal of 4% improvement over baseline.
Informatics Challenges and Solutions
The tools described above were successfully deployed at go-live and enabled image integration with the eye PACS, streamlined navigation to imaging orders and interpretation completion, and automated billing and health maintenance completion. However, in the postdeployment phase several challenges arose requiring further iteration.
Managing imaging orders. Not all imaging orders were completed, because the patient either declined imaging or could not complete imaging due to their own time constraints or primary care staffing constraints. These orders remained on the screening report for eye care providers, causing wasted time in searching for those patients in the ophthalmology PACS only to find that no images were present. To resolve this, primary care staff were instructed to cancel any imaging orders not completed, and the report was modified to reflect cancellation status to reduce processing time for eye care providers.
Undoing automated billing and health maintenance completion for poor-quality images. After implementation, we realized that low-quality images resulted in billing and health maintenance completion due to automation in the initial build, even though DR screening had not truly been completed. To address this, eye care providers manually removed the charges and added narrative text to the interpretation to inform PCPs that the health maintenance topic was not fulfilled. We are pursuing a design modification allowing the eye care provider to replace the original “Teleretina photos” order with a “Teleretina photos – Incomplete” order for poor-quality images. In testing, this design will not enter or file a charge, nor will it complete the health maintenance topic. Additionally, the PCP will be notified appropriately and the health maintenance topic will retrigger an alert for DR screening. This modification is undergoing security review and is not yet live.
DISCUSSION
This program illustrates how health IT engagement with multidisciplinary teams was critical for facilitating information flow between primary care clinics and eye care providers and subsequently contributing to improvements in DR screening and documentation important for monitoring population-level quality metrics.
Expanding DR screening to primary care clinics improved diabetic eye exam metric performance. This was consistent with findings of prior studies, such as the finding by Hatef et al that the likelihood of completing an annual diabetic eye exam in a managed care Medicaid population increased with access to a camera in the primary care clinic.30 Furthermore, we screened 316 patients over a 9-month period despite the COVID-19 pandemic. This compares favorably to prior studies, including a primary care–based program in the same state that screened 290 patients over a 12-month period preceding the pandemic.31 The increased DR screening rates with primary care integration are likely attributable to several factors, including avoiding scheduling/referral errors; decreasing barriers associated with time, costs, and transportation of attending a separate eye clinic appointment; and building upon a trusted relationship with a PCP. The scope of image acquisition will likely further expand with the availability of handheld cameras and smartphone/mobile cameras and attachments,16,32-35 building upon the growing trend of digital health and patient-generated data outside of traditional clinical settings.36-38 AI has been another tool in advancing DR screening, with a DR screening system being the first FDA-approved autonomous AI system in all of medicine.39
Health IT considerations are critical, particularly to ensure complete and accurate transfer of imaging and interpretation/results data among relevant parties. Integration with existing EHR systems and PACS is crucial. This is particularly challenging for eye care, because ophthalmic imaging vendors often have proprietary platforms and information standards adoption remains low.40,41 Ongoing efforts to encourage standards adoption among vendors to facilitate integration among eye imaging devices, PACS, and EHRs will be important for continuing to expand DR screening efforts.
Another consideration is designing IT systems to promote usability and adoption. Developing new DR screening workflows outside of typical eye clinic encounters required significant buy-in, particularly from primary clinic staff already facing multiple competing priorities in providing comprehensive care. The contributions of EHRs and health IT to burnout are well known in primary care,27,42-45 and there is increasing awareness of the time requirements and burden of health IT in ophthalmology as well.46-49 Thus, we aimed to design the DR screening workflow to incorporate automated components, facilitate imaging order processing and interpretation completion, and reduce the clicks, navigation, and time needed. Even then, some unintended consequences arose, including inappropriate billing and health maintenance documentation when screening was incomplete due to low-quality images. This required an iterative approach with ongoing modifications, illustrating the importance of health IT engagement after initial implementation.
Finally, we learned the importance of ongoing monitoring and evaluation. Monthly meetings generated opportunities to discuss site-specific issues, identify additional training needs, and provide motivation. For example, a “March Madness” competition helped increase screening volume in spring 2021. Similarly, tracking the proportion of low-quality images monthly detected an unanticipated increase, which helped identify a camera with a technical issue requiring repairs. Emphasizing the importance of the program and highlighting accomplishments helped maintain morale even when clinics were under significant strain, such as during COVID-19 surges.
Limitations
This program was implemented at a single academic center and therefore results may not be generalizable. The long-term impact cannot yet be measured given the relatively recent implementation. We did not measure provider or patient satisfaction with the program, which will be useful for future investigations. Finally, although we designed the informatics tools underlying this program with ease of use in mind, we did not formally measure usability. This reflects the origin of the program as an operational/clinical need rather than as a research study per se.
CONCLUSIONS
DR screening is continuing to expand beyond eye clinics and into primary care and other settings. The success of these expansion efforts will depend in part on health IT and clinical informatics tools to achieve integration with existing information systems and facilitate widespread adoption. Health IT plays an important role in implementing these programs and expanding DR screening efforts to improve patient outcomes.
Acknowledgments
The authors wish to thank the members of the UCSD Teleretina Committee for input and feedback throughout the process. The stakeholder committee included members from primary care, endocrinology, ophthalmology, optometry, clinical informatics, EHR analysts/information services, imaging IT specialists, the camera vendor, the UCSD Quality Department, and the UCSD Population Health Services Organization.
Author Affiliations: Viterbi Family Department of Ophthalmology and Shiley Eye Institute (SLB), Department of Biomedical Informatics (SLB, JC, CG, MM), and Department of Internal Medicine (QQ, MM, CT), University of California San Diego, La Jolla, CA.
Source of Funding: Dr Baxter has funding support from National Institutes of Health grant 1DP5OD029610 and an unrestricted departmental grant from Research to Prevent Blindness. The funders had no role in the design or conduct of the study.
Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (SLB, QQ, JC, CG, MM, CT); acquisition of data (SLB, QQ, JC, CG, MM, CT); analysis and interpretation of data (SLB, MM, CT); drafting of the manuscript (SLB); critical revision of the manuscript for important intellectual content (SLB, QQ, JC, CG, MM, CT); statistical analysis (SLB); provision of patients or study materials (SLB, QQ, MM, CT); administrative, technical, or logistic support (SLB, QQ, JC, CG, MM, CT); and supervision (SLB, MM, CT).
Address Correspondence to: Sally L. Baxter, MD, MSc, University of California San Diego, 9415 Campus Point Dr MC0946, La Jolla, CA 92093. Email: S1baxter@health.ucsd.edu.
REFERENCES
1. Zheng Y, Ley SH, Hu FB. Global aetiology and epidemiology of type 2 diabetes mellitus and its complications. Nat Rev Endocrinol. 2018;14(2):88-98. doi:10.1038/nrendo.2017.151
2. Faselis C, Katsimardou A, Imprialos K, Deligkaris P, Kallistratos M, Dimitriadis K. Microvascular complications of type 2 diabetes mellitus. Curr Vasc Pharmacol. 2020;18(2):117-124. doi:10.2174/1570161117666190502103733
3. Gregg EW, Li Y, Wang J, et al. Changes in diabetes-related complications in the United States, 1990-2010. N Engl J Med. 2014;370(16):1514-1523. doi:10.1056/NEJMoa1310799
4. Unnikrishnan R, Pradeepa R, Joshi SR, Mohan V. Type 2 diabetes: demystifying the global epidemic. Diabetes. 2017;66(6):1432-1442. doi:10.2337/db16-0766
5. Diabetic retinopathy. National Eye Institute. Accessed September 12, 2022. https://www.nei.nih.gov/learn-about-eye-health/eye-conditions-and-diseases/diabetic-retinopathy
6. Diabetes and vision loss. CDC. May 7, 2021. Accessed September 1, 2021. https://www.cdc.gov/diabetes/managing/diabetes-vision-loss.html
7. Lester H, Schmittdiel J, Selby J, et al. The impact of removing financial incentives from clinical quality indicators: longitudinal analysis of four Kaiser Permanente indicators. BMJ. 2010;340:c1898. doi:10.1136/bmj.c1898
8. Quality ID #117 (NQF 0055): diabetes: eye exam. CMS. Accessed September 1, 2021. https://qpp.cms.gov/docs/QPP_quality_measure_specifications/Claims-Registry-Measures/2019_Measure_117_MedicarePartBClaims.pdf
9. Diabetic retinopathy: communication with the physician managing ongoing diabetes care. CMS. Accessed September 12, 2022. https://cmit.cms.gov/cmit/#/FamilyView?familyId=208
10. Nguyen HV, Tan GSW, Tapp RJ, et al. Cost-effectiveness of a national telemedicine diabetic retinopathy screening program in Singapore. Ophthalmology. 2016;123(12):2571-2580. doi:10.1016/j.ophtha.2016.08.021
11. Rachapelle S, Legood R, Alavi Y, et al. The cost-utility of telemedicine to screen for diabetic retinopathy in India. Ophthalmology. 2013;120(3):566-573. doi:10.1016/j.ophtha.2012.09.002
12. Pasquel FJ, Hendrick AM, Ryan M, Cason E, Ali MK, Narayan KMV. Cost-effectiveness of different diabetic retinopathy screening modalities. J Diabetes Sci Technol. 2015;10(2):301-307. doi:10.1177/1932296815624109
13. Vujosevic S, Aldington SJ, Silva P, et al. Screening for diabetic retinopathy: new perspectives and challenges. Lancet Diabetes Endocrinol. 2020;8(4):337-347. doi:10.1016/S2213-8587(19)30411-5
14. Rein DB, Wittenborn JS, Zhang X, et al; Vision Cost-Effectiveness Study Group. The cost-effectiveness of three screening alternatives for people with diabetes with no or early diabetic retinopathy. Health Serv Res. 2011;46(5):1534-1561. doi:10.1111/j.1475-6773.2011.01263.x
15. Ogunyemi O, Terrien E, Eccles A, et al. Teleretinal screening for diabetic retinopathy in six Los Angeles urban safety-net clinics: initial findings. AMIA Annu Symp Proc. 2011;2011:1027-1035.
16. Zhang W, Nicholas P, Schuman SG, et al. Screening for diabetic retinopathy using a portable, noncontact, nonmydriatic handheld retinal camera. J Diabetes Sci Technol. 2017;11(1):128-134. doi:10.1177/1932296816658902
17. Shi L, Wu H, Dong J, Jiang K, Lu X, Shi J. Telemedicine for detecting diabetic retinopathy: a systematic review and meta-analysis. Br J Ophthalmol. 2015;99(6):823-831. doi:10.1136/bjophthalmol-2014-305631
18. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med. 2018;1:39. doi:10.1038/s41746-018-0040-6
19. Wolf RM, Channa R, Abramoff MD, Lehmann HP. Cost-effectiveness of autonomous point-of-care diabetic retinopathy screening for pediatric patients with diabetes. JAMA Ophthalmol. 2020;138(10):1063-1069. doi:10.1001/jamaophthalmol.2020.3190
20. Channa R, Wolf R, Abramoff MD. Autonomous artificial intelligence in diabetic retinopathy: from algorithm to clinical application. J Diabetes Sci Technol. 2021;15(3):695-698. doi:10.1177/1932296820909900
21. Wolf RM, Liu TYA, Thomas C, et al. The SEE study: safety, efficacy, and equity of implementing autonomous artificial intelligence for diagnosing diabetic retinopathy in youth. Diabetes Care. 2021;44(3):781-787. doi:10.2337/dc20-1671
22. Grzybowski A, Brona P, Lim G, et al. Artificial intelligence for diabetic retinopathy screening: a review. Eye (Lond). 2020;34(3):451-460. doi:10.1038/s41433-019-0566-0
23. Lee AY, Yanagihara RT, Lee CS, et al. Multicenter, head-to-head, real-world validation study of seven automated artificial intelligence diabetic retinopathy screening systems. Diabetes Care. 2021;44(5):1168-1175. doi:10.2337/dc20-1877
24. Scanlon PH, Provins EK, Craske S, et al. Updating diabetic retinopathy screening lists using automatic extraction from GP patient records. J Med Screen. 2013;20(3):111-117. doi:10.1177/0969141313505747
25. Melles RB, Conell C, Siegner SW, Tarasewicz D. Diabetic retinopathy screening using a virtual reading center. Acta Diabetol. 2020;57(2):183-188. doi:10.1007/s00592-019-01392-9
26. Gardner RL, Cooper E, Haskell J, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26(2):106-114. doi:10.1093/jamia/ocy145
27. Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med. 2017;15(5):419-426. doi:10.1370/afm.2121
28. Kroth PJ, Morioka-Douglas N, Veres S, et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open. 2019;2(8):e199609. doi:10.1001/jamanetworkopen.2019.9609
29. Colicchio TK, Cimino JJ, Del Fiol G. Unintended consequences of nationwide electronic health record adoption: challenges and opportunities in the post-Meaningful Use era. J Med Internet Res. 2019;21(6):e13313. doi:10.2196/13313
30. Hatef E, Vanderver BG, Fagan P, Albert M, Alexander M. Annual diabetic eye examinations in a managed care Medicaid population. Am J Manag Care. 2015;21(5):e297-e302.
31. Ellis MP, Bacorn C, Luu KY, et al. Cost analysis of teleophthalmology screening for diabetic retinopathy using teleophthalmology billing codes. Ophthalmic Surg Lasers Imaging Retina. 2020;51(5):S26-S34. doi:10.3928/23258160-20200108-04
32. Jamali S, Abrishami M, Lashay A, et al. Comparison of portable cameras for diabetic retinopathy community screening. J Diabetes Sci Technol. 2021;15(1):201-202. doi:10.1177/1932296820929357
33. Wintergerst MWM, Mishra DK, Hartmann L, et al. Diabetic retinopathy screening using smartphone-based fundus imaging in India. Ophthalmology. 2020;127(11):1529-1538. doi:10.1016/j.ophtha.2020.05.025
34. Xiao B, Liao Q, Li Y, et al. Validation of handheld fundus camera with mydriasis for retinal imaging of diabetic retinopathy screening in China: a prospective comparison study. BMJ Open. 2020;10(10):e040196. doi:10.1136/bmjopen-2020-040196
35. Rajalakshmi R, Prathiba V, Arulmalar S, Usha M. Review of retinal cameras for global coverage of diabetic retinopathy screening. Eye (Lond). 2021;35(1):162-172. doi:10.1038/s41433-020-01262-7
36. Nittas V, Lun P, Ehrler F, Puhan MA, Mütsch M. Electronic patient-generated health data to facilitate disease prevention and health promotion: scoping review. J Med Internet Res. 2019;21(10):e13320. doi:10.2196/13320
37. Cahn A, Akirov A, Raz I. Digital health technology and diabetes management. J Diabetes. 2018;10(1):10-17. doi:10.1111/1753-0407.12606
38. Abdolkhani R, Gray K, Borda A, DeSouza R. Patient-generated health data management and quality challenges in remote patient monitoring. JAMIA Open. 2019;2(4):471-478. doi:10.1093/jamiaopen/ooz036
39. Savoy M. IDx-DR for diabetic retinopathy screening. Am Fam Physician. 2020;101(5):307-308.
40. Lee AY, Campbell JP, Hwang TS, Lum F, Chew EY; American Academy of Ophthalmology. Recommendations for standardization of images in ophthalmology. Ophthalmology. 2021;128(7):969-970. doi:10.1016/j.ophtha.2021.03.003
41. Baxter SL, Lee AY. Gaps in standards for integrating artificial intelligence technologies into ophthalmic practice. Curr Opin Ophthalmol. 2021;32(5):431-438. doi:10.1097/ICU.0000000000000781
42. Tran B, Lenhart A, Ross R, Dorr DA. Burnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proc. 2019;2019:136-144.
43. Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc. 2020;27(4):531-538. doi:10.1093/jamia/ocz220
44. Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform. 2017;8(3):686-697. doi:10.4338/ACI-2017-01-RA-0003
45. Khairat S, Coleman C, Ottmar P, Jayachander DI, Bice T, Carson SS. Association of electronic health record use with physician fatigue and efficiency. JAMA Netw Open. 2020;3(6):e207385. doi:10.1001/jamanetworkopen.2020.7385
46. Read-Brown S, Hribar MR, Reznick LG, et al. Time requirements for electronic health record use in an academic ophthalmology center. JAMA Ophthalmol. 2017;135(11):1250-1257. doi:10.1001/jamaophthalmol.2017.4187
47. Goldstein IH, Hribar MR, Reznick LG, Chiang MF. Analysis of total time requirements of electronic health record use by ophthalmologists using secondary EHR data. AMIA Annu Symp Proc. 2018;2018:490-497.
48. Gali HE, Baxter SL, Lander L, et al. Impact of electronic health record implementation on ophthalmology trainee time expenditures. J Acad Ophthalmol. 2019;11(2):e65-e72. doi:10.1055/s-0039-3401986
49. Baxter SL, Gali HE, Chiang MF, et al. Promoting quality face-to-face communication during ophthalmology encounters in the electronic health record era. Appl Clin Inform. 2020;11(1):130-141. doi:10.1055/s-0040-1701255