Abstract

Objective. The COVID-19 pandemic prompted unprecedented expansion of telemedicine services. We sought to describe clinician experiences providing telemedicine to publiclyinsured, lowincome patients during COVID-19. Methods. Online survey of ambulatory clinicians in an urban safetynet hospital system, conducted May 28, 2020–July 14, 2020. Results. Among 311 participants (response rate 48.3%), 34.7% (n=108/311) practiced in primary/urgent care, 37.0% (n=115/311) medical specialty, and 7.7% (n=24/311) surgical clinics. A large majority (87.8%, 273/311) had conducted telephone visits, 26.0% (81/311) video. Participants reported observing both technical and nontechnical patient barriers. Clinicians reported concerns about the diagnostic safety of telephone (58.9%, 129/219) vs. video (35.3%, 24/68). However, clinician comfort with telemedicine was high for telephone (89.3%, 216/242) and for video (91.0%, 61/67), with many clinicians (92.1%, 220/239 telephone; 90.9%, 60/66 video) planning to continue telemedicine after COVID-19. Conclusions. Clinicians in a safetynet health care system report great comfort with and intention to continue telemedicine after the pandemic, despite safety concerns and patient challenges.

Key words

Telemedicine, safety-net hospitals, health care delivery, ambulatory care, vulnerable populations [End Page 220]

The coronavirus SARS-CoV-2 (COVID-19) pandemic compelled health care networks across the United States to rapidly expand telemedicine services.1,2 Telemedicine "seeks to improve a patient's health by permitting two-way, real time interactive communication between the patient, and the physician or practitioner at the distant site."3 Telemedicine provides means to conduct remote clinical visits, enabling decreased COVID-19 exposure risk. Remote approaches to clinical care are effective for the care of chronic medical and behavioral conditions4,5 and have high patient satisfaction;6,7 video visits have perceived quality equivalent to in-person visits.8

Despite their favorable attributes, many payors have not previously reimbursed telemedicine services,9 a major implementation barrier for safety-net health networks caring for Medicaid-and Medicare-covered patients.1012 On March 27, 2020, the Coronavirus Aid, Relief, and Economic Security Act was signed into law, authorizing federally qualified health centers to furnish telehealth services to Medicare beneficiaries.13 Many state health agencies also authorized Medicaid payments for telemedicine services,14 including video and telephone visits. With these policy changes, institutions serving Medicare and Medicaid populations expanded their use of telemedicine.15 For example, New York City Health + Hospitals went from 500 billable televisits in February 2020 to nearly 83,000 in April 2020;16 the Ohio State University Wexner Medical Center's family medicine programs' telemedicine volume grew from 4% to 92.5%.2

The urgency of the pandemic and shelter-in-place orders, along with shifts in reimbursement policy, propelled safety-net site clinicians to rapidly adapt to provide telemedicine care, often with little advance preparation or prior experience with this care modality. Clinicians are front-line stakeholders of telemedicine care and are key determinants of its implementation.17 As social distancing protocols and the need for telemedicine persist, clinician experience will be a key factor in ensuring overall care quality and outcomes.18 However, little research has queried clinician telemedicine experience in the safety net.

Safety-net care settings serving low-income, ethnically diverse populations are unique. Patients from vulnerable populations can benefit from some of the advantages of telehealth, such as reduced transportation barriers and enhanced convenience of appointments to assist patients in not missing work shifts.19 However, safety-net patients may have limited English proficiency (LEP), limited technology literacy, and barriers accessing smartphones and cellular or broadband service.2023 These barriers may make equitable telemedicine implementation particularly challenging in the LEP safety net.24

We sought to understand clinician experience in the safety net during implementation of telemedicine in the early phase of the COVID-19 pandemic. Highlighting frontline perspectives will be key for advancing telemedicine quality for safety-net populations, as well as long-term telemedicine sustainability.

Methods

Study setting

Our setting is a large, urban safety-net hospital network, serving publicly-insured and low-income county residents. Our patient population is ethnically and linguistically diverse, with varying health literacy25,26 and digital literacy levels27 (see Table 1 for detailed patient population demographic characteristics). Of county [End Page 221]

Table 1. PATIENT DEMOGRAPHICS OF STUDY SETTING
Click for larger view
View full resolution
Table 1.

PATIENT DEMOGRAPHICS OF STUDY SETTING

residents, 42.3% speak a language other than English at home and 19.1% have limited English proficiency.28

Our study population included clinicians (physicians, nurse midwives, nurse practitioners, physician assistants, behavioral health clinicians, and others) providing ambulatory care services in the health care network. We excluded residents and Accreditation Council for Graduate Medical Education fellows, as they rotate through different health care delivery systems and have a different telemedicine experience from staff and faculty clinicians.

The network officially began providing telemedicine care, primarily by telephone, on March 3, 2020, anticipating COVID-19 precautions. Prior to this date, telephone and video clinical visits were performed sporadically, depending on patient preference and clinician initiative. The official shelter-in-place mandate for the county began on March 17, 2020. An official electronic health record video encounter type and documentation workflow became available on June 15, 2020. The video visit software [End Page 222] endorsed by the network was Zoom (San Jose, California). Patients were required to download the Zoom mobile application on a mobile phone or tablet, or open a link on their computer device, and would receive a meeting ID from the clinical team to begin a video visit encounter. Zoom is usable by Windows, Apple, iOS, and Android devices. Other clinicians informally used Doximity (San Francisco, California) or other personal social media video platforms, although this was not sanctioned by the network. Each clinic received basic, introductory training materials on video visits using Zoom, and clinician-and patient-facing websites to provide standardized guidance and to support onboarding. However, no system-wide infrastructure or device purchase occurred, and no system-wide standard work flow was implemented.

Outcomes and data collection

We conducted an online, anonymous quality-improvement survey through Qualtrics software (Provo, Utah), provided via a secure, university-based database (see online Appendix for full survey). We compiled validated instrument questions adapted from the Consolidated Framework for Implementation Research (CFIR),28 pre-existing telemedicine assessment tools,2931 and front-line stake-holder recommendations. Study outcomes were to assess observed patient barriers (CFIR construct: outer setting, patient needs32), satisfaction and comfort with telemedicine,33 and intention to use telemedicine in the future by choice.30 We also developed a novel item to assess perceptions of telemedicine safety: "Compared with in-person visits, I'm concerned about the safety of telemedicine because of increased risk of missed or delayed diagnosis." Survey items were Likert-scale with optional free-text comments asking what had gone well with the telemedicine visits, what had not gone well, and option to share additional perspectives and comments. As this survey was to inform quality improvement, the study was exempt from institutional review board approval.34 We distributed the survey via email, and department-specific study champions provided reminders and encouragements to participate. We collected responses from May 28, 2020 to July 14, 2020.

Analysis

We dichotomized Likert-scale questions into binary variables. For example, for the item assessing clinician comfort providing telemedicine, we combined "very comfortable" and "somewhat comfortable" into "comfortable". To assess if outcomes differed by clinician specialty, we combined specialties into "primary/urgent care," "medical specialty," and "surgical specialty." We conducted cross-tabulations of key metrics and chi-squared analysis of dichotomized variables across clinician specialties using Stata 13.1 (College Station, Texas). We reviewed free-text comments and provided exemplars based on a narrative review of free-text comments (led by AES). Our study rationale was to provide a clear picture of the behaviors and perceptions of clinicians across specialty. Because it was cross-sectional, and there are few prior data on clinician telemedicine perceptions to guide a multivariate analysis, we elected to report key outcomes and specialty-based comparisons only. Because we were interested in clinicians' direct experiences with these modalities rather than their beliefs or perceptions, we restricted analysis of telephone visits to those who reported on average at least one telephone visit per half-day clinic session in the prior month, and restricted analysis of video visits to providers who reported conducting at least one video visit on average per half-day session in the prior month. [End Page 223]

Results

We had 311 final respondents, out of an eligible 643, for a response rate of 48.3%. Among participants, 37.0% (n=115) were medical specialists, 34.7% (n=108) were primary or urgent care clinicians, and 7.7% (n=24) were surgical specialists. Demographic characteristics of survey participants are listed in Table 2. A majority (57.2%, n=178) conducted four or more telephone visits per half-day session. Only 26.0% (n=81) conducted one or more video visits per half-day session. Below, we report summary statistics of key outcomes as well as exemplar quotations in italics from free-text responses that illustrate results. Additional free-text quotations are available in Box 1. There were no statistically significant differences in key outcome measures by clinician specialty.

We asked clinicians to select all applicable patient challenges that they had directly observed when conducting telemedicine visits over telephone and video (see Figure 1 for compilation of technical and non-technical barriers). The most common barriers for telephone visits included: speech, hearing, or cognitive barriers (44.1%; example: "Patients that are hard of hearing are almost impossible to communicate with, very challenging to reach some patients by phone."), communication quality (43.7%), and lack of having a phone (37.6%). The most common barriers for video visits included trouble using mobile applications (39.5%), lack of video (38.6%), lack of knowledge or skills to participate in the visit (37.9%; example: "Difficulty setting up video access due to language and educational barriers."), and lack of Internet (35.0%).

The majority of clinicians (129/219, 58.9%, see Figure 2) conducting telephone visits agreed that they had concerns about the safety of telephone visits, with regard to a missed or delayed diagnosis. However, only 35.3% (24/68) of clinicians doing video visits had diagnostic safety concerns. Diagnostic safety concerns related to inability to gain objective vital signs or physical exam findings: "There are patients, particularly those with heart failure, who just physically need to be seen to evaluate them (volume status in particular). I'm really flying blind, and though I'm trying my best, there's a good chance I'll be wrong and cause harm."

Despite patient challenges and safety concerns, clinician comfort with telemedicine services was high (Figure 3). The majority (216/242, 89.3%) expressed being comfortable with telephone visits: "I think there is a lot we can do over the phone. It was a good way to show patients that we care about them and believe in the importance of sheltering in place. There is a lot that absolutely works over the phone." For video visits, 91.0% (61/67) endorsed comfort conducting video visits. Along with comfort, satisfaction was also high; 72.3% (47/65) were satisfied with telephone visits and 89.2% (58/65) with video visits (Figure 4): "Providing information for results and following up on an in-person visit is satisfying to do over telephone."

Clinicians predicted high likelihood of continuing telemedicine by choice once clinic operations had returned to normal (see Figure 5); 92.1% (220/239) for telephone visits and 90.1% (60/66) for video visits: "I would love to be able to continue telemedicine but it depends on the department of public health's decisions about our clinic. It saves patients and staff lot of time and hassle (commuting, childcare, parking). I think it's great for visits that don't need an exam." [End Page 224]

Table 2. DEMOGRAPHICS OF CLINICIAN SURVEY RESPONDENTS
Click for larger view
View full resolution
Table 2.

DEMOGRAPHICS OF CLINICIAN SURVEY RESPONDENTS

[End Page 226]

No description available
Click for larger view
View full resolution

[End Page 228]

Figure 1. Clinician-observed patient barriers to telemedicine care in the safety-net (a) technical challenges and (b) non-technical challenges (N=311).a Note: aPercentages are proportion of total survey participants (N=311) who indicated they had observed these challenges.
Click for larger view
View full resolution
Figure 1.

Clinician-observed patient barriers to telemedicine care in the safety-net (a) technical challenges and (b) non-technical challenges (N=311).a

Note:

aPercentages are proportion of total survey participants (N=311) who indicated they had observed these challenges.

Discussion

To our knowledge, ours is the first assessment of the early experience implementing telemedicine across a multispecialty network of safety-net clinicians during COVID-19. Prior to COVID, assessments of clinician perceptions of telemedicine found high satisfaction and equivalent quality with in-person visits.8 Other surveys found clinician resistance to telemedicine implementation due to reimbursement, liability, and technical concerns.17 Since the pandemic, a survey of gastroenterologists found 88% rated [End Page 229]

Figure 2. Clinician safety concerns regarding telemedicine by specialty.
Click for larger view
View full resolution
Figure 2.

Clinician safety concerns regarding telemedicine by specialty.

video visits as better than or as good as face-to-face, with only 41% rating telephone visits as better than or as good as in-person visits.35 Another post-COVID study found clinician satisfaction with telemedicine was correlated with stated likelihood to continue telemedicine; other predictors were perceived ease of conducting a telemedicine physical exam and a flexible personality style.36 A study of primarily medical specialties found high likelihood to continue telemedicine but lower satisfaction with telephone visits than video.37 These prior surveys have not highlighted the safety net, nor safety net-specific patient barriers.

The telemedicine explosion of rapid implementation, without dedicated resources or standard work, is common to many safety-net settings that disproportionately care for low-income populations.1,28,38 Consistent with prior studies, we found that telemedicine implementation for a lower-resource patient population has been delivered primarily [End Page 230]

Figure 3. Clinician comfort with providing telemedicine care by specialty.
Click for larger view
View full resolution
Figure 3.

Clinician comfort with providing telemedicine care by specialty.

by telephone, although video visits are being piloted.16,39 This pattern is likely related to the higher complexity of technical infrastructure and clinic workflows required to support safety-net patients to successfully participate in video visits compared with phone visits. In the absence of team-based virtual workflows for video visits, particularly at lower-resourced health care sites, clinicians are required to instruct patients on how to install and use video software during their visits, cutting into the time to provide medical care. Best practices for transforming the health care systems, such as the use of standard work and teams, should be brought to bear on telemedicine practice to enable video visit implementation.28

Both technical and non-technical challenges faced by patients in the safety net may contribute to clinician concerns about reduced access for underserved patients. [End Page 231]

Figure 4. Clinician satisfaction with telemedicine care compared with in-person visits by specialty.
Click for larger view
View full resolution
Figure 4.

Clinician satisfaction with telemedicine care compared with in-person visits by specialty.

Clinicians raised equity-related concerns about patient speech, hearing, or cogn30ive ability, access to a reliable phone number, video visit capacity at home (which requires a smartphone and Internet), reliable access to language interpreters, and patient education regarding expectations for a telemedicine visit. These barriers are closely linked to health care disparities based on disability, socioeconomic status, and educational attainment, and merit close attention for equitable implementation of telemedicine.25,4144

We can extrapolate strategies to overcome such barriers from research on increasing use of online patient portals for vulnerable populations as well as telemedicine literature. For barriers accessing devices, programs to improve device access increase portal utilization45 and are recommended for telemedicine.46 For barriers accessing data, low-cost or free access to high-quality broadband is a key component of telemedicine [End Page 232]

Figure 5. Clinician reported likelihood to continue telemedicine after COVID-19 by specialty.
Click for larger view
View full resolution
Figure 5.

Clinician reported likelihood to continue telemedicine after COVID-19 by specialty.

equity given neighborhood-based disparities.22,47 For barriers in digital literacy, robust evidence exists for interventions to support individual-level patient preparation for portal access, which is likely to extend to telemedicine as well.4852 For limited English proficiency patients, dissemination of best practices for interpreter services must be standardized.24 Finally, the design of telemedicine platforms and instructional tools must engage patients and stakeholders from vulnerable populations, applying user-centered design or other participatory methods.53 Our group has created an online toolkit of examples and resources to support safety-net institutions seeking to overcome some of these barriers.54 [End Page 233]

Over half of respondents had concerns about the diagnostic safety of telephone visits. Free-text comments shared the concerns of relying on a telephone visit when objective vital signs or exam findings are required, considering diagnoses of higher urgency such as acute abdominal pain, or privacy concerns limiting diagnostic information gathered. This can be compounded by language barriers or limited health literacy, particularly among patients in the safety net. The current evidence base suggests that telemedicine video encounters have diagnostic accuracy roughly equivalent to in-person visits,55,56 but it is unknown if this holds for telephone-based encounters or for safety-net populations. Additional resources to train providers on how to build diagnostic confidence via telemedicine are needed. Payor reimbursement for home self-monitoring devices, such as home blood pressure monitors, scales, and pulse oximeters for qualifying co-morbidities, is another strategy to improve clinician confidence in safe, remote diagnosis and patient self-monitoring.57

Despite safety concerns and challenges, there was great interest in continuing to use telemedicine in the future by choice, after the COVID-19 pandemic has subsided. Our interpretation of the high likelihood to continue, despite safety concerns, is that the overall benefits of patient and staff convenience and satisfaction outweigh the potential risks. This is a promising finding, signifying that stakeholders are engaged in championing this service long-term. Since many state Medicaid programs model their Medicaid policy after Medicare, sustained Medicare telemedicine policy is key to shifting Medicaid nationwide. Moreover, as many safety-net patients do not have reliable video access, reimbursement for telephone-based visits will continue to be a priority to prevent widening the digital divide. Medicaid, Medicare, and other safety-net insurers must establish reimbursement policies that facilitate long-term financial sustainability (or incentives) for provision of telemedicine.58

Limitations

While our response rate was 48.3%, we had anticipated a 30–40% response rate for a non-compensated, voluntary survey conducted during a time of crisis; our response rate surpassed our expectations given this context. Clinicians who chose to participate in the survey may differ from those who did not; for example, participants with more enthusiasm about telemedicine may have elected to participate in the survey. We lack detailed demographic data for all eligible clinicians in the network to compare responders with non-responders. We do not have data on whether patient characteristics, including language preference, differed by type of telemedicine visit or clinical specialty type. We do not know if factors such as amount of interpreter use or prior telemedicine experience may have affected some of the clinician ratings of their experience conducting telemedicine. Some respondents did not complete every question of this survey, and many did not disclose their specialty, leading to varying total denominators for each survey item and clinical specialty subgroup. Based on the survey structure, there may have been participants with less than one telemedicine visit per session on average (which were categorized as zero); our results are therefore generalizable only to clinicians who conduct at least one telemedicine encounter per half-day or more. Finally, as telemedicine was not widely implemented prior to COVID-19, we could not compare changes in utilization or perceptions pre-and post-pandemic. However, study strengths include a diverse range of clinician participants who practice in a safety-net setting and our ability to capture front line perspectives at a health care site relatively early (in the first six months) in telemedicine implementation. [End Page 234]

Conclusions

Our survey describes the realities of the rapidly changing ambulatory landscape during COVID-19. Safety-net clinicians are facing multiple patient barriers to engagement in telemedicine care during this first wave of the COVID-19 pandemic. However, they also appreciate the benefits for their patients and attest high interest in continuing telemedicine, despite concerns about safety and equitable access. Safety-net patients require access to devices, data, interpreter services, and technical support in order to participate equitably in telemedicine. Provider-level supports are needed to ensure sustainability, promote safety of care, and improve satisfaction to prevent long-term burnout or adverse care outcomes. Best practices tailored for the safety net will be key to building capacity if telemedicine is to remain for the long term.

Supplemental Material

Online Appendix

Anjana E. Sharma, Elaine C. Khoong, Malini A. Nijagal, Courtney R. Lyles, George Su, Triveni DeFries, Urmimala Sarkar, and Delphine Tuot

ANJANA E. SHARMA, is affiliated with the Center for Excellence in Primary Care, Dept of Family and Community Medicine, UCSF (University of California San Francisco) School of Medicine and the UCSF Center for Vulnerable Populations at Zuckerberg San Francisco General Hospital (CVP). ELAINE C. KHOONG, COURTNEY R. LYLES, URMIMALA SARKAR, and DELPHINE TUOT are affiliated with the Division of General Internal Medicine, UCSF School of Medicine, and CVP. MALINI A. NIJAGAL is affiliated with the Department of Obstetrics, Gynecology and Reproductive Sciences, ZSFG and UCSF. GEORGE SU is affiliated with the Division of Pulmonary and Critical Care, ZSFG and the Department of Medicine, UCSF School of Medicine.

Please address all correspondence to: Anjana Sharma, Center for Excellence in Primary Care, Dept of Family and Community Medicine, UCSF School of Medicine, 995 Potrero Ave, Ward 83, San Francisco, CA 94110; Phone: 617-945-4776; Email: Anjana.sharma@ucsf.edu.

Acknowledgments

The authors thank the clinicians who took the time to complete this survey. We greatly thank Kristan Olazo for assistance with image and reference formatting. We thank Christina Yong for proofreading the manuscript.

Grant Support

Author 1 (Dr. Sharma) is supported by the National Center for Advancing Translational Sciences of the NIH under Award Number KL2TR001870.

Author 2 (Dr. Khoong) is supported by the National Heart Lung and Blood Institute of the NIH under Award Number K12HL138046 and National Center for Advancing Translational Sciences of the NIH under Award Number KL2TR001870.

Authors 3 and 4 (Dr. Hijagal and Dr. Lyles) are supported by a grant from the Donaghue Foundation's Greater Value Portfolio.

Author 5 (Dr. Su) is supported by NIH R01 DK117953 and NIH R01 HL143366-01A1 and P0051172 from the California HealthCare Foundation.

Author 7 (Dr. Sarkar) is supported by NIH/NCI K24CA212294.

Author 8's (Dr. Tuot) work is partially supported by the Connected Care Accelerator Program, grant #TF2007-09373 from the Center for Care Innovations, Tides Foundation and California Health Care Foundation.

Authors 1, 4, and 7 (Dr. Sharma, Dr. Lyles, and Dr. Sarkar) are supported by the Commonwealth Fund (20202842).

This manuscript's contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH or the Commonwealth Fund. The authors have no other disclosures or conflicts of interest to declare.

References

1. Mann DM, Chen J, Chunara R, Testa PA, Nov O. COVID-19 transforms health care through telemedicine: evidence from the field. J Am Med Inform Assoc. 2020 Jul 1;27(7):1132–1135. https://doi.org/10.1093/jamia/ocaa072 PMid:32324855 PMCid:PMC7188161
2. Olayiwola JN, Magaña C, Harmon A, et al. Telehealth as a bright spot of the COVID-19 pandemic: recommendations from the virtual frontlines ("frontweb"). JMIR Public Health Surveill. 2020 Jun 25;6(2):e19045. https://doi.org/10.2196/19045 PMid:32479413 PMCid:PMC7318864
3. Medicaid.gov. Telemedicine. Baltimore, MD: Centers for Medicare and Medicaid Services, 2020. Available at: https://www.medicaid.gov/medicaid/benefits/telemedicine/index.html.
4. Su D, Zhou J, Kelley MS, et al. Does telemedicine improve treatment outcomes for diabetes? A meta-analysis of results from 55 randomized controlled trials. Diabetes Res Clin Pract. 2016 Jun;116:136–148. https://doi.org/10.1016/j.diabres.2016.04.019 PMid:27321329
5. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inform. 2010 Nov;79(11):736–771. https://doi.org/10.1016/j.ijmedinf.2010.08.006 PMid:20884286
6. Mair F, Whitten P. Systematic review of studies of patient satisfaction with telemedicine. BMJ. 2000 Jun 3;320(7248):1517–1520. https://doi.org/10.1136/bmj.320.7248.1517 PMid:10834899 PMCid:PMC27397
7. Kruse CS, Krowski N, Rodriguez B, et al. Telehealth and patient satisfaction: a systematic review and narrative analysis. BMJ Open. 2017 Aug 3;7(8):e016242. https://doi.org/10.1136/bmjopen-2017-016242 PMid:28775188 PMCid:PMC5629741
8. Donelan K, Barreto EA, Sossong S, et al. Patient and clinician experiences with telehealth for patient follow-up care. Am J Manag Care. 2019 Jan;25(1):40–44.
9. Bajowala SS, Milosch J, Bansal C. Telemedicine pays: billing and coding update. Curr Allergy Asthma Rep. 2020 Jul 27;20(10):60. https://doi.org/10.1007/s11882-020-00956-y PMid:32715353 PMCid:PMC7382992
10. Rubin R. COVID-19's crushing effects on medical practices, some of which might not survive. JAMA. 2020 Jul 28;324(4):321. https://doi.org/10.1001/jama.2020.11254 PMid:32556122
11. Jaklevic MC. Telephone visits surge during the pandemic, but will they last? JAMA. 2020 Oct 7;324(16):1593–1595. https://doi.org/10.1001/jama.2020.17201 PMid:33027503
12. Center for Connected Health Policy (CCHP). State telehealth laws and reimbursement policies. West Sacramento, CA: CCHP, Public Health Institute, 2020. Available at: https://www.cchpca.org/sites/default/files/2020-05/CCHP_%2050_STATE_REPORT_SPRING_2020_FINAL.pdf.
13. New and expanded flexibilities for rural health clinics (RHCs) and federally qualified health centers (FQHs) during the COVID-19 Public Health Emergency (PHE). Baltimore, MD: Centers for Medicare and Medicaid Services, 2020. Available at: https://www.cms.gov/regulations-and-guidanceguidancetransmittals2020-transmittals/se20016.
14. Lightbourne W, Newsom G. Medi-Cal payment for telehealth and virtual/telephonic communications relative to the 2019-Novel Coronavirus (COVID-19). Sacramento, CA: Department of Health Care Services, 2020. Available at: https://www.dhcs.ca.gov/Documents/COVID-19/Telehealth_Other_Virtual_Telephonic_Communications_V4.0.pdf.
15. Verma S. Early impact of CMS expansion of Medicare telehealth during COVID-19. Bethesda, MD: Health Affairs, 2020 July 15. Available at: https://www.healthaffairs.org/do/10.1377/hblog20200715.454789/abs/.
16. Lau J, Knudsen J, Jackson H, et al. Staying connected in the COVID-19 pandemic: telehealth at the largest safety-net system in the United States. Health Aff. 2020 Jun 11;39(8):1437–1442. https://doi.org/10.1377/hlthaff.2020.00903 PMid:32525705
17. Whitten P, Holtz B. Provider utilization of telemedicine: the elephant in the room. Telemed J E Health. 2008 Nov;14(9):995–997. https://doi.org/10.1089/tmj.2008.0126 PMid:19035815
18. Bodenheimer T, Sinsky C. From Triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014 Nov–Dec;12(6):573–6. https://doi.org/10.1370/afm.1713 PMid:25384822 PMCid:PMC4226781
19. Reed ME, Huang J, Graetz I, et al. Patient characteristics associated with choosing a telemedicine visit vs office visit with the same primary care clinicians. JAMA Netw Open. 2020 Jun 1;3(6):e205873. https://doi.org/10.1001/jamanetworkopen.2020.5873 PMid:32585018 PMCid:PMC7301227
20. Chesser A, Burke A, Reyes J, Rohrberg T. Navigating the digital divide: a systematic review of eHealth literacy in underserved populations in the United States. Inform Health Soc Care. 2016;41(1):1–19. https://doi.org/10.3109/17538157.2014.948171 PMid:25710808
21. Khoong, E.C., Butler, B.A., Mesina, O., Su, G., DeFries, T.B., Nijagal, M. and Lyles, C.R., 2021. Patient interest in and barriers to telemedicine video visits in a multilingual urban safety-net system. Journal of the American Medical Informatics Association, 28(2):349–353.
22. Perzynski AT, Roach MJ, Shick S, et al. Patient portals and broadband internet inequality. J Am Med Inform Assoc. 2017 Sep 1;24(5):927–https://doi.org/10.1093/jamia/ocx020 PMid:28371853 PMCid:PMC6259664
23. Rodriguez JA, Lipsitz SR, Lyles CR, et al. Association between patient portal use and broadband access: a national evaluation. J Gen Intern Med. 2020 Dec;35(12): 3719–20. https://doi.org/10.1007/s11606-020-05633-4 PMid:31925739
24. Nouri S, Khoong EC, Lyles CR, et al. Addressing equity in telemedicine for chronic disease management during the Covid-19 pandemic. Waltham, MA: NEJM Catalyst (E-Journal), Massachusetts Medical Society, 2020 May 4. Available at: https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0123.
25. Bailey SC, O'Conor R, Bojarski EA, et al. Literacy disparities in patient access and health-related use of Internet and mobile technologies. Health Expect. 2015 Dec;18(6):3079–87. https://doi.org/10.1111/hex.12294 PMid:25363660 PMCid:PMC4417455
26. Sarkar U, Schillinger D, Lopez A, Sudore R. Validation of self-reported health literacy questions among diverse English and Spanish-speaking populations. J Gen Intern Med. 2011 Mar;26(3):265–271. https://doi.org/10.1007/s11606-010-1552-1 PMid:21057882 PMCid:PMC3043178
27. Sarkar U, Gourley GI, Lyles CR, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med. 2016 Dec;31(12):1417–26. https://doi.org/10.1007/s11606-016-3771-6 PMid:27418347 PMCid:PMC5130945
28. United States Census Bureau. Selected characteristics of the native and foreign-born population—San Francisco. Suitland-Silver Hill, MD: United States Census Bureau, 2019. Available at: https://data.census.gov/cedsci/table?q=sanfrancisco&tid=ACSST1Y2019.S0501&hidePreview=false.
29. Damschroder L, Aron D, Keith R, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009 Aug 7;4:50. https://doi.org/10.1186/1748-5908-4-50 PMid:19664226 PMCid:PMC2736161
30. Langbecker D, Caffery LJ, Gillespie N, et al. Using survey methods in telehealth research: a practical guide. J Telemed Telecare. 2017 Oct;23(9):770–9. https://doi.org/10.1177/1357633X17721814 PMid:28728502
31. Parmanto B, Lewis AN, Graham KM, et al. Development of the telehealth usability questionnaire (TUQ). Int J Telerehabil. 2016 Jul 1;8(1):3–10. https://doi.org/10.5195/IJT.2016.6196 PMid:27563386 PMCid:PMC4985278
32. Venkatesh V, Davis F. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci. 2000;46:186–204. https://doi.org/10.1287/mnsc.46.2.186.11926
33. Consolidated Framework for Implementation Research (CFIR). Constructs. Ann Arbor, MI: CFIR, 2020. Available at: https://cfirguide.org/constructs/.
34. Bakken S, Grullon-Figueroa L, Izquierdo R, et al. Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. J Am Med Inform Assoc. 2006 Nov–Dec;13(6):660–7. https://doi.org/10.1197/jamia.M2146 PMid:16929036 PMCid:PMC1656962
35. University of California San Francisco (UCSF). Quality improvement (QI) and quality assurance (QA). San Francisco, CA: UCSF Institutional Review Board, 2019. Available at: https://irb.ucsf.edu/quality-improvement-qi-and-quality-assurance-qa.
36. Serper M, Nunes F, Ahmad N, et al. Positive early patient and clinician experience with telemedicine in an academic gastroenterology practice during the COVID-19 pandemic. Gastroenterology. 2020 Oct;159(4):1589–91.e4. https://doi.org/10.1053/j.gastro.2020.06.034 PMid:32565015 PMCid:PMC7301815
37. Miner H, Fatehi A, Ring D, et al. Clinician telemedicine perceptions during the COVID-19 pandemic. Telemed J E Health. 2020 Sep 18. https://doi.org/10.1089/tmj.2020.0295 PMid:32946364
38. Malhotra K, Sivaraman A, Regunath H. Coronavirus disease 2019 pandemic as catalyst for telemedicine adoption: a single-center experience. Telemed Rep. 2020;1(1):16–21. https://doi.org/10.1089/tmr.2020.0003
39. Uscher-Pines, Lori, Kathryn E. Bouskill, Jessica L. Sousa, Mimi Shen, and Shira H. Fischer, Experiences of Medicaid Programs and Health Centers in Implementing Telehealth. Santa Monica, CA: RAND Corporation, 2019. https://www.rand.org/pubs/research_reports/RR2564.html.
40. Uscher-Pines L, Sousa J, Jones M, et al. Telehealth Use Among Safety-Net Organizations in California During the COVID-19 Pandemic. JAMA. 2021;325(11):1106–1107. doi:10.1001/jama.2021.0282
41. Reichard A, Stolzle H, Fox MH. Health disparities among adults with physical disabilities or cognitive limitations compared to individuals with no disabilities in the United States. Disabil Health J. 2011;4(2):59–67. https://doi.org/10.1016/j.dhjo.2010.05.003 PMid:21419369
42. Jensen JD, King AJ, Davis LA, et al. Utilization of internet technology by low-income adults: the role of health literacy, health numeracy, and computer assistance. J Aging Health. 2010 Sep;22(6):804–826. https://doi.org/10.1177/0898264310366161 PMid:20495159
43. Sentell TL, Halpin HA. Importance of Adult Literacy in Understanding Health Disparities. J Gen Intern Med. 2006 Aug;21(8):862–6. https://doi.org/10.1111/j.1525-1497.2006.00538.x PMid:16881948 PMCid:PMC1831569
44. Weil AR. Pursuing Health Equity. Health Aff (Millwood). 2017 Jun 1;36(6):975. https://doi.org/10.1377/hlthaff.2017.0583 PMid:28583952
45. Graetz I, Huang J, Brand R, et al. Mobile-accessible personal health records increase the frequency and timeliness of PHR use for patients with diabetes. J Am Med Inform Assoc. 2019 Jan 1;26(1):50–4. https://doi.org/10.1093/jamia/ocy129 PMid:30358866 PMCid:PMC6308012
46. Khoong E. How the telemedicine boom threatens to increase inequities. Washington, DC: Association of American Medical Colleges (AAMC), 2020. Available at: https://www.aamc.org/news-insights/how-telemedicine-boom-threatens-increase-inequities.
47. Pew. How states are expanding broadband access. Washington, DC: Pew, 2020. Available at: https://pew.org/2HIJGAb.
48. Grossman LV, Masterson Creber RM, Benda NC, et al. Interventions to increase patient portal use in vulnerable populations: a systematic review. J Am Med Inform Assoc. 2019 Aug 1;26(8–9):855–70. https://doi.org/10.1093/jamia/ocz023 PMid:30958532 PMCid:PMC6696508
49. Lyles CR, Tieu L, Sarkar U, et al. A randomized trial to train vulnerable primary care patients to use a patient portal. J Am Board Fam Med. 2019 Mar–Apr;32(2):248–58. https://doi.org/10.3122/jabfm.2019.02.180263 PMid:30850461 PMCid:PMC6647853
50. Lyles CR, Nelson EC, Frampton S, et al. Using electronic health record portals to improve patient engagement: research priorities and best practices. Ann Intern Med. 2020;172(11 Suppl):S123–S129. https://doi.org/10.7326/M19-0876 PMid:32479176 PMCid:PMC7800164
51. Casillas A, Abhat A, Mahajan A, et al. Portals of change: how patient portals will ultimately work for safety net populations. J Med Internet Res. 2020;22(10):e16835. https://doi.org/10.2196/16835 PMid:33094732 PMCid:PMC7647808
52. Casillas A, Cemballi AG, Abhat A, et al. An untapped potential in primary care: semi-structured interviews with clinicians on how patient portals will work for caregivers in the safety net. J Med Internet Res. 2020;22(7):e18466. https://doi.org/10.2196/18466 PMid:32706709 PMCid:PMC7400036
53. Lyles C, Schillinger D, Sarkar U. Connecting the dots: health information technology expansion and health disparities. PLoS Med. 2015 Jul 14;12(7):e1001852. https://doi.org/10.1371/journal.pmed.1001852 PMid:26172977 PMCid:PMC4501812
54. Center for Care Innovations. Telemedicine for health equity: considerations for reaching and engaging diverse patients. Oakland, CA: Center for Care Innovations, 2020. Available at: https://www.careinnovations.org/resources/telemedicine-for-health-equity-considerations-for-reaching-and-engaging-diverse-patients/.
55. Ohta M, Ohira Y, Uehara T, et al. How accurate are first visit diagnoses using synchronous video visits with physicians? Telemed J E Health. 2017;23(2):119–29. https://doi.org/10.1089/tmj.2015.0245 PMid:27351424
56. Amadi-Obi A, Gilligan P, Owens N, et al. Telemedicine in pre-hospital care: a review of telemedicine applications in the pre-hospital environment. Int J Emerg Med. 2014 Jul 5;7:29. https://doi.org/10.1186/s12245-014-0029-0 PMid:25635190 PMCid:PMC4306051
57. Tucker KL, Sheppard JP, Stevens R, et al. Self-monitoring of blood pressure in hyper-tension: a systematic review and individual patient data meta-analysis. PLoS Med. 2017 Sep 19;14(9). https://doi.org/10.1371/journal.pmed.1002389 PMid:28926573 PMCid:PMC5604965
58. Kim J-H, Desai E, Cole MB. How the rapid shift to telehealth leaves many community health centers behind during the COVID-19 pandemic. Health Affairs. 2020 Jun 2. Available at: https://www.healthaffairs.org/do/10.1377/hblog20200529.449762/full/

Additional Information

ISSN
1548-6869
Print ISSN
1049-2089
Pages
220-240
Launched on MUSE
2021-05-01
Open Access
Yes
Back To Top

This website uses cookies to ensure you get the best experience on our website. Without cookies your experience may not be seamless.