publisher colophon

The use of value metrics is often dependent on payer-initiated health care management incentives. There is a need for practices to define and manage their own patient panels regardless of payer to participate effectively in population health management. A key step is to define a panel of primary care patients with high comorbidity profiles. Our sample included all patients seen in an urban academic family medicine clinic over a two-year period. The simplified risk stratification was built using internal electronic health record and billing system data based on ICD-9 codes. There were 347 patients classified as high-risk out of the 5,364 patient panel. Average age was 59 years (SD 15). Hypertension (90%), hyperlipidemia (62%), and depression (55%) were the most common conditions among high-risk patients. Simplified risk stratification provides a feasible option for our team to understand and respond to the nuances of population health in our underserved community.

Key words

Primary care, risk stratification, PCMH, community health management

Payment priorities in the United States (U.S.) health care system are changing from volume metrics to value metrics, which are driven by whole person outcomes.1 However, most primary care providers (PCPs) do not have access to the full spectrum of value metrics, including outcomes data on emergency department (ED) visits, hospital bed days, and medical costs23 for their entire panel across all payers. Instead, PCPs are often dependent on payer-initiated population health care management incentives, [End Page 202] with varied content and quality of data and a myriad of performance standards for population health management across payers.45 This can lead to practices implementing payer-specific metrics, programs, and information technologies (IT) even though a practice's patient panel is covered by a variety of payers.

In order to manage practice-level panels, rather than payer-specific sub-groups of panels, the primary care practice must develop its own information technology solutions and its own collaborative care management strategies. To ready our primary care practice for this shift in payment structure, we developed the Patient Centered Medical Home and Neighborhood Project (PCMHN). Addressing social complexities for patients in Neighborhood context adds the N (neighborhood) to the classic PCMH model of team-based care, panel based management, and whole person outcomes.67 The goal of the PCMHN model is to improve health outcomes and decrease costs for a high-risk/high-cost patient population.812 Implementation of this program required development of a cost-efficient risk stratification system based on internal data sources. In this paper, we describe our approach to developing and implementing this algorithm in our urban, underserved primary care clinic.

The Morehouse Healthcare Comprehensive Family Healthcare Center (CFHC) is an academic family medicine practice located in East Point, Georgia, a community in the Atlanta metropolitan area. The clinic serves a predominantly African American population, and both medical and social complexities are common among our patients. Thirty-five percent of patients have at least two comorbid chronic diseases. Patients in the clinic's panel are older, lower-income, and more likely to rent their home than other Fulton county residents. In Fulton County, 10 of the 11 leading causes of morbidity—including mental and behavioral disorders—are most prevalent in African American residents.13 Clinicians include faculty and resident primary care physicians.

The goal of this paper is to describe the development and application of a cost-effective risk stratification algorithm based solely on internal data, designed to target population health and care management strategies for panels of patients defined at the practice-level, regardless of payer.


Data source and study population

The cohort for the risk stratification algorithm and PCMHN program included patients seen at the CFHC. We included those with at least one visit since 2013 who were 18 years or older and resided in ZIP codes surrounding the clinic (30344, 30331, 30311, 30349, 30315, 30354, or 30310). Health information was obtained from the patient management system (IDX) and the internal electronic health record (Practice Partner 11.0, McKesson Technologies, 2014) database. A risk stratification algorithm based on diagnosis codes was developed to select a panel of patients for inclusion in the initial program.

Risk stratification algorithm

The risk stratification process was initiated with compilation of an exhaustive list of ICD-9 codes14 for every CHFC patient who met the inclusion criteria above. Elixhauser comorbidity index15 conditions and chronic health conditions (developed internally based on Elixhauser comorbidity index and adjusted with results from previous researches and target population1618) have been shown to [End Page 203] predict health outcomes.19 ICD-9 codes were used to indicate whether a patient had any health condition from these two groups (Table 1). Chronic behavioral conditions including depression, schizophrenia, dementia, and alcohol abuse were extracted separately. We calculated total number of conditions for each patient. Patients with at least one behavioral condition and two or more physical conditions were identified

Click for larger view
View full resolution
Table 1.


[End Page 204] as high-risk. Low-risk patients had no behavioral conditions and at most one physical condition. Others were categorized as medium-risk. We based the approach for developing this risk stratification approach on the well documented increased risk of morbidity and mortality associated with co-occurring chronic mental and physical health conditions.20,21

Our risk stratification algorithm was designed for widest applicability to any primary care practice with an electronic health record (EHR) system and an integrated or stand-alone patient registration/management information system (MIS), using the following a priori principles:

  1. 1. Algorithm uses only data available in the patient registration and electronic clinical records on the practice's own computers.

  2. 2. Algorithm uses only fixed field variables (not free text) from EHR clinical data (e.g., ICD-9 diagnosis codes, systolic blood pressure) and from MIS patient registration data (e.g., age, gender, and insurance status).

  3. 3. Project assumes that no data from hospital or health system utilization will be available.

  4. 4. Project assumes that no data from administrative claims or insurance data will be available, other than the billing submissions generated internally from the practice. Even when such data are available from payers, they are usually in a payer-specific format and not tightly integrated into the EHR dataset.

  5. 5. EHR is assumed not to have an automated population health management or panel-based care management module.

Although we generated risk stratification profiles through direct Structured Query Language (SQL) queries of the Oracle database within our electronic health record system, we used simple selection of variables, simple calculation of an unweighted Elixhauser comorbidity score, plus the addition of a behavioral health diagnosis variable, which clinicians had previously described as adding a qualitatively different layer of complexity to primary care case management. Keeping the algorithm simple was done purposefully to design an approach that could be easily replicated by care managers in any primary care practice whose EHR's custom reporting module allowed for generating custom patient profiles or variables (risk score) from within their own data. Although modest gains in predictive accuracy have been demonstrated for more complex predictive models using prescription profiles19 and past patterns of hospital or emergency department use, we assumed that many practices would not have access to these data nor to the predictive analytics necessary to apply them to new practice settings. Once a practice establishes a simple algorithm for automatically risk-stratifying patients for care management, then future gains in accuracy could be achieved as data links become available, by adding variables related to polypharmacy, hospital utilization, and a geo-coded neighborhood risk deprivation index.

Care management and coordination

High-risk patients were offered enrollment in the PCMHN program. The PCMHN care team included a nurse care manager, community health workers (CHW), a licensed clinical social worker (LCSW), a community support liaison and a physician. The patient intervention included: one clinic [End Page 205] visit with the PCP, four home visits with a CHW and behavioral health assessments provided by the LCSW. During each one to two-hour home visit, CHWs assessed vital signs, behavioral health, medication adherence, self-management skills, patient satisfaction, health goals and connected the patients to community support programs. Digital dashboards were developed which provided a snapshot view of patient health information to the care management team. This included demographic characteristics, recent vital signs and lab results, problem list, hospital visits, members of the care team and health maintenance (Figure 1). The care coordination team reviewed dashboards prior to and after home visits.

Community intervention led by support liaison and CHW included the following components. 1) Cultivate healthy lifestyle activities, chronic disease support groups, church health promotion, and connection of high-risk patients with these resources. 2) Teach patients realistic self-management of chronic disease in home, family and community contexts. 3) Maintain active relational connection with patient using community health workers as patient navigators and peer counselors. 4) Facilitate training to access health information through patient outreach and community education program outreach. 5) Identify community resources for psychosocial and logistical support. 6) Weekly care and outcomes optimization team meetings to review rapid-cycle feedback loop data. 7) Mobilize community resources in behavioral health specific to patient needs.

Statistical analysis

We combined demographic characteristics and health information to generate a list of high, medium, and low-risk patients. We conducted descriptive statistics to characterize each group. Frequency distribution was performed on insurance status and demographic factors. Based on prevalence, top-ranked conditions, dyads (combinations of two health conditions), and triads (combinations of three health conditions) illustrated overall health status of high-risk patients. All analyses were performed using SAS 9.3 (SAS Institute, Cary, NC). The Morehouse School of Medicine Institutional Review Board found this study to be primarily a quality improvement program, and therefore exempt from human subjects review.


There were 347 patients classified as high-risk out of the 3,360 member CHFC patient panel (Table 2). Average age was 59 years (SD 15). Most patients in the high-risk group were females (74%), resided in ZIP code 30331 (29%), and were enrolled in Medicare (43%). Compared with medium and low-risk patients, the high-risk subgroup was older, had more female, and more likely to be covered by Medicare. Hypertension was the most common physical health condition among high-risk patients (90%), followed by hyperlipidemia (62%), obesity (39%), and type II diabetes (34%). More than half of the high-risk group had depression (55%). Other relatively common behavioral conditions were alcohol abuse (12%), dementia (10%) and psychosis (9%).

Among the high-risk patient panel, the most common comorbidity dyad was hypertension and hyperlipidemia (204 patients), followed by depression and hypertension (172 patients) and depression and hyperlipidemia (113 patients). Among the triads, the most frequently occurring was depression, hypertension and hyperlipidemia (109 patients). [End Page 206]

Figure 1 - No description available
Click for larger view
View full resolution
Figure 1.

[End Page 207]

Click for larger view
View full resolution
Table 2.



We describe how a simplified risk stratification approach implemented in a primary care clinic that serves an underserved, medically and socially complex population can be used to implement a clinic-to-community multidisciplinary care management program. With such an approach, a care team can quickly react to different intensities of illness in their panel and proactively manage risk. This approach allows a multidisciplinary care team to target high-risk patients inside the walls of the clinic and the communities where they live. [End Page 208]

This approach is easily replicable and low-cost, requiring input that is easily derived from an EHR, which are now prevalent primary care settings.2225 In contrast, several existing risk-stratification instruments are not transparent and not necessarily affordable for the primary care setting. For example, Adjusted Clinical Groups (ACGs), which has an algorithm that incorporates disease patterns, pharmaceutical information, and claims data, requires licensed software and has considerable charge for end-users.26,27 Additionally, measurement of the population management, registry, and quality measures can be customized by the approach presented here and leveraged to meet requirements across a variety of payers, for meaningful use requirements, and towards National Committee for Quality Assurance (NCQA) status qualifications. This approach enables a practice to implement a population health management tool for their whole patient panel, rather than managing multiple sub-panels defined by specific payers.

These strategies are in the interest of patients, clinicians, and payers, but the implementation is best managed on a practice-wide level, rather than creating multiple programs and solutions for each subset of the practice patient panel divided by various payer segments. The fully-implemented Patient-Centered Medical Home (PCMH), as measured by NCQA certification, has already been demonstrated to have significant (although variable) benefits as measured by health outcomes improvement and financial return on investment (ROI).28 Addressing behavioral and social complexity through integrated care with behavioral health professionals, social workers, and community health workers, also adds value to this model.29,30

Clearly there is an opportunity to build on this simple risk stratification model to test and then potentially strengthen the predictive power of the risk stratification algorithm. Our most common chronic conditions (hypertension and hyperlipidemia) are important targets of secondary prevention, but may have limited impact on short-term outcomes of interest to payers, such as near-term costs, emergency department visits, and hospital bed-days. Focusing on the combination of heart failure, chronic lung disease and renal insufficiency, for example, would be a triad that would be expected to generate a greater one-year return on investment (ROI) than cardio/metabolic risk reduction in otherwise healthy patients. Similarly, not all behavioral health conditions carry the same risk of morbidity. The combination of schizophrenia and diabetes, for example, is a potent dyad which leads to increasing utilization with each added comorbid condition.21 Weighting the risk stratification by annual costs or inpatient bed-day rates associated with each disease (or each dyad or triad), for example, would generate a different "high-risk" cohort and one-year ROI. County-specific Medicare cost and utilization data related to specific diseases or dyads are now available on-line for generating these weighting factors, and are available for public use by primary care clinicians on-line from the Centers for Medicare & Medicaid Services (CMS).31


This approach has limitations. Payers' outcome data are still needed for better care management. This will ultimately require real-time data feeds from payers and/or hospitals to generate actionable information. Weighting of comorbidities by impact on near-term and longer-term utilization, costs, and outcomes, will be essential. While payer agreement is not assured, this approach will allow the primary care practice to negotiate based on the ability to manage outcomes, rather than simply submitting to multiple programs, interventions, and IT solutions imposed by multiple payers. [End Page 209]

Future work

Next steps will include a detailed evaluation of the current algorithm that takes into account cost, utilization, and patient and provider perceptions of the program. Additionally, we plan to incorporate more detailed patient home visit information and claims data for patients receiving services outside CHFC into a refined dashboard. An enhanced patient engagement dashboard will be designed to support patient self-management, enhance health literacy, and encourage shared decision-making. Additionally, a care coordination toolkit and clinical support services will be developed to extend care coordination/clinical dashboard support services to small practices throughout the region.


The risk stratification algorithm presented here, built upon internally available EHR and billing system data, provides a feasible option for beginning the process of engaging in population health management in a safety-net primary care practice. Implementing this algorithm as the foundation for a Patient Centered Medical Home and Neighborhood initiative located in a high-need, high-disparity neighborhood enabled our team to understand and respond to the nuances of population health in our underserved community.

Junjun Xu, Arletha Williams-Livingston, Anne Gaglioti, Calvin McAllister, and George Rust

JUNJUN XU is affiliated with the National Center for Primary Care at Morehouse School of Medicine. ARLETHA WILLIAMS-LIVINGSTON is affiliated with the Department of Family Medicine and the Department of Community Health and Preventative Medicine at Morehouse School of Medicine. ANNE GAGLIOTI is affiliated with the National Center for Primary Care and the Department of Family Medicine at Morehouse School of Medicine. CALVIN MCALLISTER is affiliated with the Department of Community Health at Morehouse School of Medicine. GEORGE RUST is affiliated with the Department of Behavioral Science and Social Medicine and the Center for Medicine and Public Health at Florida State University College of Medicine.

Junjun Xu, Morehouse School of Medicine, 720 Westview Drive SW, Atlanta, GA 30310.


This work was produced with support from United Health Foundation and Optum Group.


1. Bendix J. From quantity to quality: meeting the new demands of value-based care. Iselin, NJ: UM Medica, 2015. Available at:,0.
2. Schoen C, Osborn R, Huynh PT, Doty M, Peugh J, Zapert K. On the front lines of care: primary care doctors' office systems, experiences, and views in seven countries. Health Aff (Millwood). 2006 Nov–Dec;25(6):w555–71. PMid: 17102164
3. Audet AM, Doty MM, Peugh J, Shamasdin J, Zapert K, Schoenbaum S. Information technologies: when will they make it into physicians' black bags? MedGenMed. 2004 Dec 6;6(4):2. PMid: 15775829
4. Hussey PS, de Vries H, Romley J, Wang MC, Chen SS, Shekelle PG, McGlynn EA. (2009). A systematic review of health care efficiency measures. Health Serv Res. 2009 Jun;44(3):784–805. PMid: 19187184
5. Mandl KD, Kohane IS. Escaping the EHR trap — the future of health IT. N Engl J Med. 2012 Jun;366(24):2240–2. PMid: 22693995 [End Page 210]
6. Agency for Healthcare Research and Quality. Primary care transformation. Rockville, MD: Agency for Healthcare Research and Quality, 2015. Available at:
7. Stange KC, Nutting PA, Miller WL, Jaén CR, Crabtree BF, Flocke SA, Gill JM. Defining and measuring the patient-centered medical home. J Gen Intern Med. 2010 Jun; 25(6):601–12. PMid: 20467909
8. Morehouse School of Medicine Patient-Centered Medical Home and Neighborhood Brief April 2015.
9. Patient Centered Medical Home—An Evolving Approach to Primary Care Medicine. (n.d.). Accessed March 28, 2016, from
10. Peikes D, Genevro J, Scholle SH, Torda P. The patient-centered medical home: strategies to put patients at the center of primary care (AHRQ 11-0029). Rockville, MD: Agency for Health care Research and Quality, 2011 Feb. Available at:
11. Rich E, Lipson D, Libersky J, Parchman M. Coordinating care for adults with complex care needs in the patient-centered medical home: challenges and solutions (AHRQ 12-0010-EF). Rockville, MD: Agency for Health care Research and Quality, 2012 Jan. Available at:
12. Taylor EF, Lake T, Nysenbaum J, Peterson G, Meyers D. Coordinating care in the medical neighborhood: critical components and available mechanisms. White Paper (Prepared by Mathematica Policy Research under Contract No. HHSA290200900019I TO2). AHRQ Publication No. 11-0064. Rockville, MD: Agency for Health care Research and Quality. June 2011.
13. 30344 zip code detailed profile. Hinsdale, IL: Advameg, Inc, 2015. Available at:
14. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005 Nov;43(11):1130–9. PMid: 16224307
15. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998 Jan;36(1):8–27. PMid: 9431328
16. Shen C, Sambamoorthi U, Rust G. Co-occurring mental illness and health care utilization and expenditures in adults with obesity and chronic physical illness. Dis Manag. 2008 Jun;11(3):153–60. PMid: 18564027
17. Rees DC, Williams TN, Gladwin MT. Sickle-cell disease. Lancet. 2010 Dec 11;376(9757):2018–31. PMid: 21131035 [End Page 211]
18. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2015. CA Cancer J Clin. 2015 Jan–Feb;65(1):5–29. PMid: 25559415
19. Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res. 2003 Aug;38(4):1103–20. PMid: 12968819
20. Katon WJ, Young BA, Russo J, Lin EH, Ciechanowski P, Ludman EJ, Von Korff MR. (2013) Association of depression with increased risk of severe hypoglycemic episodes in patients with diabetes. Ann Fam Med. 2013 May–Jun; 11(3):245–50. PMid: 23690324
21. Shim RS, Druss BG, Zhang S, Kim G, Oderinde A, Shoyinka S, Rust G. Emergency department utilization among Medicaid beneficiaries with schizophrenia and diabetes: the consequences of increasing medical complexity. Schizophr Res. 2014 Feb; 152(2–3):490–7. PMid: 24380708
22. Tanner C, Gans D, White J, Nath R, Pohl J. Electronic health records and patient safety: co-occurrence of early EHR implementation with patient safety practices in primary care settings. App Clin Inform. 2015 Mar 11;6(1), 136–47. PMid: 25848419
23. Gordon S, Baier R, Gardner R. Primary care physicians' use of electronic health records in Rhode Island: 2009–2014. R I Med J. 2015 Oct;98(10):29–32. PMid: 26422543
24. Redd TK, Doberne JW, Lattin D, Yackel TR, Eriksson CO, Mohan V, Gold JA, Ash JS, Chiang MF. Variability in electronic health record usage and perceptions among specialty vs. primary care physicians. AMIA Annu Symp Proc. 2015 Nov 5;2015: 2053–62. PMid: 26958305
25. Jha AK, Ferris TG, Donelan K, Desroches C, Shields A, Rosenbaum S, Blumenthal D. How common are electronic health records in the United States? a summary of the evidence. Health Aff (Millwood). 2006 Nov–Dec;25(6):w496–507 PMid: 17035341
26. Huntley AL, Johnson R, Purdy S, Valderas JM, Salisbury C. Measures of multimorbidity and morbidity burden for use in primary care and community settings: a systematic review and guide. Ann Fam Med. 2012 Mar–Apr;10(2):134–41. PMid: 22412005
27. Haas LR, Takahashi PY, Shah ND, Stroebel RJ, Bernard ME, Finnie DM, Naessens JM. Risk-stratification methods for identifying patients for care coordination. Am J Manag Care. 2013 Sep;19(9):725–32. PMid: 24304255
28. National Committee for Quality Assurance. Evidence showing effectiveness of NCQA recognition. Washington, DC: National Committee for Quality Assurance, 2015 Jun. [End Page 212] Available at:
29. Davis MM, Balasubramanian BA, Cifuentes M, Hall J, Gunn R, Fernald D, Gilchrist E, Miller BF, DeGruy F 3rd, Cohen DJ. Clinician staffing, scheduling, and engagement strategies among primary care practices delivering integrated care. J Am Board Fam Med. 2015 Sep–Oct;28 Suppl:S32–40. PMid: 26359470
30. Valentijn PP, Ruwaard D, Vrijhoef HJM, de Bont A, Arends RY, Bruijnzeels MA. Collaboration processes and perceived effectiveness of integrated care projects in primary care: a longitudinal mixed-methods study. BMC Health Serv Res. 2015 Oct 9;15:463. PMid: 26450573
31. Centers for Medicare and Medicaid Services. Multiple chronic conditions. Baltimore, MD: Centers for Medicare and Medicaid Services, 2017 May. Available at: [End Page 213]

Additional Information

Print ISSN
Launched on MUSE
Open Access
Back To Top

This website uses cookies to ensure you get the best experience on our website. Without cookies your experience may not be seamless.